[ale] Today's lesson: rdiff-backup restores

[Lightner, Jeff]

Big boys?   Do you know of a FOSS deduplication (block level) solution?

Backup to dedupe appliance can be done in various ways.   The two big ones are “inline” and “post-ingest”.   The former has higher resource requirements on the servers being backed up than the latter.   The latter doesn’t take any more resources than backup to tape generally.    You can also do 10 GB fibre on NFS/CIFS mounts or you can do OST over fibre SAN.

One of my great annoyances with Quantum DXi was that they made a “firmware” change a few years back that actually turned out to be full upgrade of the embedded Linux they were using (CentOS based I think) wherein they switched from post-ingest to inline.   It killed our backup window to the point I refused to even try using that new “firmware”.  They finally put us back on the old one.   They continually say they’ve made improvements to that newer “firmware” and have better hardware that makes inline work better.    Given this was a few years ago we had the issue it is possible they’re correct now but we don’t use them anymore.

I liked Data Domain but after EMC bought them we were no longer keen on them for non-technical reasons.    Since Dell bought EMC it will be interesting to see if the culture of Dell overrides EMC or vice-versa.

At the end of last year we got ExaGrid units and so far we’re quite happy with those.   We’re getting faster backups  over NFS (on 10 GB) than we do to our SAN attached tape library.

The “Vaulting” I spoke of is a setup that allows us to do copies of all backed up images from one storage unit to another based on the vaulting policy setup rather than having to do manual individual duplications of images.    The catalog that lets us search for backups shows both copies (until the primary expires from dedupe at which point we see only the duplicated image.   In a litigious society having access to old backups is important for most corporations.

From: ale-bounces at ale.org [mailto:ale-bounces at ale.org] On Behalf Of Jim Kinney
Sent: Tuesday, February 23, 2016 9:01 AM
To: Atlanta Linux Enthusiasts - Yes! We run Linux!
Subject: Re: [ale] Today's lesson: rdiff-backup restores


Most of our data is very static. More archival than not.

Excuse me while I snicker about the "big boys" "dedupe copy". :-)

No. Bacula doesn't have a dedupe process. Vaulting, IMHO, is nothing more than an intermediate copy - I've seen it used as a spinning rust hot backup with great success. Bacula can create that and later archive to slower media - tape, optical, MO, no paper (yet).

I think of dedupe at the block level. For my uses, the RAM cost is far higher than a cron job that scours user write space for copies of large, archived input data sets instead of links.
On Feb 23, 2016 8:49 AM, "Lightner, Jeff" <JLightner at dsservices.com<mailto:JLightner at dsservices.com>> wrote:
Monthly seems like a big risk.

There are companies that will pick up tapes you send and return them on expiration or on demand.   Here we offsite tapes weekly.

Recall:  http://www.recall.com/

IronMountain:  http://www.ironmountain.com/

Most of the deduplication appliance makers (e.g. ExaGrid, Quantum and Data Domain) also offer replication wherein you can send your backups to a unit in your data center then replicate that in the background to another unit at an offsite location.   Even if you use tapes for offsite you can backup everything to the dedupe units first then copy to tape from there.   In NetBackup they call that duplication from one storage unit to another, Vaulting.   One of my original questions in this thread was asking if Bacula or other OSS solution had something like this Vaulting.


From: ale-bounces at ale.org<mailto:ale-bounces at ale.org> [mailto:ale-bounces at ale.org<mailto:ale-bounces at ale.org>] On Behalf Of Jim Kinney
Sent: Tuesday, February 23, 2016 8:32 AM
To: Atlanta Linux Enthusiasts - Yes! We run Linux!
Subject: Re: [ale] Today's lesson: rdiff-backup restores


<sigh>

My backup rig is in the same rack with the machines it backs up. Politics and network limitations prevent relocating it elsewhere. So a monthly extraction of full backup tapes to my office across campus is my solution. I use a specific set of LTO6 tapes just for fulls with a rolling set for incr and diffs.
On Feb 23, 2016 8:26 AM, "Dow Hurst" <dphurst at uncg.edu<mailto:dphurst at uncg.edu>> wrote:
I have bacula setup on ~10 clients with daily incremental, weekly differential, and monthly full backups implemented.  Bacula works great and any individual file or subdirectory can be restored on demand if needed.  Anytime I've had to restore something, it just works.  Once I got the main configuration template for one client working then the rest of the clients were pretty straightforward. So, I know backups and restores work.  The problem we haven't dealt with yet and have plans to rectify is that the backup server is still in the same building as the clients being backed up...

Sincerely,
Dow
⚛Dow Hurst, Research Scientist
       340 Sullivan Science Bldg.
       Dept. of Chem. and Biochem.
       University of North Carolina at Greensboro
       PO Box 26170 Greensboro, NC 27402-6170

On Fri, Feb 19, 2016 at 7:42 PM, Alex Carver <agcarver+ale at acarver.net<mailto:agcarver+ale at acarver.net>> wrote:
The lesson is that restoring files via rdiff-backup actually works quite
well.  I'm very happy the restore was relatively painless.
_______________________________________________
Ale mailing list
Ale at ale.org<mailto:Ale at ale.org>
http://mail.ale.org/mailman/listinfo/ale
See JOBS, ANNOUNCE and SCHOOLS lists at
http://mail.ale.org/mailman/listinfo


_______________________________________________
Ale mailing list
Ale at ale.org<mailto:Ale at ale.org>
http://mail.ale.org/mailman/listinfo/ale
See JOBS, ANNOUNCE and SCHOOLS lists at
http://mail.ale.org/mailman/listinfo

_______________________________________________
Ale mailing list
Ale at ale.org<mailto:Ale at ale.org>
http://mail.ale.org/mailman/listinfo/ale
See JOBS, ANNOUNCE and SCHOOLS lists at
http://mail.ale.org/mailman/listinfo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ale.org/pipermail/ale/attachments/20160223/835ff267/attachment.html>