[ale] Hard Drive Failures

Fri Feb 22 14:32:57 EST 2013

Thanks for sharing your thoughts on backups. I hope I didn't give the
impression that all was lost when my hard drive appears to have failed. I
had everything backed up ahead of time but since I got it to boot one last
time and it prompted me to backup I figured I would back it up since it is
only data that I backup off my rigs. (Haven't started backing up any
configuration stuff but that's for another time when I am not working full
force on a master's degree)

But in the future plan to implement the suggestions I see here.

Regards,

Jonathan

On Fri, Feb 22, 2013 at 10:27 AM, Ron Frazier (ALE) <
atllinuxenthinfo at techstarship.com> wrote:

> Hi JD, and Jim K,
>
> Thanks very much for the explanations of backup methods and applications.
>  I've read each message, and will have to spend days or weeks researching
> and reimplementing my backup strategy to correct its deficiencies.
>  Unfortunately, I cannot do that today.  I have other fires to put out that
> are calling my name.  This is always true.  I'm putting these messages at
> the top of my email priority so I can explore them in much more detail
> later.
>
> My situation is compounded by the fact that I have Windows 7 or Vista or
> XP + Ubuntu or Mint + NTFS data partitions.  I need the backup to store
> EVERYTHING including disk partitioning data for all partitions.  I need the
> restore to restore EVERYTHING, so the PC boots up afterwards just the way
> it was, with all OS's and all data intact.
>
> I'm sure I'll have even more questions.  8-)
>
> Sincerely,
>
> Ron
>
>
>
> JD <jdp at algoloma.com> wrote:
>
> >Inline with lots-o-snipping ...
> >
> >On 02/21/2013 08:04 PM, Ron Frazier (ALE) wrote:
> >> Agree mostly.  Disagree slightly.
> >
> >Completely understandable.
> >
> >> What I will lose is a good amount of sleep and hassle.  I far and
> >away prefer
> >> to have image backups versus any other kind.  If the drive did puke,
> >and I
> >> didn't have a recent image, it would take me probably a week to
> >reinstall the
> >> os, reinstall all the apps, configure all the apps, install plugins
> >in the
> >> apps, configure all the plugins on all the apps, and do all the
> >system tweaks
> >> that I want to make the machine run the way I want.  It would
> >probably take
> >> me another few days to get all my email restored, and rebuild all my
> >filters
> >> to filter 10's of thousands of messages.
> >
> >If you do a backup correctly, it isn't a week to restore, but 15
> >minutes for a
> >small box.  If it is a VM, less.  These are not image-based backups
> >either.  To
> >me, after critical data, settings are the most important things to
> >backup AND
> >restore. Using current Linux backups, it is fairly simple to backup
> >everything
> >and restore it.  Plus, because it is not extremely hardware sensitive,
> >almost
> >any machine can be used for the restore.
> >
> >> On the other hand, let's say I had cloned the hdd last night.  I swap
> >in the
> >> cloned drive, and I'm literally back up and running within 15
> >minutes.  I
> >> just restore the latest data from my online backup, download any
> >recent
> >> email, and re do anything else I remember that occurred since the
> >last online
> >> backup up to 6 hours ago.
> >
> >I've cloned hard drives and still do. well, not really, but I do get a
> >compressed image that can be restored.
> >
> >The main issue with cloning is that it loses backup versioning. We end
> >up with a
> >mirror and it is HUGE, compared to what actually changed.
> >
> >rdiff-backup supports versioned backups.  The first, is a mirror.  From
> >that
> >point on, only changed files are moved over into the "mirror" area. Any
> >parts of
> >files that are replaced during that process are moved into an archived
> >area
> >permissions captured and gzipped. Notice, that I said parts of files.
> >The
> >latest backup is always a mirror, so restoring 1 file is just a cp.
> >File
> >permissions (and ACLs with an addon) are retained across systems.
> >Diffs are
> >retained in a highly efficient manner.  Depending on the change rate
> >and size of
> >your data, only 10-20% more storage is used for 30-60 days of versioned
> >backups
> >that required for the mirror. It is amazing.
> >
> >Basically, if a virus were to get onto the backup system through backup
> >processing, I'd have 30 to 60 days to realize it and would see the day
> >that the
> >file changed.
> >
> >Plus all my settings are safe.  Probably under 10 seconds to restore
> >~/.config/
> >
> >> Relatively simple and painless.  If I can use something like Spinrite
> >to
> >> recover the drive, even if it's just to clone it to a replacement
> >drive.
> >> Then I don't lose any data or have to do any reconfiguration.
> >
> >What if your clone has the virus you didn't realize that you got 3
> >weeks ago?
> >Or simply an important file that you've been working on off and on over
> >3 weeks
> >became corrupt?
> >
> >> The problem comes into play because I don't clone my drive every
> >night.  I
> >> don't like to leave the backup media attached, since a virus or
> >electrical
> >> problem could take it out.  So, I prefer to attach the backup media
> >only when
> >> I clone the drive.  Also, I have to reboot the machine and boot from
> >a CD to
> >> do the imaging, then reboot the machine to use it again.
> >
> >Backups need to be 100% automatic or they are not done.  I used to do
> >manual
> >backups. After about 18 months, that slowed to monthly manual backups,
> >then
> >annual .... which is next to worthless.  The data far outgrew my
> >ability to back
> >it up, then a RAID0 set had 1 HDD fail.  I lost 80% of my data, due to
> >my own
> >foolishness.  These days, I don't add storage without adding backup
> >storage too.
> >
> >> The end result is that my image backups don't get done too often,
> >> particularly with 4 PC's.  So, if I could find a way to automatically
> >clone
> >> each PC's hard drive every night, or at least every week, and
> >automatically
> >> detach and shut down the backup media when done, and reattach it when
> >the
> >> next clone job is due, then I could be in a position to really not
> >care too
> >> much if a drive fails.  If I could get snapshot versions like JD has
> >> mentioned elsewhere in this thread, so much the better.  My number
> >one goal
> >> for my backup is to enable me to restore a complete system, including
> >all
> >> data, all applications, and all settings, either within 15 minutes,
> >or at
> >> most within a few hours.
> >
> >For cloning PC OSes, check out partimage. It can write over the
> >network, but
> >because it does an image, it does need to be booted outside the normal
> >OS. A
> >500MB partition just for imaging could make a lot of sense, rather than
> >having a
> >USB drive that needs to be moved around.  Data should be backed up more
> >efficiently. IMHO. Images need to be minimal and only have a place for
> >MS-Windows. On Linux, it simply is not necessary.  librsync is
> >fantastic and
> >most Linux-based backup tools use it.
> >
> >30 minutes to image after booting a different OS
> > or
> >2 minutes to backup all the data while leaving the machine running.
> >You can pick.
> >
> >Backup tools have come a long way in the last 2 years.  Duplicity (and
> >the 10
> >GUIs built on top of it) have brought the "best practices" to a home
> >user.
> >
> >Bacula is impressive, but it is definitely an enterprise-class tool
> >with
> >enterprise-class complexity.
> >
> >BTW, I know that most of my backups work.  I've moved machines for
> >about 7
> >machines recently. Basically, I just did a backup, shutdown 1 box,
> >brought up
> >the other and restored.
> >
> >Isn't that how it should work?
> >_______________________________________________
> >Ale mailing list
> >Ale at ale.org
> >http://mail.ale.org/mailman/listinfo/ale
> >See JOBS, ANNOUNCE and SCHOOLS lists at
> >http://mail.ale.org/mailman/listinfo
>
>
> --
>
> Sent from my Android Acer A500 tablet with bluetooth keyboard and K-9 Mail.
> Please excuse my potential brevity if I'm typing on the touch screen.
>
> (PS - If you email me and don't get a quick response, you might want to
> call on the phone.  I get about 300 emails per day from alternate energy
> mailing lists and such.  I don't always see new email messages very
> quickly.)
>
> Ron Frazier
> 770-205-9422 (O)   Leave a message.
> linuxdude AT techstarship.com
>
>
> _______________________________________________
> Ale mailing list
> Ale at ale.org
> http://mail.ale.org/mailman/listinfo/ale
> See JOBS, ANNOUNCE and SCHOOLS lists at
> http://mail.ale.org/mailman/listinfo
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ale.org/pipermail/ale/attachments/20130222/81467171/attachment-0001.html>