[ale] Hard Drive Failures

[Ron Frazier (ALE)]

Hi JD, and Jim K,

Thanks very much for the explanations of backup methods and applications.  I've read each message, and will have to spend days or weeks researching and reimplementing my backup strategy to correct its deficiencies.  Unfortunately, I cannot do that today.  I have other fires to put out that are calling my name.  This is always true.  I'm putting these messages at the top of my email priority so I can explore them in much more detail later.

My situation is compounded by the fact that I have Windows 7 or Vista or XP + Ubuntu or Mint + NTFS data partitions.  I need the backup to store EVERYTHING including disk partitioning data for all partitions.  I need the restore to restore EVERYTHING, so the PC boots up afterwards just the way it was, with all OS's and all data intact.

I'm sure I'll have even more questions.  8-)

Sincerely,

Ron



JD <jdp at algoloma.com> wrote:

>Inline with lots-o-snipping ...
>
>On 02/21/2013 08:04 PM, Ron Frazier (ALE) wrote:
>> Agree mostly.  Disagree slightly.
>
>Completely understandable.
>
>> What I will lose is a good amount of sleep and hassle.  I far and
>away prefer
>> to have image backups versus any other kind.  If the drive did puke,
>and I
>> didn't have a recent image, it would take me probably a week to
>reinstall the
>> os, reinstall all the apps, configure all the apps, install plugins
>in the
>> apps, configure all the plugins on all the apps, and do all the
>system tweaks
>> that I want to make the machine run the way I want.  It would
>probably take
>> me another few days to get all my email restored, and rebuild all my
>filters
>> to filter 10's of thousands of messages.
>
>If you do a backup correctly, it isn't a week to restore, but 15
>minutes for a
>small box.  If it is a VM, less.  These are not image-based backups
>either.  To
>me, after critical data, settings are the most important things to
>backup AND
>restore. Using current Linux backups, it is fairly simple to backup
>everything
>and restore it.  Plus, because it is not extremely hardware sensitive,
>almost
>any machine can be used for the restore.
>
>> On the other hand, let's say I had cloned the hdd last night.  I swap
>in the
>> cloned drive, and I'm literally back up and running within 15
>minutes.  I
>> just restore the latest data from my online backup, download any
>recent
>> email, and re do anything else I remember that occurred since the
>last online
>> backup up to 6 hours ago.
>
>I've cloned hard drives and still do. well, not really, but I do get a
>compressed image that can be restored.
>
>The main issue with cloning is that it loses backup versioning. We end
>up with a
>mirror and it is HUGE, compared to what actually changed.
>
>rdiff-backup supports versioned backups.  The first, is a mirror.  From
>that
>point on, only changed files are moved over into the "mirror" area. Any
>parts of
>files that are replaced during that process are moved into an archived
>area
>permissions captured and gzipped. Notice, that I said parts of files. 
>The
>latest backup is always a mirror, so restoring 1 file is just a cp. 
>File
>permissions (and ACLs with an addon) are retained across systems. 
>Diffs are
>retained in a highly efficient manner.  Depending on the change rate
>and size of
>your data, only 10-20% more storage is used for 30-60 days of versioned
>backups
>that required for the mirror. It is amazing.
>
>Basically, if a virus were to get onto the backup system through backup
>processing, I'd have 30 to 60 days to realize it and would see the day
>that the
>file changed.
>
>Plus all my settings are safe.  Probably under 10 seconds to restore
>~/.config/
>
>> Relatively simple and painless.  If I can use something like Spinrite
>to
>> recover the drive, even if it's just to clone it to a replacement
>drive.
>> Then I don't lose any data or have to do any reconfiguration.
>
>What if your clone has the virus you didn't realize that you got 3
>weeks ago?
>Or simply an important file that you've been working on off and on over
>3 weeks
>became corrupt?
>
>> The problem comes into play because I don't clone my drive every
>night.  I
>> don't like to leave the backup media attached, since a virus or
>electrical
>> problem could take it out.  So, I prefer to attach the backup media
>only when
>> I clone the drive.  Also, I have to reboot the machine and boot from
>a CD to
>> do the imaging, then reboot the machine to use it again.
>
>Backups need to be 100% automatic or they are not done.  I used to do
>manual
>backups. After about 18 months, that slowed to monthly manual backups,
>then
>annual .... which is next to worthless.  The data far outgrew my
>ability to back
>it up, then a RAID0 set had 1 HDD fail.  I lost 80% of my data, due to
>my own
>foolishness.  These days, I don't add storage without adding backup
>storage too.
>
>> The end result is that my image backups don't get done too often,
>> particularly with 4 PC's.  So, if I could find a way to automatically
>clone
>> each PC's hard drive every night, or at least every week, and
>automatically
>> detach and shut down the backup media when done, and reattach it when
>the
>> next clone job is due, then I could be in a position to really not
>care too
>> much if a drive fails.  If I could get snapshot versions like JD has
>> mentioned elsewhere in this thread, so much the better.  My number
>one goal
>> for my backup is to enable me to restore a complete system, including
>all
>> data, all applications, and all settings, either within 15 minutes,
>or at
>> most within a few hours.
>
>For cloning PC OSes, check out partimage. It can write over the
>network, but
>because it does an image, it does need to be booted outside the normal
>OS. A
>500MB partition just for imaging could make a lot of sense, rather than
>having a
>USB drive that needs to be moved around.  Data should be backed up more
>efficiently. IMHO. Images need to be minimal and only have a place for
>MS-Windows. On Linux, it simply is not necessary.  librsync is
>fantastic and
>most Linux-based backup tools use it.
>
>30 minutes to image after booting a different OS
> or
>2 minutes to backup all the data while leaving the machine running.
>You can pick.
>
>Backup tools have come a long way in the last 2 years.  Duplicity (and
>the 10
>GUIs built on top of it) have brought the "best practices" to a home
>user.
>
>Bacula is impressive, but it is definitely an enterprise-class tool
>with
>enterprise-class complexity.
>
>BTW, I know that most of my backups work.  I've moved machines for
>about 7
>machines recently. Basically, I just did a backup, shutdown 1 box,
>brought up
>the other and restored.
>
>Isn't that how it should work?
>_______________________________________________
>Ale mailing list
>Ale at ale.org
>http://mail.ale.org/mailman/listinfo/ale
>See JOBS, ANNOUNCE and SCHOOLS lists at
>http://mail.ale.org/mailman/listinfo


--

Sent from my Android Acer A500 tablet with bluetooth keyboard and K-9 Mail.
Please excuse my potential brevity if I'm typing on the touch screen.

(PS - If you email me and don't get a quick response, you might want to
call on the phone.  I get about 300 emails per day from alternate energy
mailing lists and such.  I don't always see new email messages very quickly.)

Ron Frazier
770-205-9422 (O)   Leave a message.
linuxdude AT techstarship.com