[ale] Linux GUI or CLI backup tool
jdp at algoloma.com
Fri Feb 18 08:18:02 EST 2011
On 02/18/2011 12:06 AM, Brian Pitts wrote:
> On 02/16/2011 11:37 AM, JD wrote:
>> In my experience, rsync has trouble with large files - anything over
>> 4GB. This isn't a file system limitation (though it used to be many yrs
>> ago). Perhaps someone else here knows how to make it work better?.
> I use it with files up to a couple hundred gigabytes in size without a
> problem. If you don't have enough disk space or your disk is slower than
> your network, '--inplace' can help.
Interesting. GigE backup network and plenty of storage (hundreds of GB
in excess) here. I stopped using rsync for 20GB blobs about 3 yrs ago
due to performance concerns. Backups were taking 45 minutes per blob
(sparse VMs). Did a little time optimization by switching to
rdiff-backup and got the backups to under 3 minutes for the same data.
I'm always looking for a better way to backup, especially blobs.
>> $ rsync -v -u -a --rsh=ssh --links --stats --progress --recursive
>> /source/dir srvname:backups/dir/
> There are some redundancies here. '-a' means '-rlptgoD', so you don't
> need '--links' or '--recursive'. Also, ssh is default so you don't need
> to specify '--rsh'.
ssh wasn't always the default and I don't want to assume it now. People
still use FTP, rsh and telnet after all. This is a case of getting
something working back in '95 when I was fairly new to UNIX and staying
with it all these years. ;) Definitely worth going through those old
scripts and cleaning up. Since I work across multiple platforms and some
are very old, I tend to worry about different defaults on different
systems (as with 'df -k') and writing very portable scripts that work
across them all. I've had to work on hardware that were many years out
of support with very old software. So the clearer command would be:
rsync --rsh=ssh -v -u -a --stats --progress /source/dir srvname:backups/dir/
I hoped that someone would comment on the command and arguments. Thank you!
It may be worth mentioning that using the ~/.ssh/config file to simplify
"srvname", use an alternative userid and run over non-default ports is a
big help too, at least for me.
More information about the Ale