[ale] Linux GUI or CLI backup tool

JD jdp at algoloma.com
Fri Feb 18 08:18:02 EST 2011


On 02/18/2011 12:06 AM, Brian Pitts wrote:
> On 02/16/2011 11:37 AM, JD wrote:
>> In  my experience, rsync has trouble with large files - anything over
>> 4GB. This isn't a file system limitation (though it used to be many yrs
>> ago). Perhaps someone else here knows how to make it work better?.
> I use it with files up to a couple hundred gigabytes in size without a
> problem. If you don't have enough disk space or your disk is slower than
> your network, '--inplace' can help.
Interesting. GigE backup network and plenty of storage (hundreds of GB 
in excess) here. I stopped using rsync for 20GB blobs about 3 yrs ago 
due to performance concerns. Backups were taking 45 minutes per blob 
(sparse VMs). Did a little time optimization by switching to 
rdiff-backup and got the backups to under 3 minutes for the same data. 
I'm always looking for a better way to backup, especially blobs.
<snip>
>> $ rsync  -v -u -a --rsh=ssh   --links --stats --progress --recursive
>> /source/dir srvname:backups/dir/
> There are some redundancies here. '-a' means '-rlptgoD', so you don't
> need '--links' or '--recursive'. Also, ssh is default so you don't need
> to specify '--rsh'.
ssh wasn't always the default and I don't want to assume it now. People 
still use FTP, rsh and telnet after all. This is a case of getting 
something working back in '95 when I was fairly new to UNIX and staying 
with it all these years. ;) Definitely worth going through those old 
scripts and cleaning up. Since I work across multiple platforms and some 
are very old, I tend to worry about different defaults on different 
systems (as with 'df -k') and writing very portable scripts that work 
across them all. I've had to work on hardware that were many years out 
of support with very old software. So the clearer command would be:

rsync --rsh=ssh -v -u -a --stats --progress /source/dir srvname:backups/dir/

I hoped that someone would comment on the command and arguments. Thank you!
It may be worth mentioning that using the ~/.ssh/config file to simplify 
"srvname", use an alternative userid and run over non-default ports is a 
big help too, at least for me.



More information about the Ale mailing list