[ale] rsync with big files (.5GB and above)

Greg Freemyer greg.freemyer at gmail.com
Mon Apr 30 15:41:55 EDT 2007


All,

I setup a dreamhost account like I think I mentioned before.  I've
been rsync'ing my encrypted backups there for a few weeks.

As part of my implementation process I started only sending small
files (via the --max-size option of rsync).

When I was working with files below .5 GB I had the occasional
time-out, but in general it worked most of the time.

I set my max to 2 GB about a week ago.  Since then I get a few
additional files copied each attempt, but I don't think I have sent
more than 2 files in any one rsync instance.

Does anyone know if this is a generic issue with rsync, or am I just
seeing more timeouts because my discrete rsync instances are running
longer and dreamhost is having issues.  (I've never run more than 3 or
4 hours without a timeout).

FYI: I have --timeout set to 5 minutes in my backup script, but I've
been manually running rsync during the day with timeouts as long as 2
hours.  Longer timeouts seem more reliable, but rsync still fails
after a couple of files.  And I think the timeout is per i/o, not per
file.

Thanks
Greg
-- 
Greg Freemyer
The Norcross Group
Forensics for the 21st Century



More information about the Ale mailing list