[ale] Duplicity very slow

[Edward Holcroft]

I tested this on my production box with all the identical pieces of kit. A
full rsync of the 1.5 TB or so of data on this server takes maybe 10 -11
hours. A full backup using Duplicity to the same USB drive on the same
server was less than half that - I gave up on 26% after some 16 hours. I'd
expect some added overhead for the encrypting and tarring that's going on,
but this is nuts. I have to be doign something wrong or nobody would be
using duplicity. Here's the command I use, which is pretty stock as far as
I know:

duplicity /data --exclude /data/ret --exclude /data/rsynclogs
file:///mnt/daily/MONDAY >> /data/rsynclogs/daily-`date +%Y%m%d_%H%M%S`.log

I also tried with --no-encyrption and dropped the logging with no
noticeable difference.

ed

On Tue, Apr 21, 2015 at 9:07 PM, James Sumners <james.sumners at gmail.com>
wrote:

>
> On Tue, Apr 21, 2015 at 8:00 PM, Edward Holcroft <eholcroft at mkainc.com>
> wrote:
>
>> Nice, thanks James. If I can get it to run faster, I may use this.
>>
>> ed
>>
>
> Er, the time estimate was a bit off. I forgot that the system I was
> quoting uses multiple partitions and the "--exclude-other-filesystems"
> switch skips those. Still, it doesn't take hours to run.
>
> I think your issue is down to the USB stack. Check the following:
>
> 1) You are really plugged into a USB 3 port
> 2) You are using a USB 3 cable
> 3) Your caddy really supports USB 3 speeds[1]
> 4) Your kernel supports USB 3
> 5) Your kernel's USB 3 support is enabled
> 6) Ditto for the BIOS
>
> [1] -- https://en.wikipedia.org/wiki/USB_3.0#Speed_and_compatibility
>
>
> --
> James Sumners
> http://james.sumners.info/ (technical profile)
> http://jrfom.com/ (personal site)
> http://haplo.bandcamp.com/ (band page)
>
> _______________________________________________
> Ale mailing list
> Ale at ale.org
> http://mail.ale.org/mailman/listinfo/ale
> See JOBS, ANNOUNCE and SCHOOLS lists at
> http://mail.ale.org/mailman/listinfo
>
>


-- 
Edward Holcroft | Madsen Kneppers & Associates Inc.
11695 Johns Creek Parkway, Suite 250 | Johns Creek, GA 30097
O (770) 446-9606 | M (770) 630-0949

-- 
MADSEN, KNEPPERS & ASSOCIATES USA, MKA Canada Inc. WARNING/CONFIDENTIALITY 
NOTICE: This message may be confidential and/or privileged. If you are not 
the intended recipient, please notify the sender immediately then delete it 
- you should not copy or use it for any purpose or disclose its content to 
any other person. Internet communications are not secure. You should scan 
this message and any attachments for viruses. Any unauthorized use or 
interception of this e-mail is illegal.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ale.org/pipermail/ale/attachments/20150422/ce9c5c9f/attachment.html>