[ale] File server syncronization

[Brian Pitts]

On 01/27/2010 03:42 PM, David Ritchie wrote:
> ALEers...
> 
> Looking trying to synchronize two file systems on an ongoing basis
> (i.e. a distributed file system). Just curious
> about your thoughts about things like rsync vs. Unison  or other tools
> you have seen...  Need to have files migrate over
> semi realtime (i.e. I drop a file in a directory, it should auto
> magically appear on a local directory on the remote machine - and
> there could be both data files and executable transferred)... Thoughts?

Unison is a simple way to do it, if a frequent cronjob qualifies as semi
realtime.

unison -batch /directory/to/sync ssh://remotehostname//directory/to/sync

You would need to run it once first without batch to build the archive
file (where unison keeps info on the current state of files in the path
for comparison to later) and sort out any differences. After that it
will be very smart about making changes. See below.

--------------------------------------------------------------------------------

bdp at lime:~$ unison /home/bdp/bow-20020213 /home/bdp/bow2
Contacting server...
Looking for changes
Warning: No archive files were found for these roots.  This can happen
either
because this is the first time you have synchronized these roots,
or because you have upgraded Unison to a new version with a different
archive format.

Update detection may take a while on this run if the replicas are
large.

Unison will assume that the 'last synchronized state' of both replicas
was completely empty.  This means that any files that are different
will be reported as conflicts, and any files that exist only on one
replica will be judged as new and propagated to the other replica.
If the two replicas are identical, then no changes will be reported.
Press return to continue.[<spc>]    HACKING
  HACKING
Reconciling changes

bow-20020213   bow2
file     <==== file       README  [] <

Proceed with propagating updates? [] y
Propagating updates


UNISON started propagating changes at 15:27:36 on 02 Nov 2009
[BGN] Updating file README
  from /home/bdp/bow2
  to /home/bdp/bow-20020213
[END] Updating file README
UNISON finished propagating changes at 15:27:36 on 02 Nov 2009


Saving synchronizer state
Synchronization complete  (1 item transferred, 0 skipped, 0 failures)

--------------------------------------------------------------------------------

bdp at lime:~$ touch bow2/foo
bdp at lime:~$ rm bow-20020213/debian/README.Debian
bdp at lime:~$ echo "Changed" >> bow-20020213/ChangeLog
bdp at lime:~$ echo "Same" >> bow-20020213/README
bdp at lime:~$ echo "Same" >> bow2/README
bdp at lime:~$ echo "These are" >> bow2/Makefile
bdp at lime:~$ echo "Different" >> bow-20020213/Makefile

--------------------------------------------------------------------------------

bdp at lime:~$ unison -batch /home/bdp/bow-20020213 /home/bdp/bow2
Contacting server...
Looking for changes
  HACKING
Reconciling changes

bow-20020213   bow2
changed  <-?-> changed    Makefile
bow-20020213 : changed file       modified on 2009-11-02 at 15:30:29
size 12850     rw-r--r--
bow2         : changed file       modified on 2009-11-02 at 15:30:17
size 12850     rw-r--r--
changed  ---->            ChangeLog
bow-20020213 : changed file       modified on 2009-11-02 at 15:29:01
size 358016    rw-r--r--
bow2         : unchanged file     modified on 2002-02-13 at 17:55:30
size 358008    rw-r--r--
deleted  ---->            debian/README.Debian
bow-20020213 : deleted
bow2         : unchanged file     modified on 2009-11-02 at  3:17:04
size 1161      rw-r--r--
         <---- new file   foo
bow-20020213 : absent
bow2         : new file           modified on 2009-11-02 at 15:28:05
size 0         rw-r--r--
Propagating updates


UNISON started propagating changes at 15:30:59 on 02 Nov 2009
[CONFLICT] Skipping Makefile
[BGN] Updating file ChangeLog
  from /home/bdp/bow-20020213
  to /home/bdp/bow2
[END] Updating file ChangeLog
[BGN] Copying foo
  from /home/bdp/bow2
  to /home/bdp/bow-20020213
[END] Copying foo
[BGN] Deleting debian/README.Debian
  from /home/bdp/bow2
[END] Deleting debian/README.Debian
UNISON finished propagating changes at 15:30:59 on 02 Nov 2009


Saving synchronizer state
Synchronization complete  (3 items transferred, 1 skipped, 0 failures)
  skipped: Makefile

-- 
All the best,
Brian Pitts