UNIX / Linux: HowTo Use unison File Synchronizer

I know how to use rsync for Unix systems which synchronizes files and directories from one location to another while minimizing data transfer. However, I would like to synchronizing files between two directories, either on one computer, or between a computer and another server. How do I maintain the same version of files on multiple servers?

Unison is a file-synchronization tool for Unix and Windows.

It allows two replicas of a collection of files and directories to be stored on different hosts (or different disks on the same host), modified separately, and then brought up to date by propagating the changes in each replica to the other. This is useful for:

1. Backups.
2. Web server cluster.
3. Sync home and office files.

Install unison

Type the following command under RHEL / CentOS Linux (make sure you turn on EPEL repo):
# yum install unison
Type the following command under Debian / Ubuntu Linux:
# apt-get update && apt-get install unison
Type the following command under FreeBSD:
# cd /usr/ports/net/unison/ && make install clean

How Do I Use unison?

In this example, sync /tmp/test1 to /tmp/test2 as follows:
# mkdir /tmp/test{1,2}
# cd /tmp/test1
# touch file{1,2,3}
# ls -l
Sample outputs:

total 12
-rw-r--r-- 1 root root 0 Aug 16 12:09 file1
-rw-r--r-- 1 root root 0 Aug 16 12:09 file2
-rw-r--r-- 1 root root 0 Aug 16 12:09 file3

Now, try to sync it:
# unison /tmp/test1 /tmp/test2
Sample outputs:

Contacting server...
Connected [//vivek-desktop//tmp/test1 -> //vivek-desktop//tmp/test2]
Looking for changes
Warning: No archive files were found for these roots, whose canonical names are:
	/tmp/test1
	/tmp/test2
This can happen either
because this is the first time you have synchronized these roots, 
or because you have upgraded Unison to a new version with a different
archive format.  

Update detection may take a while on this run if the replicas are 
large.

Unison will assume that the 'last synchronized state' of both replicas
was completely empty.  This means that any files that are different
will be reported as conflicts, and any files that exist only on one
replica will be judged as new and propagated to the other replica.
If the two replicas are identical, then no changes will be reported.

If you see this message repeatedly, it may be because one of your machines
is getting its address from DHCP, which is causing its host name to change
between synchronizations.  See the documentation for the UNISONLOCALHOSTNAME
environment variable for advice on how to correct this.

Donations to the Unison project are gratefully accepted: 
http://www.cis.upenn.edu/~bcpierce/unison

Press return to continue.[] Reconciling changes

test1          test2              
file     ---->            file1  [f] 
file     ---->            file2  [f] 
file     ---->            file3  [f] 

Proceed with propagating updates? [] y
Propagating updates


UNISON 2.27.57 started propagating changes at 12:11:18 on 16 Aug 2010
[BGN] Copying file1 from /tmp/test1 to /tmp/test2
[END] Copying file1
[BGN] Copying file2 from /tmp/test1 to /tmp/test2
[END] Copying file2
[BGN] Copying file3 from /tmp/test1 to /tmp/test2
[END] Copying file3
UNISON 2.27.57 finished propagating changes at 12:11:18 on 16 Aug 2010


Saving synchronizer state
Synchronization complete  (3 items transferred, 0 skipped, 0 failures)

The -batch (batch mode) option ask no questions at all, enter:
# rm /tmp/test1/file3
# echo 'foo' >> /tmp/test2/file2
# unison -batch /tmp/test1 /tmp/test2
Sample outputs:

Contacting server...
Connected [//vivek-desktop//tmp/test1 -> //vivek-desktop//tmp/test2]
Looking for changes
Reconciling changes
         <---- changed    file2  
test1        : unchanged file     modified on 2010-08-16 at 12:11:18  size 0         rw-r--r--
test2        : changed file       modified on 2010-08-16 at 12:14:59  size 4         rw-r--r--
deleted  ---->            file3  
test1        : deleted
test2        : unchanged file     modified on 2010-08-16 at 12:09:31  size 0         rw-r--r--
Propagating updates
UNISON 2.27.57 started propagating changes at 12:15:11 on 16 Aug 2010
[BGN] Updating file file2 from /tmp/test2 to /tmp/test1
[END] Updating file file2
[BGN] Deleting file3 from /tmp/test2
[END] Deleting file3
UNISON 2.27.57 finished propagating changes at 12:15:11 on 16 Aug 2010
Saving synchronizer state
Synchronization complete  (2 items transferred, 0 skipped, 0 failures)

Remote Server Synchronizer

First, make sure you use the same version on local and remote server. Use the following command to test that the local unison client can start and connect to the remote server:
# unison -testServer /tmp/test1 ssh://server1.cyberciti.com//tmp/test1
Sample outputs:

Contacting server...
Connected [//vivek-desktop//tmp/test1 -> //server1.cyberciti.com//tmp/test1]

To sync, enter:
# unison -batch /tmp/test1 ssh://server1.cyberciti.com//tmp/test1
Delete or add a new file from or to server1.cyberciti.com//tmp/test1 directory and try again:
# unison -batch /tmp/test1 ssh://server1.cyberciti.com//tmp/test1

Sample Shell Scripts

Create a sample shell script as follows (sync.dirs.sh):

#!/bin/bash
# set paths / dirs
_paths="/var/www/html/ \
/etc/ \
/home/vivek/ \
/projects/scripts/*.pl"
 
# binary file name
_unison=/usr/bin/unison
 
# server names 
# sync server1.cyberciti.com with rest of the server in cluster
_rserver="server2.cyberciti.com server3.cyberciti.com"
 
# sync it
for r in ${_rserver}
do
	for p in ${_paths}
	do
        	${_unison} -batch "${p}"  "ssh://${r}/${p}"
	done
done

Save and close the file. Setup a cronjob as follows:

*/30 * * * * /path/to/sync.dirs.sh &>/tmp/sync.dirs.sh.log

Make sure you setup ssh keys or use keychain to avoid the password prompt.

How Do I Call unison On Demand?

You need to use the incrond (inotify cron daemon) is a daemon which monitors filesystem events (such as add a new file, delete a file and so on) and executes commands or shell scripts. It’s use is generally similar to cron. In this example, call sync.dirs.sh whenever files uploaded or deleted from /var/www/html (see inotify FAQ for more info):
/var/www/html IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /path/to/sync.dirs.sh

References:

• unison home page