UNIX / Linux: HowTo Use unison File Synchronizer

by on August 16, 2010 · 8 comments· LAST UPDATED August 16, 2010

in

I know how to use rsync for Unix systems which synchronizes files and directories from one location to another while minimizing data transfer. However, I would like to synchronizing files between two directories, either on one computer, or between a computer and another server. How do I maintain the same version of files on multiple servers?

Unison is a file-synchronization tool for Unix and Windows.

It allows two replicas of a collection of files and directories to be stored on different hosts (or different disks on the same host), modified separately, and then brought up to date by propagating the changes in each replica to the other. This is useful for:

  1. Backups.
  2. Web server cluster.
  3. Sync home and office files.

Install unison

Type the following command under RHEL / CentOS Linux (make sure you turn on EPEL repo):
# yum install unison
Type the following command under Debian / Ubuntu Linux:
# apt-get update && apt-get install unison
Type the following command under FreeBSD:
# cd /usr/ports/net/unison/ && make install clean

How Do I Use unison?

In this example, sync /tmp/test1 to /tmp/test2 as follows:
# mkdir /tmp/test{1,2}
# cd /tmp/test1
# touch file{1,2,3}
# ls -l

Sample outputs:

total 12
-rw-r--r-- 1 root root 0 Aug 16 12:09 file1
-rw-r--r-- 1 root root 0 Aug 16 12:09 file2
-rw-r--r-- 1 root root 0 Aug 16 12:09 file3

Now, try to sync it:
# unison /tmp/test1 /tmp/test2
Sample outputs:

Contacting server...
Connected [//vivek-desktop//tmp/test1 -> //vivek-desktop//tmp/test2]
Looking for changes
Warning: No archive files were found for these roots, whose canonical names are:
	/tmp/test1
	/tmp/test2
This can happen either
because this is the first time you have synchronized these roots,
or because you have upgraded Unison to a new version with a different
archive format.
Update detection may take a while on this run if the replicas are
large.
Unison will assume that the 'last synchronized state' of both replicas
was completely empty.  This means that any files that are different
will be reported as conflicts, and any files that exist only on one
replica will be judged as new and propagated to the other replica.
If the two replicas are identical, then no changes will be reported.
If you see this message repeatedly, it may be because one of your machines
is getting its address from DHCP, which is causing its host name to change
between synchronizations.  See the documentation for the UNISONLOCALHOSTNAME
environment variable for advice on how to correct this.
Donations to the Unison project are gratefully accepted:
http://www.cis.upenn.edu/~bcpierce/unison
Press return to continue.[] Reconciling changes
test1          test2
file     ---->            file1  [f]
file     ---->            file2  [f]
file     ---->            file3  [f]
Proceed with propagating updates? [] y
Propagating updates
UNISON 2.27.57 started propagating changes at 12:11:18 on 16 Aug 2010
[BGN] Copying file1 from /tmp/test1 to /tmp/test2
[END] Copying file1
[BGN] Copying file2 from /tmp/test1 to /tmp/test2
[END] Copying file2
[BGN] Copying file3 from /tmp/test1 to /tmp/test2
[END] Copying file3
UNISON 2.27.57 finished propagating changes at 12:11:18 on 16 Aug 2010
Saving synchronizer state
Synchronization complete  (3 items transferred, 0 skipped, 0 failures)

The -batch (batch mode) option ask no questions at all, enter:
# rm /tmp/test1/file3
# echo 'foo' >> /tmp/test2/file2
# unison -batch /tmp/test1 /tmp/test2

Sample outputs:

Contacting server...
Connected [//vivek-desktop//tmp/test1 -> //vivek-desktop//tmp/test2]
Looking for changes
Reconciling changes
         <---- changed    file2
test1        : unchanged file     modified on 2010-08-16 at 12:11:18  size 0         rw-r--r--
test2        : changed file       modified on 2010-08-16 at 12:14:59  size 4         rw-r--r--
deleted  ---->            file3
test1        : deleted
test2        : unchanged file     modified on 2010-08-16 at 12:09:31  size 0         rw-r--r--
Propagating updates
UNISON 2.27.57 started propagating changes at 12:15:11 on 16 Aug 2010
[BGN] Updating file file2 from /tmp/test2 to /tmp/test1
[END] Updating file file2
[BGN] Deleting file3 from /tmp/test2
[END] Deleting file3
UNISON 2.27.57 finished propagating changes at 12:15:11 on 16 Aug 2010
Saving synchronizer state
Synchronization complete  (2 items transferred, 0 skipped, 0 failures)

Remote Server Synchronizer

First, make sure you use the same version on local and remote server. Use the following command to test that the local unison client can start and connect to the remote server:
# unison -testServer /tmp/test1 ssh://server1.cyberciti.com//tmp/test1
Sample outputs:

Contacting server...
Connected [//vivek-desktop//tmp/test1 -> //server1.cyberciti.com//tmp/test1]

To sync, enter:
# unison -batch /tmp/test1 ssh://server1.cyberciti.com//tmp/test1
Delete or add a new file from or to server1.cyberciti.com//tmp/test1 directory and try again:
# unison -batch /tmp/test1 ssh://server1.cyberciti.com//tmp/test1

Sample Shell Scripts

Create a sample shell script as follows (sync.dirs.sh):

#!/bin/bash
# set paths / dirs
_paths="/var/www/html/ \
/etc/ \
/home/vivek/ \
/projects/scripts/*.pl"
 
# binary file name
_unison=/usr/bin/unison
 
# server names 
# sync server1.cyberciti.com with rest of the server in cluster
_rserver="server2.cyberciti.com server3.cyberciti.com"
 
# sync it
for r in ${_rserver}
do
	for p in ${_paths}
	do
        	${_unison} -batch "${p}"  "ssh://${r}/${p}"
	done
done
 

Save and close the file. Setup a cronjob as follows:

*/30 * * * * /path/to/sync.dirs.sh &>/tmp/sync.dirs.sh.log

Make sure you setup ssh keys or use keychain to avoid the password prompt.

How Do I Call unison On Demand?

You need to use the incrond (inotify cron daemon) is a daemon which monitors filesystem events (such as add a new file, delete a file and so on) and executes commands or shell scripts. It’s use is generally similar to cron. In this example, call sync.dirs.sh whenever files uploaded or deleted from /var/www/html (see inotify FAQ for more info):
/var/www/html IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /path/to/sync.dirs.sh

References:

TwitterFacebookGoogle+PDF versionFound an error/typo on this page? Help us!

{ 8 comments… read them below or add one }

1 marc October 4, 2010 at 9:02 am

Hi,

I’ve looked at unison some time ago as a corporate solution to replace dropbox, it’s good stuff but not good enough to replace such a killer-app as dropbox.

Do you have any knowledge about an unison configuration to replace dropbox on a enterprise level? (share between groups, share with clients with an url, versioning,..)

Amazing blog/site/forum, congratulations.

Reply

2 Mohamed April 6, 2011 at 4:26 pm

Thanks for writing this ! It was very helpful for me.

Mohamed

Reply

3 Yoni January 29, 2013 at 2:12 pm

Hello,
Thanks for a great post.
I have a question regarding your use of incron.
It seems that incron doesn’t detect changes in subdirectories.
does your use-case don’t deal with subdirectories? and if it does, what do you do?

Thanks,
Yoni.

Reply

4 Rahul Janghel February 28, 2013 at 11:53 am

Thanks Vivek,

I was the stuff i was looking for.

regards.

Reply

5 Rahul Janghel February 28, 2013 at 12:17 pm

Hi Vivek,

after a small evaluation found that it is not fitting to my requirement.
I have two server which received approx 100 files in a minute sizing from few kbs to 10Mb. now the problem is, if it sync when a file is partly arrived on any of server, it will copy only that part. on next sync it will scape that same file.

Can you pls suggest what are the option available for this scenario.

Regards,
Rahul Janghel.

Reply

6 Jens-Konrad Preem June 26, 2013 at 8:30 am

I have a question about incrond and unison combination (oer even rsync or whatever).
When I have incrond daemon looking for read/write/create jobs in folder and it’ll start unison – which will then start syncing – are then some inbuilt “lock” systems in incrond to avoid it starting a massive amount of colliding synch jobs.
When unison moves files from remote server into watched folder wouldn’t incornd fire up additional unisons doing the same thing for each one?
Or is there something witty already put into your shell script there that I didn’t catch?

Reply

7 Thomas Lynch February 28, 2014 at 2:37 am

I would like to use unison, but it doesn’t actually do batch mode, still popping up long error messages in dialog boxes.

take for example:
mkdir a
touch a/fa1
touch a/fa2
mkdir b
unison -batch a b

This then pops up *two windows* on the screen! Apparently ignoring the batch option. One of the windows is a long long error messages that promises to return every time I use unison because we serve addresses on the machine via dhcp. The other window is some GUI interface which apparently has no purpose (as we ran in batch)

Thus unison is unusable in scripts, unless you want to close 50 windows manually, unusable with dhcp, and is high maintenance with new users who don’t understand all the messages.

Too bad -batch doesn’t actually do batch processing, that would make it a completely different program.

Reply

8 Thomas Lynch February 28, 2014 at 2:44 am

The same results of two windows popping up on the screen also happens with the example above with:

unison -batch -silent a b

Silent mode is far from silent, spitting out a hundred line message and popping windows up on gnome. Though after all the noise and promises of making more noise, it does unify the directories …

Reply

Leave a Comment

Tagged as: , , , , , , , , , , , ,

Previous Faq:

Next Faq: