How do I sync data between two Load balanced Linux / UNIX servers?

Posted on in Categories Backup, FreeBSD, Linux, RedHat/Fedora Linux, Solaris, Tips, UNIX last updated March 29, 2006

Load balancing is way to provide to handle the load of the large number of visitors particular website or network service receives. It helps provide redundancy to your website. There are several ways to accomplish load balancing:
* Use of Linux Virtual Server
* Use of layer 4 routers
* Round robin DNS with squid cache
* Proprietary clustering solution from vendor such as Microsoft or HP/IBM, Cisco, Nortel etc

However, one frequently asked question is how to keep your webpages (HTML/PHP/PERL scripts) synchronized with each server. For example if you create a new web page called viewnews.php on one www2 server, how does new page get copied over to the second server www1?

You can use rsync – a network file distribution/synchronization utility on Unixish (Linux, FreeBSD, Solaris etc) systems. It does not simply send new files; it updates all files by sending only changed files. This saves time.

Install the rsync

Debian Linux user type the following command:# apt-get install rsyncFedora Linux user, user type the following command:# yum install rsyncRed Hat Linux user, user type the following command:# up2date rsyncFreeBSD user, user type the following command:# pkg_add -r -v rsyncALTERNATIVELY, use FreeBSD ports collection:# cd /usr/ports/net/rsync
# make; make install; make clean

How do I use rsync command?

You do not need to run rsync as a service or daemon. For example, if you would like to sync’d between www1 and www2 servers, type the following command on www1 server:rsync -avrR --links --rsh=/usr/bin/ssh /var/htmlWhere,

  • -avrR : archive mode (a), verbose (v), recurse into directories (r), use relative path names (R)
  • –links : copy symlinks as symlinks
  • –rsh=/usr/bin/ssh : Use to specify the remote shell ssh to use (secure copy).
  • : WWW2 server IP address and path to synchronize to www1 server
  • /var/html : WWW1 server path

A sample shell script for same job

rsync -avrR --links --rsh=$SSH $MASTER:$DIR $LDIR

See also:

  • If you are going to use rsync command from cron job/shell script, generate ssh keys for password less login over ssh.
  • See official rsync site for more complex examples.

Posted by: Vivek Gite

The author is the creator of nixCraft and a seasoned sysadmin and a trainer for the Linux operating system/Unix shell scripting. He has worked with global clients and in various industries, including IT, education, defense and space research, and the nonprofit sector. Follow him on Twitter, Facebook, Google+.

4 comment

  1. So, and what will you do if your load balancer will point user to another server and dymanical script will change some data, which will be deleted due to syncronization…?

  2. -a is equivalent to -rlptgoD, so you can replace

    rsync -avrR –links –rsh=/usr/bin/ssh /var/html
    rsync -avR –rsh=/usr/bin/ssh /var/html

  3. Useless!!

    If you have a “load-balanced” server config, and have to sync them, this will not do the job.
    File 1 updated on server a
    File 2 updated on server b.

    one of the files get’s ‘corrected’ with the data from the other server (either file 1 and 2 come from server a and overwrite server b, or they come from server b and overwrite server a’s copies).

  4. paul_one – why useless? think sideways! here’s a couple of options:
    (1) Query your Load Balancer API and get the IP address of individual nodes. Then loop through the nodes and rsycn directly to each.
    (2) Perform the rsync as a “pull” from A and B from to the Master, rather than a push from Master through the load balancer to A and B.
    Sideways thinking. Problem solved. NOT useless. ๐Ÿ™‚

Comments are closed.