How do I sync data between two Load balanced Linux / UNIX servers?

by on March 29, 2006 · 4 comments· LAST UPDATED October 23, 2007

in , ,

Load balancing is way to provide to handle the load of the large number of visitors particular website or network service receives. It helps provide redundancy to your website. There are several ways to accomplish load balancing:
* Use of Linux Virtual Server
* Use of layer 4 routers
* Round robin DNS with squid cache
* Proprietary clustering solution from vendor such as Microsoft or HP/IBM, Cisco, Nortel etc

However, one frequently asked question is how to keep your webpages (HTML/PHP/PERL scripts) synchronized with each server. For example if you create a new web page called viewnews.php on one www2 server, how does new page get copied over to the second server www1?

You can use rsync - a network file distribution/synchronization utility on Unixish (Linux, FreeBSD, Solaris etc) systems. It does not simply send new files; it updates all files by sending only changed files. This saves time.

Install the rsync

Debian Linux user type the following command:# apt-get install rsyncFedora Linux user, user type the following command:# yum install rsyncRed Hat Linux user, user type the following command:# up2date rsyncFreeBSD user, user type the following command:# pkg_add -r -v rsyncALTERNATIVELY, use FreeBSD ports collection:# cd /usr/ports/net/rsync
# make; make install; make clean

How do I use rsync command?

You do not need to run rsync as a service or daemon. For example, if you would like to sync'd between www1 and www2 servers, type the following command on www1 server:rsync -avrR --links --rsh=/usr/bin/ssh 202.54.1.11:/var/html/ /var/htmlWhere,

  • -avrR : archive mode (a), verbose (v), recurse into directories (r), use relative path names (R)
  • --links : copy symlinks as symlinks
  • --rsh=/usr/bin/ssh : Use to specify the remote shell ssh to use (secure copy).
  • 202.54.1.11:/var/html/ : WWW2 server IP address and path to synchronize to www1 server
  • /var/html : WWW1 server path

A sample shell script for same job

#!/bin/bash
MASTER="master-server-ip"
DIR="/var/www/change-me"
LDIR="/local/dir"
SSH="/usr/bin/ssh"
rsync -avrR --links --rsh=$SSH $MASTER:$DIR $LDIR

See also:

  • If you are going to use rsync command from cron job/shell script, generate ssh keys for password less login over ssh.
  • See official rsync site for more complex examples.
TwitterFacebookGoogle+PDF versionFound an error/typo on this page? Help us!

{ 4 comments… read them below or add one }

1 Alexander September 2, 2008 at 11:42 am

So, and what will you do if your load balancer will point user to another server and dymanical script will change some data, which will be deleted due to syncronization…?

Reply

2 Fish March 18, 2010 at 4:04 pm

-a is equivalent to -rlptgoD, so you can replace

rsync -avrR –links –rsh=/usr/bin/ssh 202.54.1.11:/var/html/ /var/html
by
rsync -avR –rsh=/usr/bin/ssh 202.54.1.11:/var/html/ /var/html

Reply

3 paul_one June 28, 2011 at 4:47 pm

Useless!!

If you have a “load-balanced” server config, and have to sync them, this will not do the job.
File 1 updated on server a
File 2 updated on server b.

one of the files get’s ‘corrected’ with the data from the other server (either file 1 and 2 come from server a and overwrite server b, or they come from server b and overwrite server a’s copies).

Reply

4 regan November 29, 2013 at 1:36 am

paul_one – why useless? think sideways! here’s a couple of options:
(1) Query your Load Balancer API and get the IP address of individual nodes. Then loop through the nodes and rsycn directly to each.
(2) Perform the rsync as a “pull” from A and B from to the Master, rather than a push from Master through the load balancer to A and B.
Sideways thinking. Problem solved. NOT useless. :)

Reply

Leave a Comment

Tagged as: , ,

Previous post:

Next post: