How do I sync data between two Load balanced Linux / UNIX servers?

last updated in Categories Backup, FreeBSD, Linux, RedHat/Fedora Linux, Solaris, Tips, UNIX

Load balancing is way to provide to handle the load of the large number of visitors particular website or network service receives. It helps provide redundancy to your website. There are several ways to accomplish load balancing:
* Use of Linux Virtual Server
* Use of layer 4 routers
* Round robin DNS with squid cache
* Proprietary clustering solution from vendor such as Microsoft or HP/IBM, Cisco, Nortel etc

However, one frequently asked question is how to keep your webpages (HTML/PHP/PERL scripts) synchronized with each server. For example if you create a new web page called viewnews.php on one www2 server, how does new page get copied over to the second server www1?

You can use rsync – a network file distribution/synchronization utility on Unixish (Linux, FreeBSD, Solaris etc) systems. It does not simply send new files; it updates all files by sending only changed files. This saves time.

Install the rsync

Debian Linux user type the following command:# apt-get install rsyncFedora Linux user, user type the following command:# yum install rsyncRed Hat Linux user, user type the following command:# up2date rsyncFreeBSD user, user type the following command:# pkg_add -r -v rsyncALTERNATIVELY, use FreeBSD ports collection:# cd /usr/ports/net/rsync
# make; make install; make clean

How do I use rsync command?

You do not need to run rsync as a service or daemon. For example, if you would like to sync’d between www1 and www2 servers, type the following command on www1 server:rsync -avrR --links --rsh=/usr/bin/ssh /var/htmlWhere,

  • -avrR : archive mode (a), verbose (v), recurse into directories (r), use relative path names (R)
  • –links : copy symlinks as symlinks
  • –rsh=/usr/bin/ssh : Use to specify the remote shell ssh to use (secure copy).
  • : WWW2 server IP address and path to synchronize to www1 server
  • /var/html : WWW1 server path

A sample shell script for same job

rsync -avrR --links --rsh=$SSH $MASTER:$DIR $LDIR

See also:

  • If you are going to use rsync command from cron job/shell script, generate ssh keys for password less login over ssh.
  • See official rsync site for more complex examples.