How do you install and use rsync to synchronize files and directories from one location (or one server) to another location? - A common question asked by new sys admin.
rsync is a free software computer program for Unix and Linux like systems which synchronizes files and directories from one location to another while minimizing data transfer using delta encoding when appropriate. An important feature of rsync not found in most similar programs/protocols is that the mirroring takes place with only one transmission in each direction.
So what is unique about rsync?
It can perform differential uploads and downloads (synchronization) of files across the network, transferring only data that has changed. The rsync remote-update protocol allows rsync to transfer just the differences between two sets of files across the network connection.
How do I install rsync?
Use any one of the following commands to install rsync.
If you are using Debian or Ubuntu Linux, type the following command
# apt-get install rsync
OR
$ sudo apt-get install rsync
If you are using Red Hat Enterprise Linux (RHEL), type the following command
# up2date rsync
If you are using CentOS/Fedora Core Linux, type the following command
# yum install rsync
Always use rsync over ssh
Since rsync does not provide any security while transferring data it is recommended that you use rsync over ssh . This allows a secure remote connection. Now let us see some examples of rsync.
rsync command common options
- --delete : delete files that don't exist on sender (system)
- -v : Verbose (try -vv for more detailed information)
- -e "ssh options" : specify the ssh as remote shell
- -a : archive mode
- -r : recurse into directories
- -z : compress file data
Task : Copy file from a local computer to a remote server
Copy file from /www/backup.tar.gz to a remote server called openbsd.nixcraft.in
$ rsync -v -e ssh /www/backup.tar.gz jerry@openbsd.nixcraft.in:~Output:
Password: sent 19099 bytes received 36 bytes 1093.43 bytes/sec total size is 19014 speedup is 0.99
Please note that symbol ~ indicate the users home directory (/home/jerry).
Task : Copy file from a remote server to a local computer
Copy file /home/jerry/webroot.txt from a remote server openbsd.nixcraft.in to a local computer /tmp directory:
$ rsync -v -e ssh jerry@openbsd.nixcraft.in:~/webroot.txt /tmp
Password
Task: Synchronize a local directory with a remote directory
$ rsync -r -a -v -e "ssh -l jerry" --delete openbsd.nixcraft.in:/webroot/ /local/webroot
Task: Synchronize a remote directory with a local directory
$ rsync -r -a -v -e "ssh -l jerry" --delete /local/webroot openbsd.nixcraft.in:/webroot
Task: Synchronize a local directory with a remote rsync server
$ rsync -r -a -v --delete rsync://rsync.nixcraft.in/cvs /home/cvs
Task: Mirror a directory between my "old" and "new" web server/ftp
You can mirror a directory between my "old" (my.old.server.com) and "new" web server with the command (assuming that ssh keys are set for password less authentication)
$ rsync -zavrR --delete --links --rsh="ssh -l vivek" my.old.server.com:/home/lighttpd /home/lighttpd
Read related previous articles
- How do I sync data between two Load balanced Linux/UNIX servers?
- How do I sync data between two Load balanced Windows 2003 servers?
Other options - rdiff and rdiff-backup
There also exists a utility called rdiff, which uses the rsync algorithm to generate delta files Using rdiff. A utility called rdiff-backup has been created which is capable of maintaining a backup mirror of a file or directory over the network, on another server. rdiff-backup stores incremental rdiff deltas with the backup, with which it is possible to recreate any backup point. Next time I will write about these Utilities
rsync for Windows server/XP
Please note if you are using Windows, try any one of the program:
Further readings
=> Read rsync man page
=> Official rsync documentation
- Email this to a friend
- Printable version
- Rss Feed
- Last Updated: Aug/2/2007

{ 15 comments… read them below or add one }
I have RSYNC running to copy files from a mapped linux server (through a windows PC running cwrsync) to a remote linux server. Everything works fine, but the permissions and ownership changes. I added this line:
–chmod=u+rwx,g+rx,o+x
and that seems to work fine in ensuring I have chmod 0755 set for my folders and files. However, it keeps reverting my users:groups from myuser:nobody to 400:401.
What is the code that I add to rsync to have it:
a) Retain the existing ownerships, or
b) Force myuser:nobody for all files and folders?
Oops. Second question. I’d like for all directories to retain their original ownership and permissions (rather than forcing the chmod). The reason is because I’d like for most directories to be 0755, but need 0777 on some specific ones for a templating system and tiny_mc plug-in to work. How can i retain the original ownership and permissions? rather than having to go through and modify them one-by-one after the Rsync, or giving a blanket 0777 for everything? Regardless, I don’t know how to specify folder-specific privileges in rsync and would prefer to just have them remain the same as before the rsync.
<owever, it keeps reverting my users:groups from myuser:nobody to 400:401.
You need to add myuser:nobody to your local server.
<I don’t know how to specify folder-specific privileges in rsync and would prefer to just have them remain the same as before the rsync.
Make sure files are transferred in “archive” mode, which ensures that symbolic links, devices, attributes, permissions, ownerships, etc. are preserved in the transfer.
rsync -avr src/path /dest/path2
HiFriends,
Can you give a help about. I have some unix specific setup.
We have the following directories on p1cpe, and p2cpe:
/opt/Apps/StaticContent
/opt/Apps/html_templates
/opt/Apps/bea/user_projects/domains/NDC-Dev/lib
/opt/Apps/webapps
/opt/Apps/src
I need them to be available, and in sync to both systems. The originals should all be one p1cpe, so if you have to copy or do anything, those should be considered the source.
The idea is, if a developer makes changes to files on p1cpe:/opt/Apps/StaticContent, I want it also available on p2cpe, without them having to modify it twice.
All the work should be done as the user “webdude”. You should have sudo rights to webdude on p1cpe and p2cpe.
Couls you please help me on that.
Thanks,
Babu
I want to rsync 800Gb data over 4Mbps link. there are many folders which more than 100Gb, so it is keep failing rsync. i am using even –bwlimit also.
i need some script which look into folders and rsync one by one folder instead of one 100Gb folder. does it make sense?
any kind of script shell or php?
e.g. $cd mainfolder–
$ls
folder1
folder2
folder3
folder4
folder5
thanks
hi, i hope someone could help me. I need to synchronize files between two servers like every few minues, but i need to do it automatedly using crontab. My problem is I need to use SSH with a different port, so i need to use the ssh n the rsync command line. If i put the command line into cronjob, it will prompt for password rite? then it will erroneous. Please help how to solve, as in it will rsync successfully in cronjob.
You need to use password less login using ssh keys.
hmm.. i found the answer here
thanks alot
Hello;
I want to mirror our web server into a backup server. How can I use the rsync? I see –delete in the command line above and I dont know why? I am also afraid to run the rsync from the WRONG server putting useless files into our web server, so where do you run rsync from and what files do you copy to make a good mirror/backup server?
This is for in case the web server goes down we can bring this one up(back up) and use it until the other one is fixed.
I have setup cert on both servers so ssh-ing is no problem.
Thank You Very Much!
@ Mehdi,
I suggest you use rsnapshot tool, which uses rsync as backup tool. rsnapshot makes hourly, daily, weekly and monthly backup. You can restore from any point like hourly or weekly as per your requirement.
HTH
Thank You Very Much Vivek for quick response!
I will try that.
Regards;
mehdi
Isn’t the “r” redundant in the examples as it is implied by the “a” option? A look at the man page says that “a” is short for “rlptgoD”.
how to do automatic backup using rsync.
How to sync specific files (in this case *.dat) from multiple directories where directory name is different?
For example, I’d like to copy/sync all server1:/dir1/1*/*.dat files to corresponding server2:/dir2/2*/*.dat files (where 1* directory names are 101,102.. 199 and 2* are corresponding 201,202 .. 299 directories).
Something like (calling rsync from server2):
rsync -zrv server1:/dir1/1*/*.dat /dir2/
Any suggestions?
Try:
rsync -zrv --include="*.dat" server1:/dir1/1 /dir2/