≡ Menu

rsync command

How do I change my rsync command port number while making backup to remote server at backup1.example.com port 10253 (my ssh server runs on tcp port # 10253)? How do I change port number while using rsync command?
[click to continue…]

How do I securely transfer files from one UNIX / Linux server to another UNIX server using Windows or Linux desktop clients without using ftp client?
[click to continue…]

Rsync Show Progress Bar While Copying Files

I am a new Unix user. I am using rsync to upload and download a large size files. Is there a way to display the progress bar for file? How do I show progress while copying files using rsync locally or remotely on Linux or Unix like operating systems?
[click to continue…]

Q. How do I use rsync command to back up a home directory under Linux operating system?

A. rsync command easily backup your home directory to local secondary hard disk or remote server using ssh protocol. rsync is a software application for Unix systems which synchronizes files and directories from one location to another while minimizing data transfer using delta encoding when appropriate.

Task: Backup /home/tom to /mnt/usbpen

Assuming that USB pen or external USB hard disk is mounted at /mnt/usbpen, enter the following command to backup (sync) new files and changes to existing files but don't remove files in backup directory:
$ rsync -au /home/tom /mnt/usbpen
To delete files in backup directory that no longer exist in directory being backed up, enter:
$ rsync -au --delete /home/tom /mnt/usbpen

See our previous article about using rsync for transferring files under Linux or UNIX from local or remote network computer.

How to backup the remote files in Linux / UNIX

Q. How do I make remote backups under Linux? I’ve CentOS 5 Linux server located in remote data center and I’d like to backup it to local or another server?

A. Both Linux / UNIX come with handy tools to make secure remote backups. You can use tool called rsync for automating remote backups of your Linux, UNIX, Windows server, Mac OS X and BSD systems. rsync is a program with many more options and uses the rsync remote-update protocol to greatly speed up file transfers when the destination file is being updated.

Task: Copy files / backup files from remote Linux server

Let us say you would like to backup files from remote server called server.nixcraft.in and directory called /home/vivek to local directory called /backup, type the command as follows on local system:
$ rsync -avz -e ssh vivek@server.nixcraft.in:/home/vivek/ /backup
You need to supply password for vivek user.

Task: Exclude files from backup

You can also skip few files from backup. Let us say you don't want to backup all C source code file, enter:
$ rsync --exclude '*.cpp' -avz -e ssh vivek@server.nixcraft.in:/home/vivek/ /backup

Task: Automatic backup using a shell script

SSH always prompts for a password. To automate process via a shell script you need to remove password using SSH key i.e. generate passphraseless keys, enter (type at local system):
$ ssh-keygen -t dsa
When asked for to enter passphrase, just press [ENTER] key twice. Now copy public key to remote server:
$ scp ~/.ssh/id_dsa.pub vivek@server.nixcraft.in:.ssh/authorized_keys
Now you can login without a password. For more information see - howto setup SSH with DSA public key authentication and RSA key authentication for password less login.

Now create a simple shell script as follows:
$ vi backup.sh
Append code:
rsync --exclude '*.cpp' --exclude '*.log' -avz -e ssh vivek@server.nixcraft.in:/home/vivek/ /backup

Setup executable permission using chmod command:
$ chmod +x backup.sh
Use cron to command to backup remote server:
$ crontab -e
Make a backup everyday:
@daily /path/to/backup.sh
Save and close the file.

Q. I would like to use duplicity package to automatically encrypting and storing remote backups in a bandwidth efficient manner in our backup server. The server is already configured and I’d like to install the same under RedHat Enterprise Linux. How do I install duplicity w/o compiling from source code?

A. Duplicity is a piece of software which provides easy encrypted versioned remote backup of files requiring little of the remote server. It does this using GnuPG, tar, and rdiff. To transmit data it can use ssh/scp, local file access, rsync, ftp, and Amazon S3.

The best way to install it to use rpm files from DAG's repo. For example RHEL 4 64 bit version can be installed using RPM command:
# wget http://dag.wieers.com/rpm/packages/duplicity/duplicity-0.4.2-1.el4.rf.x86_64.rpm
# rpm -ivh duplicity-0.4.2-1.el4.rf.x86_64.rpm

In order to use duplicity you need to create a GPG key:
# gpg --gen-key
Next run duplicity as follows to make backups:
# duplicity --encrypt-key="YOUR-KEY" /home scp://user@backup.server.com/directory
You can verify backups with --verify option:
# duplicity --encrypt-key="YOUR-KEY" --verify scp://user@backup.server.com/directory /home

Finally, if you need to restore file use command as follows:
# mkdir -p /disk3/resored.file
# duplicity --encrypt-key="YOUR-KEY" scp://user@backup.server.com/directory /disk3/resored.file

Read the man page of duplicity and gpg for more information and command options.