Debian / Ubuntu Linux Install and Configure Remote Filesystem Snapshot with rsnapshot Incremental Backup Utility

by on February 11, 2008 · 19 comments· LAST UPDATED January 6, 2010

in , ,

I'd like to configure my Debian box to backup two remote servers using rsnapshot software. It should make incremental snapshots of local and remote filesystems for any number of machines on 2nd hard disk located at /disk1 ( /dev/sdb2).

rsnapshot is perfect open source solution for making backups on local system. It supports both remote and local systems. From the man page:

rsnapshot saves much more disk space than you might imagine. The amount of space required is roughly the size of one full backup, plus a copy of each additional file that is changed. rsnapshot makes extensive use of hard links, so if the file doesn’t change, the next snapshot is simply a hard link to the exact same file. The following instructions are compatible with both Debian and Ubuntu Linux.

Required software / setup on local backup system

  1. rsnapshot
  2. rsync
  3. ssh client
  4. 2nd hard disk ( RAID array is suggested) - you can also use primary hard disk
  5. Password less login configured using ssh keys
  6. /disk1/backup - Backup directory
  7. /disk1/backup/server1 - Backup directory for remote server called server1
  8. /disk1/backup/server2 - Backup directory for remote server called server2
  9. /disk1/backup/localhost - Backup directory for local server

Required software on remote server

  1. OpenSSH sshd server
  2. Password less login configured using ssh keys

Step #1: Install rsync and rsnapshot software

Use apt-get command, enter:
$ sudo apt-get install rsync rsnapshot

Step #2: Configure passwordless login / public key based login

Type the following command
# ssh-keygen -t rsa
# scp .ssh/id_rsa.pub root@remotebox1.server.com:.ssh/authorized_keys2
# scp .ssh/id_rsa.pub root@remotebox2.server.com:.ssh/authorized_keys2

See how to configure RSA / DSA SSH public key based authentication.

Step #3: Configure rsnapshot utility

The configuration file is located at /etc/rsnapshot.conf. The configuration file requires tabs between elements and all drectories require a trailing slash. Just open config file using a text editor such as vi or gedit:
# vi /etc/rsnapshot.conf
OR
$ sudo vi /etc/rsnapshot.conf
Set snapshots root directory:

snapshot_root	/disk1/backup/

Note you must separate snapshot_root and /disk1/ by a [tab] key i.e. type snapshot_root hit [tab] key once and type /disk1/backup/. All snapshots will be stored under this root directory (/disk1/backup/).

Configure backup policy

You can make hourly, daily, weekly or monthly snapshots of local and remote systems. To make a snapshot every four hours (six times a day) and keep a second set, which are taken once a day, and stored for a seven days, enter:

interval    hourly  6
interval    daily   7

Feel free to adapt configuration as per your backup needs.

Specify local and remote backup directories

Find out comments that read as follows:

###############################
### BACKUP POINTS / SCRIPTS ###
###############################

You need to comment out / delete default backup directories. To make snapshots for /home/, /etc/, /webroot/ directories to /disk1/backup/localhost, enter:

backup	/home/	localhost/
backup	/etc/	localhost/
backup	/webroot/	localhost/

To backup remote server1 /home/, /etc/, /var/spool/mail/, /webroot/ directories to /disk1/backup/server1, enter:

backup	root@remotebox1.server.com:/home/	server1/
backup	root@remotebox1.server.com:/etc/	server1/
backup	root@remotebox1.server.com:/webroot/	server1/
backup	root@remotebox1.server.com:/var/spool/mail/	server1/
backup	root@remotebox2.server.com:/home/	server2/

Save and close the file.

Test your config file for errors

Type the following to test your configuration file for errors
# rsnapshot configtest
Output:

Syntax OK

You can also run rsnapshot in a test mode to display its action:
# rsnapshot -t hourly

Step #4: Run rsnapshot for first time

To run first time, enter:
# rsnapshot hourly

Step #5: Configure cron job

Edit /etc/cron.d/rsnapshot file to setup backup snapshot job. This is a sample cron file for rsnapshot. The values used correspond to the examples in /etc/rsnapshot.conf. There you can also set the backup points and many other things. To activate this cron file you have to uncomment the lines below.
Feel free to adapt it to your needs.

0 */4         * * *           root    /usr/bin/rsnapshot hourly
30 3          * * *           root    /usr/bin/rsnapshot daily
0  3          * * 1           root    /usr/bin/rsnapshot weekly
30 2          1 * *           root    /usr/bin/rsnapshot monthly

See crontab related faq for more information about cronjob under UNIX / Linux.

How do I exclude files from backup?

rsnapshot allows you to set the include and exclude parameters, if enabled, simply get passed directly to rsync. If you have multiple include/exclude patterns, put each one on a separate line. For example,
exclude_file /etc/rsnapshot.server1.conf
Append exclude file list to /etc/rsnapshot.server1.conf:
# vi /etc/rsnapshot.server1.conf
Exclude files matching pattern from backup:

var/lib/php/session/*
/var/spool/mail/nobody/*
cache/wp-cache-*.????
/var/logs/apache/access.log.*
/var/logs/apache/error.log.*
/linux-kernel/*
/tmp/cache/*
/var/lib/mysql/mysql.sock*
/tmp/php.socket-*
/tmp/*socket*

Save and close the file.

How do I backup remote MySQL database?

You can backup default database directory /var/lib/mysql. However, you can backup remote or local MySQL database with the following script:

#!/bin/sh
NOW=$(date +"%d-%m-%Y")
# set mysql login info
MUSER="MySQL-UserNAME"                # Username
MPASS="MySQL-SERVER-PASSWORD"   # Password
MHOST="MySQL-SERVER-IP-ADDRESS"  # Server Name
# guess binary names 
MYSQL="$(which mysql)"
MYSQLDUMP="$(which mysqldump)"
GZIP="$(which gzip)"
 
# get all db names 
DBS="$($MYSQL -u $MUSER -h $MHOST -p$MPASS -Bse 'show databases')"
for db in $DBS
do
 FILE=mysql-$db.$NOW-$(date +"%T").gz
 $MYSQLDUMP -u $MUSER -h $MHOST -p$MPASS $db | $GZIP -9 > $FILE
done

Now add following line to /etc/rsnapshot.conf file:
backup_script /root/scripts/mysql.backup.sh server1/mysql/

How do I restore backup?

You can simply copy back file using regular scp / rsync command. In this example, restore all *.html file to remote web server called www-03.example.com:

 
rsync -av *.html user@www-03.example.com:/home/httpd/example/html/

Further readings:

Updated for accuracy and mysql database section.

TwitterFacebookGoogle+PDF versionFound an error/typo on this page? Help us!

{ 19 comments… read them below or add one }

1 Vinícius February 11, 2008 at 1:13 pm

I’ll try to backup my VPS from my notebook.
Thanks for the info! :)

Reply

2 Scott Carlson February 11, 2008 at 7:54 pm

I do use this technique. Since you are running this from cron, the ssh-key you are using should probably be passwordless. In your .ssh/config on the the server you are going to backup use :
http://ipython.scipy.org/moin/Documentation
command=”/root/backupscript” ssh-rsa AAAAB…..

Then “/root/backupscript” can be setup like this:
# allow rsnapshot to backup /etc
if [ “$SSH_ORIGINAL_COMMAND” = “rsync –server –sender -logDtprRx –numeric-ids . /etc” ]
then
eval $SSH_ORIGINAL_COMMAND
exit 0
fi
echo no match $SSH_ORIGINAL_COMMAND >> /root/backupnomatch.log

#If the rsync server sends a slightly different command, modify backed on what is in /root/backupnomatch.log

Reply

3 Kevin Ernst July 19, 2010 at 7:05 pm

Scott, I believe that command=”/root/backupscript” ssh-rsa AAAAB….. should be added to ~/.ssh/authorized_keys instead of ~/.ssh/config.

Reply

4 Scott Carlson February 11, 2008 at 7:55 pm

Never mind that ipython link, cut and paste issue

Reply

5 Scott Carlson February 11, 2008 at 8:03 pm

Also, I use a script around rsnapshot that helps cut out a lot of my unneeded files from being backed-up. (Think cache files, or rapidly changing files like history in eclipse)

In rsnapshot.conf I have “exclude_file /var/backups/snapshot/excludefile.txt”

Then my script does a
find /home/user1/ -xdev -name .nobackup -print0 | xargs -0 -n1 dirname > $EXCLUDEFILE
cat /home/user1/.backupexcludes >> $EXCLUDEFILE

So this first uses the hdup style .nobackup flag files to ignore whole directories. This works pretty well, except for directories that get deleted like firefox cache directories. Those I put in the .backupexcludes file.

I do a similiar thing for the remote servers I backup.

Reply

6 nixCraft February 12, 2008 at 9:56 am

Scott,

Good suggestion. I’m adding additional content to this FAQ including your exclude_file and mysql backup issue.

Reply

7 Michael February 13, 2008 at 2:03 am

Another good tool to look at is rdiff-backup (http://www.nongnu.org/rdiff-backup/). It keeps a full copy of the current version of each file, with a diff of changed files. If you have a large file that changes a little at a time, you will use even less disk space than with rsnapshot.

Reply

8 Klas Wirholm February 27, 2008 at 4:09 pm

I’m doing a backup to an external usb drive who I mounted in fstab like this:

/dev/sdc1 /media/usb_backup auto rw,user,auto 0 0

I get an error that the disk is full when I do my second backup. It looks like rsnapshot thinks that the catalog where the partion is mounted is the messure of who big the backup can bee.The usb drive is 250 Gb and my main partion where I mount it is (root) is 39Gb.

How do I solve this?

Reply

9 nixCraft February 27, 2008 at 5:06 pm

Klas,

Are you sure your usb hard disk mounted at /media/usb_backup? Use df or mount command to verify the same. df will also give out information about used disk space.

Reply

10 Klas Wirholm February 27, 2008 at 6:33 pm

Looks like it functions know. I just changed the line in fstab to:
/dev/sdc1 /media/usb_backup auto rw,user,auto 0 0

So the problem occured when I had noauto, like this:

/dev/sdc1 /media/usb_backup auto rw,user,noauto 0 0

Can that bee true, that the disk whasnt mounted at al becourse of that (trying to learn al about linux)?

Thanks for your help!

Reply

11 uid0 August 14, 2008 at 7:39 pm

Ok, what if you configure your backup server as a raid5, how much different would this set up?

Furthermore, i want to be able to mount an external tape drive to the back up server, so to store to tape the back up media.

Thanks.

Reply

12 Riger November 13, 2008 at 9:16 pm

When I run:

rsnapshot hourly

I still have to enter the root password on the remote server (WHM/cPanel).

I checked and the remote server does have my public key and it is also authorized.

Where in the local script can I try to force rsnapshot/rsync to use the ssh key? Maybe that’s the issue?

I really hope someone can help thanks.

After manually logging in the rsnapshot does work, so the only sticking point (for me) is to get the automated ssh key login to be used.

Thanks for any help!

Reply

13 Riger November 13, 2008 at 9:54 pm

Also, when I was connected, something happened in the transmission and my network card disconnected.

Now whenever I run the:

rsnapshot hourly

command to test, I keep getting this error:

ERROR: Could not write lockfile +"/var/run/rsnapshot.pid"

There is no file by this name and I cannot find any online resource to fix this issue. So it seems there is a serious weakness in this – Meaning if the connection gets broken, they the system also breaks?

Sorry, I’d appreciate anyone who knows how to fix this. :)

THANK YOU!

Reply

14 justin December 20, 2008 at 9:04 pm

Riger:
how about:

sudo rsnapshot hourly

Reply

15 syaman June 20, 2010 at 1:06 am

Thank you for this great article! I also needed to uncomment the cmd_ssh directive /etc/rsnapshot.conf in order to backup a remote server to my backup machine.

Reply

16 Kevin Ernst July 19, 2010 at 7:02 pm

I believe Scott meant to say that:

command=”/root/backupscript” ssh-rsa AAAAB…..

should be added to ~/.ssh/authorized_keys instead of ~/.ssh/config. Anyone interested in using this method should also look into the PermitRootLogin forced-commands-only option in sshd_config.

Reply

17 syaman December 9, 2010 at 10:45 pm

Riger you will need to edit the lockfile line in /etc/rsnapshot.conf to point to a location to the user running rsnapshot

e.g.

lockfile /home/rsnapshotuser/rsnapshot.pid

Reply

18 gyre007 March 9, 2012 at 10:11 am

What I don’t understand is that I have the following backup jobs set up in rsnapshot configuration:

backup someuser@somehost:/etc/ etc/
backup someuser@somehost:/home/ home/
backup someuser@somehost:/var/somedir/ somedir/

why when I run the test run to see the commands which will be running is trying to CREATE a new directory EVERY TIME IT RUNS THE BACKUP JOB ?? #makesnosense

echo 14526 > /var/run/rsnapshot.pid
mkdir -m 0700 -p /var/backup/
mkdir -m 0755 -p /var/backup/daily.0/
/usr/bin/rsync -az –stats –delete –numeric-ids –relative \
–delete-excluded –exclude-from=/home/user1/excludefile.conf \
–rsh=/usr/bin/ssh someuser@somehost:/etc \
/var/backup/daily.0/etc/
mkdir -m 0755 -p /var/backup/daily.0/
/usr/bin/rsync -az –stats –delete –numeric-ids –relative \
–delete-excluded –exclude-from=/home/user1/excludefile.conf \
–rsh=/usr/bin/ssh someuser@somehost:/home \
/var/backup/daily.0/home/
mkdir -m 0755 -p /var/backup/daily.0/
/usr/bin/rsync -az –stats –delete –numeric-ids –relative \
–delete-excluded –exclude-from=/home/user1/excludefile.conf \
–rsh=/usr/bin/ssh \
someuser@somehost:/var/abinitio \
/var/backup/daily.0/abinitio/
touch /var/backup/daily.0/

Reply

19 gyre007 March 9, 2012 at 10:19 am

I had the following in mind:

/var/backup/daily.0/

Why would rsnapshot create the SAME directory several times ??

Reply

Leave a Comment

Tagged as: , , , , , , , , , , , , , , , , , ,

Previous Faq:

Next Faq: