Loading ...Consistent backup with Linux Logical Volume Manager (LVM) snapshots
LVM is an implementation of a logical volume manager for the Linux kernel. The biggest advantage is that LVM provides the ability to make a snapshot of any logical volume.
In a production environment many users access the same file (file is open) or database. Suppose you start backup process when a file is open, you will not get correct or updated copy of file i.e. inconsistent backup (see accurate definition of inconsistent backup).
Read-only partition - to avoid inconsistent backup
You need to mount partition as read only, so that no one can make changes to file and make a backup:
# umount /home
# mount -o ro /home
# tar -cvf /dev/st0 /home
# umount /home
# mount -o rw /home
As you see, the draw back is service remains unavailable during backup time to end users as partition mounted as read-only. If you are using database then shutdown database server and make a backup.
Logical Volume Manager snapshot to avoid inconsistent backup
This solution will only work if you have created the partition with LVM. A snapshot volume is a special type of volume that presents all the data that was in the volume at the time the snapshot was created. This means you can back up that volume without having to worry about data being changed while the backup is going on, and you don't have to take the database volume offline while the backup is taking place.
# lvcreate -L1000M -s -n dbbackup /dev/ops/databases
Output:
lvcreate -- WARNING: the snapshot must be disabled if it gets full lvcreate -- INFO: using default snapshot chunk size of 64 KB for "/dev/ops/dbbackup" lvcreate -- doing automatic backup of "ops" lvcreate -- logical volume "/dev/ops/dbbackup" successfully created
Create a mount-point and mount the volume:
# mkdir /mnt/ops/dbbackup
# mount /dev/ops/dbbackup /mnt/ops/dbbackup
Output:
mount: block device /dev/ops/dbbackup is write-protected, mounting read-only
Do the backup
# tar -cf /dev/st0 /mnt/ops/dbbackup
Now remove it:
# umount /mnt/ops/dbbackup
# lvremove /dev/ops/databases
Please note that LVM snapshots cannot be used with non-LVM filesystems i.e. you need LVM partitions. You can also use third party commercial proprietary (see below for discussion) or GPL backup solutions/software.
Further reading
Want to stay up to date with the latest Linux tips, news and announcements? Subscribe to our free e-mail newsletter or RSS feed to get all updates.
You can Email this page to a friend.
You may also be interested in other helpful articles:
- Howto: Make consistent MySQL database backups using Solaris ZFS snapshots
- Perform backups for the Linux operating system
- How to MySQL backup and data recovery with mysql-zrm
- How to collection: Solaris ZFS Online demo or tutorials
- Dual network interface card to optimize Linux server backup process
Discussion on This Article:
Leave a Reply
We encourage your comments, and suggestions. But please stay on topic, be polite, and avoid spam. Thank you very much for stopping by our site!
By the way, “commercial or GPL” backup programs are not opposites. You don’t mean commercial here, you mean non-GPL’d backup programs or possibly you mean proprietary versus free software backup programs.
Many business redistribute GPL-covered software as part of business activity, hence GPL-covered software is commercial. Since GPL-covered software is commercial software, GPL-covered software is not the opposite of commercial software, GPL-covered software is a proper subset of commercial software.
See http://www.gnu.org/philosophy/words-to-avoid.html#Commercial and http://www.gnu.org/philosophy/selling.html for more on this common confusion of terms.
J.B. Nicholson-Owens,
Thanks for pointing out this issue.
Appreciate your post.
It should probably be pointed out that you still need to take the database offline while you make the snapshot. Snapshots don’t eliminate the need for downtime to get consistent backups, they just make it very short (a few seconds, instead of the full backup time).
I’ve worked with a number of people who read articles like this one, and came away with that misconception.
To echo what Gordon Messmer says above, LVM snapshots will not obviate the need to make applications that use the filesystem quiescent before making the snapshot. Otherwise, there would be no guarantee that the application that writes to the filesystem has consistent, workable data that it can recover from at the instant in which the snapshot was created.
Looks very similar to:
http://tldp.org/HOWTO/LVM-HOWTO/snapshots_backup.html
I have heard about lvm2 snapshots not being stable, and lot of people loosing data over it. What are your thoughts on that ?
From my own testing and according to other’s people articles you need to FLUSH TABLES WITH READ LOCK; on MySQL prior to making the snapshot, and release the lock right after the snapshot command have completed.
Note that normally, MySQL can recover from any interruption as long as its fsync() calls secure the data (That means the storage subsystems must not cache writes unless the memory is backed up by a battery, this is usually called WRITE-TROUGH caching on RAID subsystems). That was part of my testing too and I had nodes of a busy MySQL database of a few hundread GBs being power-cycled hundreads of times without a glitch. A single power-cycle on the same database with a WRITE-BACK cache destroyed the database (obviously a testing copy).
This is a limitation of LVM and I experienced issues with both LVM1 and LVM2, both RO and RW (LVM2 only) snapshots. Entreprise-grade hardware raids with snapshot features shouldn’t suffer from this issue.
I realize this is a very old post and so it’s possible my comment will never be seen… but I figure it’s worth a try.
I am using this technique to back up a mysql database that is currently relatively small 1.5gb on disk. Everything is working great… except once I have the snapshot mounted, and I try to do something like TAR the results for a backup… the tar is taking about 25 minutes the FIRST TIME… immediately after the snapshot was created. Any subsequent attempts to TAR will happen in about 45seconds.
Is this expected behavior? I am guesing it has something to do with how the snapshot is actually filling in the disk image on the first read… or something, and so the TAR is actually causing a create.
I’ve tried mounting read only to see if that speeds things up… but it doesn’t seem to.
Thoughts?