How to backup and restore LXD containers

Procedure to backup and restore LXD containers

Backup LXD server configuration info such as profile name, storage, network using the lxd command. Next, we are going to use the lxc export command to backup running containers. Finally will see how to restore all containers and server config.

Step 1 – Gather information about LXD

As per the official docs we need the following stuff to backup:

1. Instances
2. Images
3. Network
4. Profile
5. Storage volumes (zfs/dir/btrfs/lvm)

Warning: You need LXD version 4.x or above for the export and dump feature to work. To find LXD version, run:
lxd version

List all instances, run:
# lxc list
To get information about current networking, storage pool, and profile, run:
# lxd init --dump
Save it to /backups/lxd/ dir:
# mkdir -pv /backups/lxd/
# lxd init --dump >/backups/lxd/lxd.config.$(date +'%m-%d-%Y')
# ls -l /backups/lxd/lxd.config.$(date +'%m-%d-%Y')
I am using ZFS as storage volumes so I am going to list it using the following commands (make sure zfs-utils installed using the apt command):
# zpool list
# zfs list

I installed lxd using the snap command ,and full data located at /var/snap/lxd/common/lxd/ directory:
# ls -l /var/snap/lxd/common/lxd/
We are going to back up this directory too. Remember backing up everything important. Each set up is a little different in terms of storage and networking. So you need to study all these components carefully.

Step 2 – Full backup of /var/snap/lxd/common/lxd/ directory

Let us use the rsync command:
# rsync --sparse -avrP \
/var/snap/lxd/common/lxd /backups/lxd/lxd-full-backup/
For remote backup server, run:
# rsync --sparse -avrP \
/var/snap/lxd/common/lxd user@server-ip:/backups/lxd/lxd-full-backup/
Where,

• --sparse : Turn sequences of nulls into sparse blocks. This is needed to speed up and I am using /var/snap/lxd/common/lxd/disks/lxdzfs.img. After first full backup we need to pass the--partial option. Hence, my command would be: rsync --partial -avrP \
  /var/snap/lxd/common/lxd /backups/lxd/lxd-full-backup/
• -a : Archive mode. Good for backups.
• -v : Verbose option.
• -r : Copy directories recursively
• -P : Show progress on screen
• /var/snap/lxd/common/lxd : Source directory
• /backups/lxd/lxd-full-backup/ : Destination backup directory.
• user@server-ip:/backups/lxd/lxd-full-backup/ : Same as above but remote backup server.

Step 3 – Backup all running instances with snapshots

I am going to export instances as backup tarballs using the following syntax:
lxc export {container} /path/to/{container}-backup-$(date +'%m-%d-%Y').tar.xz
## For zfs or btrfs storage --optimized-storage option will save disk space but less portable ##
lxc export {container} /path/to/{container}-backup-$(date +'%m-%d-%Y').tar.xz --optimized-storage
In this example, I am going to backup container named nginx-www as follows:
# lxc export nginx-www \
/backups/lxd/nginx-www-backup-$(date +'%m-%d-%Y').tar.xz \
--optimized-storage
Sample outputs:

Backup exported successfully! 

Here is bash for loop based example to backup all instances:

#!/bin/bash
# Basic shell script to backup required LXD parts ##
## Backup and restore LXD config ##
## Today's date ##
NOW=$(date +'%m-%d-%Y')
 
## Dump LXD server config ##
lxd init --dump > "/backups/lxd/lxd.config.${NOW}"
 
## Dump all instances list ##
lxc list > "/backups/lxd/lxd.instances.list.${NOW}"
 
## Make sure we know LXD version too ##
snap list lxd > "/backups/lxd/lxd-version.${NOW}"
 
## Backup all Instances
for i in $(lxc list -c n --format csv)
do
     echo "Making backup of ${i} ..."
     lxc export "${i}" "/backups/lxd/${i}-backup-$(date +'%m-%d-%Y').tar.xz" --optimized-storage
done

Feel free to modify the above script and run it daily from a cron job. Remember to backup everything offsite. Try s3 or NAS server.

Step 4 – Restore lxd instance

At this point we backed up everything. However, once in a while, we need to practice full restore and see if everything is working correctly or not. The following steps would help you to build confidence in backups. You will also learn how to restore LXD in case the server crashed or burned down during alien invasion. For demo purpose and to simulate server failure, I will delete my existing lxd server and all data/containers using the following syntax:
# lxc list
# snap remove lxd --purge
# snap list
## get rid of data stored in zfs too ##
# zpool destroy lxdzfs
# zfs list

Restore commands

Now, everything is gone. Nothing is left except our backups. So, let us start LXD restore procedure. First install lxd:

## Step 1. Find lxd version ##
# cat /backups/lxd/lxd-version.05-26-2020

## Step 2. Install that LXD version on a brand new server ##
# snap install lxd --channel 4.0/stable

## Step 3. Non-interactive configuration (restore lxd server config) ##
# cat /backups/lxd/lxd.config.05-26-2020 | lxd init --preseed
# lxc list

## Step 4. Restore instance ##
# ls -l /backups/lxd/*backup*xz
# lxc import /backups/lxd/nginx-www-backup-05-26-2020.tar.xz
# lxc list
# lxc start nginx-www

There you have it full LXD server and containers restored to original glory.

How do I only restore specific instance when rest of LXD running fine and dandy?

Simply jump to the step # 4. In other words, run:
# lxc import /backups/lxd/{container}-backup-${NOW}.xz
# lxc import /backups/lxd/mariadb-node-01-backup-01-13-2020.xz

Backup LXD containers using ‘lxc copy’ command

Another alternative is to copy all containers to another LXD server. See “How to copy/move/migrate LXD VM to another host on Linux” for more info. For example, say server2 is our backup LXD server:
## on $BACKUP_SERVER (server2) ##
BACKUP_SERVER=192.168.100.100
lxc config set core.https_address ${BACKUP_SERVER}:8443
lxc config set core.trust_password Same_super_PASSWORD_here
Next on server1, run:
BACKUP_SERVER=192.168.100.100
lxc remote add server2 BACKUP_SERVER
lxc remote list
## backup www-nginx to server2 using snapshots ##
lxc snapshot www-nginx
lxc info www-nginx
lxc copy www-nginx/snap0 server2:www-nginx-backup --verbose
## Restore www-nginx from server2 ##
lxc copy server2:www-nginx-backup www-nginx

Conclusion

In this in-depth tutorial, we learned about backing and restoring the LXD server, including configurations and all instances. Please note that each server OS, storage, and networking profiles are different. However, the overall steps remain the same, and you need to adopt and test those steps per your local setups. I would strongly suggest using a NAS server or cloud storage for all your backups. Writing shell script for automation purposes is left as an exercise to readers. See LXD docs here.