How to move/migrate LXD VM to another host on Linux

How do I migrate LXD VMs/containers between two Linux nodes powered by Ubuntu Linux or Fedora Linux?

There are two methods to migrate LXD VM between two nodes. I am going to assume that you already setup and running LXD (see how to install LXD on a Fedora Linux). I recommend method # 2 for everyone. It is more elegant and support live migration too.

ADVERTISEMENTS

How to move/migrate LXD VM to another host on Linux

I am going to demonstrate two different methods in this tutorial.

Method #1: Use backup and restore over ssh

First, you need to backup /var/lib/lxd including storage pool. Next, you need copy backups to remote LXD instance in the /var/lib/lxd/ including storage pools. Say you have a setup as follows:

Fig.01: Our sample setup to move/migrate LXD VM to another host on Linux

Fig.01: Our sample setup

To see list of vms on server1, enter:
$ lxc list
Sample outputs:

+-----------------+---------+---------------------+------+------------+-----------+
|      NAME       |  STATE  |        IPV4         | IPV6 |    TYPE    | SNAPSHOTS |
+-----------------+---------+---------------------+------+------------+-----------+
| newsletter      | RUNNING | 10.105.28.45 (eth0) |      | PERSISTENT | 0         |
+-----------------+---------+---------------------+------+------------+-----------+
| www-vm          | RUNNING | 10.105.28.42 (eth0) |      | PERSISTENT | 0         |
+-----------------+---------+---------------------+------+------------+-----------+

I am going to assume that you are using DIR as backup storage pool.

Step 1 – Create a backup of www-vm on server1

Type the following tar command:
$ sudo tar -zcvf /root/www-vm.tar.gz /var/lib/lxd/storage-pools/default/containers/www-vm/

Step 2 – Copy www-vm-tar.gz from server1 to server2

Type the following scp command/rsync command:
$ sudo scp /root/www-vm.tar.gz root@server2:/root/
OR
$ sudo rsync -v /root/www-vm.tar.gz root@server2:/root/

Step 3 – Restore backup file named www-vm-tar.gz on server2

Type the following tar command to untar tarball named www-vm.tar.gz in /var/lib/lxd/storage-pools/default/containers/ directory:
$ sudo -i
# cd /var/lib/lxd/storage-pools/default/containers/
# tar -zxvf /root/www-vm-tar.gz

Finally, create a soft-link using ln command, run:
# cd /var/lib/lxd/containers/
# ln -s /var/lib/lxd/storage-pools/default/containers/www-vm/

Step 4 – Restore and import container on server2

LXD maintains a backup.yaml file in each container’s storage volume. Use this to recover or restore a given container, such as container configuration, attached devices and storage. This file can be processed by the following command:
# lxd import {containerNameHere}
# lxd import www-vm

Step 5 – Start imported container on server2

Type the following commands:
$ lxc list
$ lxc start www-vm
$ lxc list
$ lxc exec www-vm bash

Method #2: LXD VM container migration using LXD API and Simplestreams

Here is my sample setup to use LXD API and Simplestreams protocol for migration purpose:

Fig.01: Local to remote hosts and container migration using LXD API

Fig.01: Local to remote hosts and container migration using LXD API

Again, you need two server with LXD configured. Your local machine I will call server1 and a remote server I will call server2.

Step 1 – Configure a remote server named server2

Type the following command to enable a remote access via API:
$ lxc config set core.https_address 192.168.1.6:8443
Set the password for server2 lxd daemon:
$ lxc config set core.trust_password PASSWORDhere
Replace the PASSWORDhere with actual password. You need to use the same in step #2 below. You must open port 8443 using the ufw firewall on Ubuntu Linux, run:
$ sudo ufw allow from 192.168.1.5 to 192.168.1.6 port 8443 proto tcp comment 'Allow lxd client to talk to lxd-server'

Step 2 – Configure a local server named server1

Type the following command on server1 to add server2:
$ lxc remote add server2 192.168.1.6
Sample outputs:

Certificate fingerprint: f4fb0a34a61498d79726079bc...
ok (y/n)? y
Admin password for server2: PASSWORDhere
Client certificate stored at server:  server2

You can list your remotes and you will see “server2” listed as follows:
$ lxc remote list
Sample outputs:

+-----------------+------------------------------------------+---------------+--------+--------+
|      NAME       |                   URL                    |   PROTOCOL    | PUBLIC | STATIC |
+-----------------+------------------------------------------+---------------+--------+--------+
| server2         | https://192.168.1.6:8443                 | lxd           | NO     | NO     |
+-----------------+------------------------------------------+---------------+--------+--------+
| images          | https://images.linuxcontainers.org       | simplestreams | YES    | NO     |
+-----------------+------------------------------------------+---------------+--------+--------+
| local (default) | unix://                                  | lxd           | NO     | YES    |
+-----------------+------------------------------------------+---------------+--------+--------+
| ubuntu          | https://cloud-images.ubuntu.com/releases | simplestreams | YES    | YES    |
+-----------------+------------------------------------------+---------------+--------+--------+
| ubuntu-daily    | https://cloud-images.ubuntu.com/daily    | simplestreams | YES    | YES    |
+-----------------+------------------------------------------+---------------+--------+--------+

You must allow communication between two servers. Open the port 8443 on server1 as well:
$ sudo ufw allow from 192.168.1.6 to 192.168.1.5 port 8443 proto tcp comment 'Allow lxd server2 client to talk to server1 lxd-server'

Step 3 – Copying container named www-vm from server1 to server2

First create a snapshot by running the following command (first snapshot will be named as snap0 and so on). The following requires “www-vm” to be stopped first, but I am going to copy a snapshot instead and do it while the source container is running as follows:
$ lxc snapshot www-vm
$ lxc info www-vm
$ lxc copy www-vm/snap0 server2:www-vm --verbose

Sample outputs:

INFO[08-15|20:03:32] Connecting to a local LXD over a Unix socket 
INFO[08-15|20:03:32] Sending request to LXD                   etag= method=GET url=http://unix.socket/1.0
INFO[08-15|20:03:32] Connecting to a remote LXD over HTTPs 
INFO[08-15|20:03:32] Sending request to LXD                   etag= method=GET url=https://192.168.1.6:8443/1.0
INFO[08-15|20:03:33] Sending request to LXD                   etag= method=GET url=http://unix.socket/1.0/containers/www-vm/snapshots/snap0
INFO[08-15|20:03:33] Sending request to LXD                   etag= method=POST url=http://unix.socket/1.0/containers/www-vm/snapshots/snap0
INFO[08-15|20:03:33] Sending request to LXD                   etag= method=POST url=https://192.168.1.6:8443/1.0/containers
INFO[08-15|20:03:33] Sending request to LXD                   etag= method=GET url=https://192.168.1.6:8443/1.0/operations/f4df846e-d7ca-4c15-a429-6d1256180a84
Transferring container: www-vm: 341.40MB (56.87MB/s)

Step 4 – Start container named www-vm on server2

Type the following command on server1:
$ lxc list server2:
$ lxc start server2:www-vm
$ lxc list cbz02:
$ lxc exec server2:www-vm bash

Conclusion

And there you have it, you just learned how to move/migrate LXD VM to another host on Linux operating system. For more info see the following docs:

  • LXD 2.0 : Remote hosts and container migration [6/12]
  • man lxc
This entry is 10 of 17 in the LXD Tutorial series. Keep reading the rest of the series:
  1. Install LXD container hypervisor on Ubuntu 16.04 LTS
  2. How to install and setup LXC (Linux Container) on Fedora Linux 26
  3. Set up LXD container under KVM or Xen virtual machine
  4. List VM images in LXD (Linux Containers)
  5. Upgrade LXD containers powered by Ubuntu/Debian or CentOS Linux
  6. Auto start LXD containers at boot time in Linux
  7. Command to rename LXD / LXC container
  8. Run commands on Linux Container (LXD) instance at provision launch time
  9. Use LXD (Linux containers) in a shell script to create VM when the cloud instance launches
  10. Move/migrate LXD VM to another host on Linux
  11. Fedora install and set up LXD
  12. CentOS 7.x install and set up LXD server
  13. Install LXD pure-container hypervisor on Ubuntu 18.04 LTS
  14. Create snapshots with lxc command for LXD
  15. Set up and install LXD on CentOS/RHEL 8
  16. Ubuntu 20.04 LTS install and set up LXD
  17. Full backup and restore LXD containers
🐧 Get the latest tutorials on SysAdmin, Linux/Unix, Open Source/DevOps topics:
CategoryList of Unix and Linux commands
File Managementcat
FirewallCentOS 8 OpenSUSE RHEL 8 Ubuntu 16.04 Ubuntu 18.04 Ubuntu 20.04
Network Utilitiesdig host ip nmap
OpenVPNCentOS 7 CentOS 8 Debian 10 Debian 8/9 Ubuntu 18.04 Ubuntu 20.04
Package Managerapk apt
Processes Managementbg chroot cron disown fg jobs killall kill pidof pstree pwdx time
Searchinggrep whereis which
User Informationgroups id lastcomm last lid/libuser-lid logname members users whoami who w
WireGuard VPNCentOS 8 Debian 10 Firewall Ubuntu 20.04

ADVERTISEMENTS
1 comment… add one
  • Casper Aug 16, 2017 @ 16:45

    Would it not be better to store the VM’s on a DRBD device which would ensure that the VM’s would be in sync on all the servers which has this memory device.

    Much easier to deal with than having to use scp/rsync.

Leave a Reply

Your email address will not be published.

Use HTML <pre>...</pre>, <code>...</code> and <kbd>...</kbd> for code samples.