How to move/migrate LXD VM to another host on Linux

How do I migrate LXD VMs/containers between two Linux nodes powered by Ubuntu Linux or Fedora Linux?

There are two methods to migrate LXD VM between two nodes. I am going to assume that you already setup and running LXD (see how to install LXD on a Fedora Linux). I recommend method # 2 for everyone. It is more elegant and support live migration too.
Tutorial details
Difficulty level Advanced
Root privileges No
Requirements LXD on Linux
Est. reading time 5 minutes

How to move/migrate LXD VM to another host on Linux

I am going to demonstrate two different methods in this tutorial.

Method #1: Use backup and restore over ssh

First, you need to backup /var/lib/lxd including storage pool. Next, you need copy backups to remote LXD instance in the /var/lib/lxd/ including storage pools. Say you have a setup as follows:

Fig.01: Our sample setup to move/migrate LXD VM to another host on Linux

Fig.01: Our sample setup

To see list of vms on server1, enter:
$ lxc list
Sample outputs:

+-----------------+---------+---------------------+------+------------+-----------+
|      NAME       |  STATE  |        IPV4         | IPV6 |    TYPE    | SNAPSHOTS |
+-----------------+---------+---------------------+------+------------+-----------+
| newsletter      | RUNNING | 10.105.28.45 (eth0) |      | PERSISTENT | 0         |
+-----------------+---------+---------------------+------+------------+-----------+
| www-vm          | RUNNING | 10.105.28.42 (eth0) |      | PERSISTENT | 0         |
+-----------------+---------+---------------------+------+------------+-----------+

I am going to assume that you are using DIR as backup storage pool.

Step 1 – Create a backup of www-vm on server1

Type the following tar command:
$ sudo tar -zcvf /root/www-vm.tar.gz /var/lib/lxd/storage-pools/default/containers/www-vm/

Step 2 – Copy www-vm-tar.gz from server1 to server2

Type the following scp command/rsync command:
$ sudo scp /root/www-vm.tar.gz root@server2:/root/
OR
$ sudo rsync -v /root/www-vm.tar.gz root@server2:/root/

Step 3 – Restore backup file named www-vm-tar.gz on server2

Type the following tar command to untar tarball named www-vm.tar.gz in /var/lib/lxd/storage-pools/default/containers/ directory:
$ sudo -i
# cd /var/lib/lxd/storage-pools/default/containers/
# tar -zxvf /root/www-vm-tar.gz

Finally, create a soft-link using ln command, run:
# cd /var/lib/lxd/containers/
# ln -s /var/lib/lxd/storage-pools/default/containers/www-vm/

Step 4 – Restore and import container on server2

LXD maintains a backup.yaml file in each container’s storage volume. Use this to recover or restore a given container, such as container configuration, attached devices and storage. This file can be processed by the following command:
# lxd import {containerNameHere}
# lxd import www-vm

Step 5 – Start imported container on server2

Type the following commands:
$ lxc list
$ lxc start www-vm
$ lxc list
$ lxc exec www-vm bash

Method #2: LXD VM container migration using LXD API and Simplestreams

Here is my sample setup to use LXD API and Simplestreams protocol for migration purpose:

Fig.01: Local to remote hosts and container migration using LXD API

Fig.01: Local to remote hosts and container migration using LXD API

Again, you need two server with LXD configured. Your local machine I will call server1 and a remote server I will call server2.

Step 1 – Configure a remote server named server2

Type the following command to enable a remote access via API:
$ lxc config set core.https_address 192.168.1.6:8443
Set the password for server2 lxd daemon:
$ lxc config set core.trust_password PASSWORDhere
Replace the PASSWORDhere with actual password. You need to use the same in step #2 below. You must open port 8443 using the ufw firewall on Ubuntu Linux, run:
$ sudo ufw allow from 192.168.1.5 to 192.168.1.6 port 8443 proto tcp comment 'Allow lxd client to talk to lxd-server'

Step 2 – Configure a local server named server1

Type the following command on server1 to add server2:
$ lxc remote add server2 192.168.1.6
Sample outputs:

Certificate fingerprint: f4fb0a34a61498d79726079bc...
ok (y/n)? y
Admin password for server2: PASSWORDhere
Client certificate stored at server:  server2

You can list your remotes and you will see “server2” listed as follows:
$ lxc remote list
Sample outputs:

+-----------------+------------------------------------------+---------------+--------+--------+
|      NAME       |                   URL                    |   PROTOCOL    | PUBLIC | STATIC |
+-----------------+------------------------------------------+---------------+--------+--------+
| server2         | https://192.168.1.6:8443                 | lxd           | NO     | NO     |
+-----------------+------------------------------------------+---------------+--------+--------+
| images          | https://images.linuxcontainers.org       | simplestreams | YES    | NO     |
+-----------------+------------------------------------------+---------------+--------+--------+
| local (default) | unix://                                  | lxd           | NO     | YES    |
+-----------------+------------------------------------------+---------------+--------+--------+
| ubuntu          | https://cloud-images.ubuntu.com/releases | simplestreams | YES    | YES    |
+-----------------+------------------------------------------+---------------+--------+--------+
| ubuntu-daily    | https://cloud-images.ubuntu.com/daily    | simplestreams | YES    | YES    |
+-----------------+------------------------------------------+---------------+--------+--------+

You must allow communication between two servers. Open the port 8443 on server1 as well:
$ sudo ufw allow from 192.168.1.6 to 192.168.1.5 port 8443 proto tcp comment 'Allow lxd server2 client to talk to server1 lxd-server'

Step 3 – Copying container named www-vm from server1 to server2

First create a snapshot by running the following command (first snapshot will be named as snap0 and so on). The following requires “www-vm” to be stopped first, but I am going to copy a snapshot instead and do it while the source container is running as follows:
$ lxc snapshot www-vm
$ lxc info www-vm
$ lxc copy www-vm/snap0 server2:www-vm --verbose

Sample outputs:

INFO[08-15|20:03:32] Connecting to a local LXD over a Unix socket 
INFO[08-15|20:03:32] Sending request to LXD                   etag= method=GET url=http://unix.socket/1.0
INFO[08-15|20:03:32] Connecting to a remote LXD over HTTPs 
INFO[08-15|20:03:32] Sending request to LXD                   etag= method=GET url=https://192.168.1.6:8443/1.0
INFO[08-15|20:03:33] Sending request to LXD                   etag= method=GET url=http://unix.socket/1.0/containers/www-vm/snapshots/snap0
INFO[08-15|20:03:33] Sending request to LXD                   etag= method=POST url=http://unix.socket/1.0/containers/www-vm/snapshots/snap0
INFO[08-15|20:03:33] Sending request to LXD                   etag= method=POST url=https://192.168.1.6:8443/1.0/containers
INFO[08-15|20:03:33] Sending request to LXD                   etag= method=GET url=https://192.168.1.6:8443/1.0/operations/f4df846e-d7ca-4c15-a429-6d1256180a84
Transferring container: www-vm: 341.40MB (56.87MB/s)

Step 4 – Start container named www-vm on server2

Type the following command on server1:
$ lxc list server2:
$ lxc start server2:www-vm
$ lxc list cbz02:
$ lxc exec server2:www-vm bash

Conclusion

And there you have it, you just learned how to move/migrate LXD VM to another host on Linux operating system. For more info see the following docs:

  • LXD 2.0 : Remote hosts and container migration [6/12]
  • man lxc
This entry is 10 of 20 in the LXD Tutorial series. Keep reading the rest of the series:
  1. Install LXD container hypervisor on Ubuntu 16.04 LTS
  2. How to install and setup LXC (Linux Container) on Fedora Linux 26
  3. Set up LXD container under KVM or Xen virtual machine
  4. List VM images in LXD (Linux Containers)
  5. Upgrade LXD containers powered by Ubuntu/Debian or CentOS Linux
  6. Auto start LXD containers at boot time in Linux
  7. Command to rename LXD / LXC container
  8. Run commands on Linux Container (LXD) instance at provision launch time
  9. Use LXD (Linux containers) in a shell script to create VM when the cloud instance launches
  10. Move/migrate LXD VM to another host on Linux
  11. Fedora install and set up LXD
  12. CentOS 7.x install and set up LXD server
  13. Install LXD pure-container hypervisor on Ubuntu 18.04 LTS
  14. Create snapshots with lxc command for LXD
  15. Set up and install LXD on CentOS/RHEL 8
  16. Ubuntu 20.04 LTS install and set up LXD
  17. Full backup and restore LXD containers
  18. Disable firewall and NAT rules on the LXD bridge
  19. Delete or remove LXD container using the lxc
  20. Linux Filesystem Error: Transaction failed solution


🐧 Get the latest tutorials on Linux, Open Source & DevOps via RSS feed or Weekly email newsletter.

🐧 7 comments so far... add one

CategoryList of Unix and Linux commands
Disk space analyzersncdu pydf
File Managementcat
FirewallAlpine Awall CentOS 8 OpenSUSE RHEL 8 Ubuntu 16.04 Ubuntu 18.04 Ubuntu 20.04
Network UtilitiesNetHogs dig host ip nmap
OpenVPNCentOS 7 CentOS 8 Debian 10 Debian 8/9 Ubuntu 18.04 Ubuntu 20.04
Package Managerapk apt
Processes Managementbg chroot cron disown fg jobs killall kill pidof pstree pwdx time
Searchinggrep whereis which
User Informationgroups id lastcomm last lid/libuser-lid logname members users whoami who w
WireGuard VPNAlpine CentOS 8 Debian 10 Firewall Ubuntu 20.04
7 comments… add one
  • Casper Aug 16, 2017 @ 16:45

    Would it not be better to store the VM’s on a DRBD device which would ensure that the VM’s would be in sync on all the servers which has this memory device.

    Much easier to deal with than having to use scp/rsync.

  • Daniel Lo Nigro Dec 29, 2020 @ 18:46

    This guide is missing a step. You have to set the core.https_address on both the source server AND the destination server. Without that, you’ll get “Error: The source server isn’t listening on the network”

  • Ximon Apr 26, 2021 @ 11:42

    Honestly for Method2 I am unsure in which ssh connection I should type it.

    Should the commands in Step1 be executed on server1 or server2?

    • 🐧 Vivek Gite Apr 26, 2021 @ 13:07

      Please read again. The page clearly says:
      >Step 1 – Create a backup of www-vm on server1

      • Ximon Apr 26, 2021 @ 13:57

        As mentioned, I was talking about #method2, you are referring to #method1. But thanks, I got it. It needs to be executed on server 2 (remote)

  • Simon Apr 26, 2021 @ 13:54

    I am getting this error when trying to start the container:

    “Error: Failed preparing container for start: Failed to run: zfs mount rpool/lxd/containers/OpenHAB: cannot mount ‘rpool/lxd/containers/OpenHAB’: n o mountpoint set”

  • Ximon Apr 26, 2021 @ 15:00

    I am getting this error message aftery trying to start the copied container:

    Error: Failed preparing container for start: Failed to run: zfs mount rpool/lxd/containers/OpenHAB: cannot mount ‘rpool/lxd/containers/OpenHAB’: no mountpoint set

Leave a Reply

Your email address will not be published.

Use HTML <pre>...</pre> for code samples. Still have questions? Post it on our forum