How to move/migrate LXD VM to another host on Linux

last updated in Categories , , , ,

How do I migrate LXD VMs/containers between two Linux nodes powered by Ubuntu Linux or Fedora Linux?

There are two methods to migrate LXD VM between two nodes. I am going to assume that you already setup and running LXD (see how to install LXD on a Fedora Linux). I recommend method # 2 for everyone. It is more elegant and support live migration too.

How to move/migrate LXD VM to another host on Linux

I am going to demonstrate two different methods in this tutorial.

Method #1: Use backup and restore over ssh

First, you need to backup /var/lib/lxd including storage pool. Next, you need copy backups to remote LXD instance in the /var/lib/lxd/ including storage pools. Say you have a setup as follows:

Fig.01: Our sample setup to move/migrate LXD VM to another host on Linux
Fig.01: Our sample setup

To see list of vms on server1, enter:
$ lxc list
Sample outputs:

+-----------------+---------+---------------------+------+------------+-----------+
|      NAME       |  STATE  |        IPV4         | IPV6 |    TYPE    | SNAPSHOTS |
+-----------------+---------+---------------------+------+------------+-----------+
| newsletter      | RUNNING | 10.105.28.45 (eth0) |      | PERSISTENT | 0         |
+-----------------+---------+---------------------+------+------------+-----------+
| www-vm          | RUNNING | 10.105.28.42 (eth0) |      | PERSISTENT | 0         |
+-----------------+---------+---------------------+------+------------+-----------+

I am going to assume that you are using DIR as backup storage pool.

Step 1 – Create a backup of www-vm on server1

Type the following tar command:
$ sudo tar -zcvf /root/www-vm.tar.gz /var/lib/lxd/storage-pools/default/containers/www-vm/

Step 2 – Copy www-vm-tar.gz from server1 to server2

Type the following scp command/rsync command:
$ sudo scp /root/www-vm.tar.gz root@server2:/root/
OR
$ sudo rsync -v /root/www-vm.tar.gz root@server2:/root/

Step 3 – Restore backup file named www-vm-tar.gz on server2

Type the following tar command to untar tarball named www-vm.tar.gz in /var/lib/lxd/storage-pools/default/containers/ directory:
$ sudo -i
# cd /var/lib/lxd/storage-pools/default/containers/
# tar -zxvf /root/www-vm-tar.gz

Finally, create a soft-link using ln command, run:
# cd /var/lib/lxd/containers/
# ln -s /var/lib/lxd/storage-pools/default/containers/www-vm/

Step 4 – Restore and import container on server2

LXD maintains a backup.yaml file in each container’s storage volume. Use this to recover or restore a given container, such as container configuration, attached devices and storage. This file can be processed by the following command:
# lxd import {containerNameHere}
# lxd import www-vm

Step 5 – Start imported container on server2

Type the following commands:
$ lxc list
$ lxc start www-vm
$ lxc list
$ lxc exec www-vm bash

Method #2: LXD VM container migration using LXD API and Simplestreams

Here is my sample setup to use LXD API and Simplestreams protocol for migration purpose:

Fig.01: Local to remote hosts and container migration using LXD API
Fig.01: Local to remote hosts and container migration using LXD API

Again, you need two server with LXD configured. Your local machine I will call server1 and a remote server I will call server2.

Step 1 – Configure a remote server named server2

Type the following command to enable a remote access via API:
$ lxc config set core.https_address 192.168.1.6:8443
Set the password for server2 lxd daemon:
$ lxc config set core.trust_password PASSWORDhere
Replace the PASSWORDhere with actual password. You need to use the same in step #2 below. You must open port 8443 using the ufw firewall on Ubuntu Linux, run:
$ sudo ufw allow from 192.168.1.5 to 192.168.1.6 port 8443 proto tcp comment 'Allow lxd client to talk to lxd-server'

Step 2 – Configure a local server named server1

Type the following command on server1 to add server2:
$ lxc remote add server2 192.168.1.6
Sample outputs:

Certificate fingerprint: f4fb0a34a61498d79726079bc...
ok (y/n)? y
Admin password for server2: PASSWORDhere
Client certificate stored at server:  server2

You can list your remotes and you will see “server2” listed as follows:
$ lxc remote list
Sample outputs:

+-----------------+------------------------------------------+---------------+--------+--------+
|      NAME       |                   URL                    |   PROTOCOL    | PUBLIC | STATIC |
+-----------------+------------------------------------------+---------------+--------+--------+
| server2         | https://192.168.1.6:8443                 | lxd           | NO     | NO     |
+-----------------+------------------------------------------+---------------+--------+--------+
| images          | https://images.linuxcontainers.org       | simplestreams | YES    | NO     |
+-----------------+------------------------------------------+---------------+--------+--------+
| local (default) | unix://                                  | lxd           | NO     | YES    |
+-----------------+------------------------------------------+---------------+--------+--------+
| ubuntu          | https://cloud-images.ubuntu.com/releases | simplestreams | YES    | YES    |
+-----------------+------------------------------------------+---------------+--------+--------+
| ubuntu-daily    | https://cloud-images.ubuntu.com/daily    | simplestreams | YES    | YES    |
+-----------------+------------------------------------------+---------------+--------+--------+

You must allow communication between two servers. Open the port 8443 on server1 as well:
$ sudo ufw allow from 192.168.1.6 to 192.168.1.5 port 8443 proto tcp comment 'Allow lxd server2 client to talk to server1 lxd-server'

Step 3 – Copying container named www-vm from server1 to server2

First create a snapshot by running the following command (first snapshot will be named as snap0 and so on). The following requires “www-vm” to be stopped first, but I am going to copy a snapshot instead and do it while the source container is running as follows:
$ lxc snapshot www-vm
$ lxc info www-vm
$ lxc copy www-vm/snap0 server2:www-vm --verbose

Sample outputs:

INFO[08-15|20:03:32] Connecting to a local LXD over a Unix socket 
INFO[08-15|20:03:32] Sending request to LXD                   etag= method=GET url=http://unix.socket/1.0
INFO[08-15|20:03:32] Connecting to a remote LXD over HTTPs 
INFO[08-15|20:03:32] Sending request to LXD                   etag= method=GET url=https://192.168.1.6:8443/1.0
INFO[08-15|20:03:33] Sending request to LXD                   etag= method=GET url=http://unix.socket/1.0/containers/www-vm/snapshots/snap0
INFO[08-15|20:03:33] Sending request to LXD                   etag= method=POST url=http://unix.socket/1.0/containers/www-vm/snapshots/snap0
INFO[08-15|20:03:33] Sending request to LXD                   etag= method=POST url=https://192.168.1.6:8443/1.0/containers
INFO[08-15|20:03:33] Sending request to LXD                   etag= method=GET url=https://192.168.1.6:8443/1.0/operations/f4df846e-d7ca-4c15-a429-6d1256180a84
Transferring container: www-vm: 341.40MB (56.87MB/s)

Step 4 – Start container named www-vm on server2

Type the following command on server1:
$ lxc list server2:
$ lxc start server2:www-vm
$ lxc list cbz02:
$ lxc exec server2:www-vm bash

Conclusion

And there you have it, you just learned how to move/migrate LXD VM to another host on Linux operating system. For more info see the following docs:

  • LXD 2.0 : Remote hosts and container migration [6/12]
  • man lxc
This entry is 10 of 13 in the LXD Tutorial series. Keep reading the rest of the series:
  1. How to install LXD container hypervisor on Ubuntu 16.04 LTS
  2. How to install and setup LXC (Linux Container) on Fedora Linux 26
  3. How to install LXD container under KVM or Xen virtual machine
  4. How to list VM images in LXD (Linux Containers)
  5. How to upgrade LXD VMs powered by Ubuntu/Debian or CentOS Linux
  6. How to auto start LXD containers at boot time in Linux
  7. How to rename LXD / LXC container
  8. How to run commands on Linux Container (LXD) instance at provision launch time
  9. How to use LXD (Linux containers) in a shell script to create VM when the cloud instance launches
  10. How to move/migrate LXD VM to another host on Linux
  11. How to install and setup LXD on Fedora Linux server
  12. How to install and setup LXD on CentOS Linux 7.x server
  13. Install LXD pure-container hypervisor on Ubuntu 18.04 LTS

Posted by: Vivek Gite

The author is the creator of nixCraft and a seasoned sysadmin, DevOps engineer, and a trainer for the Linux operating system/Unix shell scripting. Get the latest tutorials on SysAdmin, Linux/Unix and open source topics via RSS/XML feed or weekly email newsletter.

Start the discussion at www.nixcraft.com

Historical Comment Archive

1 comment

  1. Would it not be better to store the VM’s on a DRBD device which would ensure that the VM’s would be in sync on all the servers which has this memory device.

    Much easier to deal with than having to use scp/rsync.

    Have a question? Post it on our forum!