How to move/migrate LXD VM to another host on Linux

Posted on in Categories , , last updated August 15, 2017

How do I migrate LXD VMs/containers between two Linux nodes powered by Ubuntu Linux?

There are two methods to migrate LXD VM between two nodes. I am going to assume that you already setup and running LXD. I recommend method # 2 for everyone. It is more elegant and support live migration too.

Method #1: Use backup and restore over ssh

First, you need to backup /var/lib/lxd including storage pool. Next, you need copy backups to remote LXD instance in the /var/lib/lxd/ including storage pools. Say you have a setup as follows:

Fig.01: Our sample setup
Fig.01: Our sample setup

To see list of vms on server1, enter:
$ lxc list
Sample outputs:

+-----------------+---------+---------------------+------+------------+-----------+
|      NAME       |  STATE  |        IPV4         | IPV6 |    TYPE    | SNAPSHOTS |
+-----------------+---------+---------------------+------+------------+-----------+
| newsletter      | RUNNING | 10.105.28.45 (eth0) |      | PERSISTENT | 0         |
+-----------------+---------+---------------------+------+------------+-----------+
| www-vm          | RUNNING | 10.105.28.42 (eth0) |      | PERSISTENT | 0         |
+-----------------+---------+---------------------+------+------------+-----------+

I am going to assume that you are using DIR as backup storage pool.

Step 1 – Create a backup of www-vm on server1

Type the following tar command:
$ sudo tar -zcvf /root/www-vm.tar.gz /var/lib/lxd/storage-pools/default/containers/www-vm/

Step 2 – Copy www-vm-tar.gz from server1 to server2

Type the following scp command/rsync command:
$ sudo scp /root/www-vm.tar.gz root@server2:/root/
OR
$ sudo rsync -v /root/www-vm.tar.gz root@server2:/root/

Step 3 – Restore backup file named www-vm-tar.gz on server2

Type the following tar command to untar tarball named www-vm.tar.gz in /var/lib/lxd/storage-pools/default/containers/ directory:
$ sudo -i
# cd /var/lib/lxd/storage-pools/default/containers/
# tar -zxvf /root/www-vm-tar.gz

Finally, create a soft-link using ln command, run:
# cd /var/lib/lxd/containers/
# ln -s /var/lib/lxd/storage-pools/default/containers/www-vm/

Step 4 – Restore and import container on server2

LXD maintains a backup.yaml file in each container’s storage volume. Use this to recover or restore a given container, such as container configuration, attached devices and storage. This file can be processed by the following command:
# lxd import {containerNameHere}
# lxd import www-vm

Step 5 – Start imported container on server2

Type the following commands:
$ lxc list
$ lxc start www-vm
$ lxc list
$ lxc exec www-vm bash

Method #2: LXD VM container migration using LXD API and Simplestreams

Here is my sample setup to use LXD API and Simplestreams protocol for migration purpose:

Fig.01: Local to remote hosts and container migration using LXD API
Fig.01: Local to remote hosts and container migration using LXD API

Again, you need two server with LXD configured. Your local machine I will call server1 and a remote server I will call server2.

Step 1 – Configure a remote server named server2

Type the following command to enable a remote access via API:
$ lxc config set core.https_address 192.168.1.6:8443
Set the password for server2 lxd daemon:
$ lxc config set core.trust_password PASSWORDhere
Replace the PASSWORDhere with actual password. You need to use the same in step #2 below. You must open port 8443 using the ufw firewall on Ubuntu Linux, run:
$ sudo ufw allow from 192.168.1.5 to 192.168.1.6 port 8443 proto tcp comment 'Allow lxd client to talk to lxd-server'

Step 2 – Configure a local server named server1

Type the following command on server1 to add server2:
$ lxc remote add server2 192.168.1.6
Sample outputs:

Certificate fingerprint: f4fb0a34a61498d79726079bc...
ok (y/n)? y
Admin password for server2: PASSWORDhere
Client certificate stored at server:  server2

You can list your remotes and you will see “server2” listed as follows:
$ lxc remote list
Sample outputs:

+-----------------+------------------------------------------+---------------+--------+--------+
|      NAME       |                   URL                    |   PROTOCOL    | PUBLIC | STATIC |
+-----------------+------------------------------------------+---------------+--------+--------+
| server2         | https://192.168.1.6:8443                 | lxd           | NO     | NO     |
+-----------------+------------------------------------------+---------------+--------+--------+
| images          | https://images.linuxcontainers.org       | simplestreams | YES    | NO     |
+-----------------+------------------------------------------+---------------+--------+--------+
| local (default) | unix://                                  | lxd           | NO     | YES    |
+-----------------+------------------------------------------+---------------+--------+--------+
| ubuntu          | https://cloud-images.ubuntu.com/releases | simplestreams | YES    | YES    |
+-----------------+------------------------------------------+---------------+--------+--------+
| ubuntu-daily    | https://cloud-images.ubuntu.com/daily    | simplestreams | YES    | YES    |
+-----------------+------------------------------------------+---------------+--------+--------+

You must allow communication between two servers. Open the port 8443 on server1 as well:
$ sudo ufw allow from 192.168.1.6 to 192.168.1.5 port 8443 proto tcp comment 'Allow lxd server2 client to talk to server1 lxd-server'

Step 3 – Copying container named www-vm from server1 to server2

First create a snapshot by running the following command (first snapshot will be named as snap0 and so on). The following requires “www-vm” to be stopped first, but I am going to copy a snapshot instead and do it while the source container is running as follows:
$ lxc snapshot www-vm
$ lxc info www-vm
$ lxc copy www-vm/snap0 server2:www-vm --verbose

Sample outputs:

INFO[08-15|20:03:32] Connecting to a local LXD over a Unix socket 
INFO[08-15|20:03:32] Sending request to LXD                   etag= method=GET url=http://unix.socket/1.0
INFO[08-15|20:03:32] Connecting to a remote LXD over HTTPs 
INFO[08-15|20:03:32] Sending request to LXD                   etag= method=GET url=https://192.168.1.6:8443/1.0
INFO[08-15|20:03:33] Sending request to LXD                   etag= method=GET url=http://unix.socket/1.0/containers/www-vm/snapshots/snap0
INFO[08-15|20:03:33] Sending request to LXD                   etag= method=POST url=http://unix.socket/1.0/containers/www-vm/snapshots/snap0
INFO[08-15|20:03:33] Sending request to LXD                   etag= method=POST url=https://192.168.1.6:8443/1.0/containers
INFO[08-15|20:03:33] Sending request to LXD                   etag= method=GET url=https://192.168.1.6:8443/1.0/operations/f4df846e-d7ca-4c15-a429-6d1256180a84
Transferring container: www-vm: 341.40MB (56.87MB/s)

Step 4 – Start container named www-vm on server2

Type the following command on server1:
$ lxc list server2:
$ lxc start server2:www-vm
$ lxc list cbz02:
$ lxc exec server2:www-vm bash

References

Posted by: Vivek Gite

The author is the creator of nixCraft and a seasoned sysadmin and a trainer for the Linux operating system/Unix shell scripting. He has worked with global clients and in various industries, including IT, education, defense and space research, and the nonprofit sector. Follow him on Twitter, Facebook, Google+.

1 comment

  1. Would it not be better to store the VM’s on a DRBD device which would ensure that the VM’s would be in sync on all the servers which has this memory device.

    Much easier to deal with than having to use scp/rsync.

Leave a Comment