Tutorial requirements | |
---|---|
Operating system/app | FreeBSD |
Root privileges required | Yes |
Difficulty | Advanced (rss) |
Estimated completion time | 30m |
How to install Linux VM on FreeBSD using bhyve
Our sample set up is as follows:
Our network interface name is em0 with static IP. We will create a network bridge called em0bridge, which will act as a logical link between two or more networks that use the same framing format. We will create a new tap interface for each VM and attach it to our VM. Next, that tap interface will be added to em0bridge. Creating a virtual machine in bhyve is configuring the FreeBSD host system involves various steps. Enough of theory. Let us get our hands dirty with actual commands.
Step 1 – Make tap device UP by default
Append the following entry to the /etc/sysctl.conf:
echo "net.link.tap.up_on_open=1" >> /etc/sysctl.conf
sysctl net.link.tap.up_on_open=1
When net.link.tap.up_on_open is 1/non-zero value, the tunnel device will be marked “up” when the control device is opened.
Step 2 – Loading vmm FreeBSD kernel bhyve virtual machine monitor
Use the kldload command to load FreeBSD kernel module at runtime, run:
kldload vmm
kldload nmdm
Make sure we load the driver as a module at boot time:
echo 'vmm_load="YES"' >> /boot/loader.conf echo 'nmdm_load="YES"' >> /boot/loader.conf echo 'if_tap_load="YES"' >> /boot/loader.conf echo 'if_bridge_load="YES"' >> /boot/loader.conf
Step 3 – Creating bridge and tap device
Create a new bridge, run:
ifconfig bridge create
Add the interface named em0 as a member of the bridge. The interface is put into promiscuous mode so that it can receive every packet sent on the network:
ifconfig bridge0 addm em0
Set the interface name to em0bridge for ease of identification:
ifconfig bridge0 name em0bridge
ifconfig em0bridge up
Finally create a new tap interface called tap0 and add it to the em0bridge:
ifconfig tap0 create
ifconfig em0bridge addm tap0
Make sure we set and activate network devices to auto boot in /etc/rc.conf file:
# byhve and jail settings bridges cloned_interfaces="bridge0 tap0" ifconfig_bridge0_name="em0bridge" ifconfig_em0bridge="addm em0 addm tap0 up"
Step 4 – Creating ZFS volumes for Linux bhyve guest/VM
ZFS gives us significant performance over disk image. We can easily do other operations, such as cloning and backups. Hence, I am going to create a 20GB volume as follows:
zfs create -V20G -o volmode=dev zroot/debianvm
Step 5 – Downloading Debian Linux installer iso image
Use the wget command to download image. Alternatively, we can use curl to download image too:
cd /tmp/
wget https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-10.7.0-amd64-netinst.iso
Step 6 – Install Grub support for bhyve under FreeBSD
GNU/GRUB is a multiboot boot loader and is recommended for Linux guests. The following package allows us to run the grub-bhyve binary. In other words, it is allowing us booting of non-FreeBSD guest operating systems such as Debian or Ubuntu Linux.
pkg install grub2-bhyve
Step 7 – Installing Linux VM on FreeBSD
Now everything is in place. It is time to install the actual Debian Linux 10.x VM under FreeBSD using the bhyve command:
bhyve -c 1 -m 1G -w -H \
-s 0,hostbridge \
-s 3,ahci-cd,/tmp/debian-10.7.0-amd64-netinst.iso \
-s 4,virtio-blk,/dev/zvol/zroot/debianvm \
-s 5,virtio-net,tap0 \
-s 29,fbuf,tcp=0.0.0.0:5900,w=800,h=600,wait \
-s 30,xhci,tablet \
-s 31,lpc -l com1,stdio \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
debianvm
Understanding bhyve command-line options:
- -c : number of virtual cpus
- -H : host filesystem to export to the loader
- -l : the OS loader to use (uefi needed for non-FreeBSD oses such as Linux/Windows guests)
- -m : ram/memory size for VM
- -w : ignore unimplemented MSRs
- -s : Configure a virtual PCI slot and other function such as hard disk, cdrom, and other devices
Getting help is easy too:
man bhyve
bhyve -s help
bhyve -l help
A note about VNC client:
To access Debian 10 Linux installer, open your VNC client and type server_IP:5900. For instance:
Using VNC client to access Debian 10 installer from my Ubuntu Linux desktop
Debian 10 Linux VM installer running on FreeBSD bhyve host
Make sure you scroll down and choose the “Execute a shell” option and click on the “Continue” button:
Type the following mkdir command and [nixmcd name=”cp”]:
mkdir /target/boot/efi/EFI/BOOT/
# copy file - workaround for bhyve grub package #
# Pay attention to destination file bootx64.efi #
cp /target/boot/efi/EFI/debian/grubx64.efi /target/boot/efi/EFI/BOOT/bootx64.efi
And we are done fixing GRUB. Type the “exit” command to return to the installation menu:
exit
Make sure you finish the installation and reboot Debian 10 bhvye VM:
Step 8 – Booting Deiban 10 Linux bhvye VM on FreeBSD
After installing a Debian 10 virtual machine, we rebooted the virtual machine and forced bhyve to exit. Therefore, the instance of the virtual machine needs to be destroyed before it can be started again:
bhyvectl --destroy --vm=debianvm
Finally, boot the Debian 10 virtual machine and enjoy the fruits of our labor:
bhyve -c 2 -m 1G -w -H \
-s 0,hostbridge \
-s 4,virtio-blk,/dev/zvol/zroot/debianvm \
-s 5,virtio-net,tap0 \
-s 29,fbuf,tcp=0.0.0.0:5900,w=1024,h=768,wait \
-s 30,xhci,tablet \
-s 31,lpc -l com1,stdio \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
debianvm
Debian 10 booted successfully under FreeBSD bhyve without UEFI boot problems
Step 9 – Starting Bhyve Debian 10 VM on boot
Create a simple shell script as follows:
#!/bin/sh # Name: startdebianvm # Purpose: Simple script to start my Debian 10 VM using bhyve on FreeBSD # Author: Vivek Gite {https://www.cyberciti.biz} under GPL v2.x+ ------------------------------------------------------------------------- # Lazy failsafe (not needed but I will leave them here) ifconfig tap0 create ifconfig em0bridge addm tap0 if ! kldstat | grep -w vmm.ko then kldload -v vmm fi if ! kldstat | grep -w nmdm.ko then kldload -v nmdm fi bhyve -c 1 -m 1G -w -H \ -s 0,hostbridge \ -s 4,virtio-blk,/dev/zvol/zroot/debianvm \ -s 5,virtio-net,tap0 \ -s 29,fbuf,tcp=0.0.0.0:5900,w=1024,h=768 \ -s 30,xhci,tablet \ -s 31,lpc -l com1,stdio \ -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \ debianvm
Run it using cron job. We can execute cron job after system reboot:
@reboot /path/to/startdebianvm
Summing up
That is all for now. I hope you enjoyed this tutorial. Next time we will talk about firewall NAT settings, reverse proxy, and cloud images. FreeBSD ports have many Bhyve management tools too. Make sure you check out man page and handbook chapter too:
man 8 bhyve
🐧 3 comments so far... add one ↓
Category | List of Unix and Linux commands |
---|---|
File Management | cat |
Firewall | Alpine Awall • CentOS 8 • OpenSUSE • RHEL 8 • Ubuntu 16.04 • Ubuntu 18.04 • Ubuntu 20.04 |
Network Utilities | dig • host • ip • nmap |
OpenVPN | CentOS 7 • CentOS 8 • Debian 10 • Debian 8/9 • Ubuntu 18.04 • Ubuntu 20.04 |
Package Manager | apk • apt |
Processes Management | bg • chroot • cron • disown • fg • jobs • killall • kill • pidof • pstree • pwdx • time |
Searching | grep • whereis • which |
User Information | groups • id • lastcomm • last • lid/libuser-lid • logname • members • users • whoami • who • w |
WireGuard VPN | Alpine • CentOS 8 • Debian 10 • Firewall • Ubuntu 20.04 |
Good read! But you shouldn’t use echo to change base system config files. Use sysrc instead, it provides a minimal form of syntax checking.
ah, yes. will update it later. thank you for the feedback.
Any idea how to get pulseaudio listening in tcp on host and connecting to it from vm?