With OmniOS r151028, support for bhyve and KVM branded zones has been introduced. These branded zones are incredibly small and do not need to be kept up-to-date using pkg since they share files with the global zone. Their purpose is to allow kvm and bhyve to be managed as a zone keeping them isolated from the rest of the system and enabling protection from known CPU vulnerabilities.
Zones are configured largely through the use of attributes which can make
them quite verbose to set up through
zonecfg. You may be interested in
the work on ZCage which has support for
configuring these types of zone.
Most attributes have reasonable default value as described in the next
section so here’s a complete example bhyve zone configuration for installing
Debian from an iso file. Since nothing is explicitly specified, this machine
will default to a single virtual CPU and 1GiB of RAM. The machine’s serial
console is accessible via
zlogin. It is also possible to configure VNC for
all KVM machines and bhyve VMs that use UEFI boot. The below example uses VNC because
the Debian netinstall default assumes a graphical console. See below example for
using socat to access it from a vnc client.
omnios# dladm create-vnic -l net0 bhyve0 omnios# zfs create -V 30G rpool/bhyve0 omnios# zonecfg -z debian create -b set brand=bhyve set zonepath=/data/zone/bhyve set ip-type=exclusive add net set allowed-address=10.0.0.112/24 set physical=bhyve0 end add device set match=/dev/zvol/rdsk/rpool/bhyve0 end add attr set name=bootdisk set type=string set value=rpool/bhyve0 end add fs set dir=/rpool/iso/debian-9.4.0-amd64-netinst.iso set special=/rpool/iso/debian-9.4.0-amd64-netinst.iso set type=lofs add options ro add options nodevices end add attr set name=cdrom set type=string set value=/rpool/iso/debian-9.4.0-amd64-netinst.iso end add attr set name=acpi set type=string set value=off end add attr set name=bootrom set type=string set value=BHYVE_RELEASE end add attr set name=vnc set type=string set value=on end omnios# zoneadm -z debian install omnios# zoneadm -z debian boot omnios# zlogin -C debian
The following table shows the available attributes for bhyve and KVM zones along with their default values. Attributes are added to the zone configuration as shown in the example above; all attributes have the string type.
||firmware image name|
|extra||Arbitrary hypervisor arguments|
||i440fx, q35, amd, netapp, none|
||generic, windows, openbsd|
- bhyve only;
- You will also need to pass the underlying disk device through to the zone via a device entry as shown in the example above;
- The ISO file needs passing through to the zone via a lofs mount as shown in the example above;
- Available firmware files can be found in
- Setting vnc to
onis the same as setting it to
- You can connect to the virtual machine console from the global zone with
zlogin -C zonename;
- For KVM, the extended syntax can cause problems with the guest; it’s best to stick to simple numbers here.
vnc attribute is set to
on, a VNC server will be started
listening on a UNIX socket at
/tmp/vm.vnc within the zone. Note that this
only functions for bhyve zones if the guest is booted via UEFI. In order to
connect the socket to a TCP port so that it can be accessed using a VNC client
one option is to use the mini
socat utility that comes with the brand.
omnios# /usr/lib/brand/bhyve/socat /data/zone/bhyve/root/tmp/vm.vnc 5905
or use the full socat utility from extra:
omnios# socat TCP-LISTEN:5905,reuseaddr,fork UNIX-CONNECT:/data/zone/bhyve/root/tmp/vm.vnc
It is intended that future zone management tools incorporate this feature in an easy-to-use way.
Once the image is installed and it reboots, you’ll get back to the install menu. (Note, you’ll have to reconnect after VNC disconnects at the reboot to see this). To avoid going into another full reinstall, you need to remove the boot cdrom from the zone config.
omnios# zonecfg -z debian zonecfg:debian> remove attr name=cdrom zonecfg:debian> verify zonecfg:debian> commit zonecfg:debian> exit
One final note on Debian. The install image does not setup (as of Debian 10) a proper UEFI boot in the right place for Bhyve to find it, so you’ll have to boot into the Debian OS following a few steps from the uefi shell, get the OS up, then copy a file in place and run update-grub.
- exit from the UEFI Shell
- Now select Boot Maintenance Manager from the Bhyve boot screen
- select Boot From File from the next screen
- at the following screen select the single
- Then select
- Finally, select grubx64.efi from the last screen and your guest OS will boot
- The last step to make this permanent is to issue the following command to put the UEFI boot code into the right place for subsequent boots to be automatic
root# mkdir /boot/efi/EFI/BOOT root# cp /boot/efi/EFI/debian/grubx64.efi /boot/efi/EFI/BOOT/bootx64.efi root# /usr/sbin/update-grub