View source for ArchLinux:KVM

{{DISPLAYTITLE:{{TitleIcon|arch=true}} KVM on Arch Linux}}<metadesc>How to automate VMs using QEMU KVM with libvirt, Packer and Vagrant on Arch Linux.</metadesc>
<div id="tocalign">__TOC__</div>
{{UnderConstruction}}
{{Back|Arch Linux}}
{{SeeAlso|ArchLinux:OVH|OVH Custom Installation}}
= {{Icon24|sitemap}} Introduction =
This is a tutorial on how to automate the setup of VMs using KVM on Arch Linux. This tutorial utilizes [//www.qemu.org QEMU] as a back-end for KVM using [//libvirt.org libvirt]. System base images will be generated using [//www.packer.io Packer]. And finally, [//www.vagrantup.com/ Vagrant] and [//github.com/vagrant-libvirt/vagrant-libvirt vagrant-libvirt] will be utilized for KVM test environments.

''Tutorial environment consists of a database server, a DNS server, a web server and one or more test servers (which may or may not be clones of the three main servers). Additional servers should be able available on demand for any use case. All machine images should be built in-house so that image security can be maintained.''

= {{Icon24|sitemap}} Installation =
Before getting started there are a few packages that will be needed to set all of this up.
{{Console|1=pacaur -S bridge-utils libguestfs libvirt openbsd-netcat openssl-1.0 ovmf {{GreenBold|\}}<br/>	    packer-io qemu-headless qemu-headless-arch-extra vagrant}}
== {{Icon|notebook}} Swap ==
''"The idea is you assign each guest a large amount of memory (more than you can actually give out) because they're generally not using it. Then you do the math to ensure you have enough swap space that the guests can actually swap out to disk in the worst-case-scenario where they all actually do use all that memory."'' <ref>[//unix.stackexchange.com/questions/137509/kvm-and-swap-space Unix & Linux Stack Exchange]. KVM and Swap space</ref> As user ndt points out, the swap space on the host machine should be equivalent to {{mono|host Total Host Swap {{=}} Sum of Guest Memory Assigned + Recommended OS Swap Size}}.

During the installation of the host machine two {{mono|8GB}} partitions were left on each drive for this reason. If more space is needed, it can always be cut from LVM.

Setup the two swap partitions as Linux swap space.
{{Console|1=sudo mkswap /dev/sda2|2=sudo mkswap /dev/sdb2}}
Activate the two swap partitions.
{{Console|1=sudo swapon /dev/sda2|2=sudo swapon /dev/sdb2}}
The swap partitions will also need to be activated on boot.
{{Console|1=sudo lsblk -no UUID /dev/sda2|2=sudo lsblk -no UUID /dev/sdb2}}
{{margin}}
{{Console|title=/etc/fstab|prompt=false|1=UUID=XXX-XXXX…	none	swap	defaults	0	0<br/>UUID=XXX-XXXX…	none	swap	defaults	0	0}}

== {{Icon|notebook}} Hugepages ==
Enabling hugepages can improve the performance of virtual machines. First add an entry to the fstab, make sure to first check what the group id of the group {{mono|kvm}} is.
{{Console|1=grep kvm /etc/group|2=sudoedit /etc/fstab}}<br/>
{{Console|title=/etc/fstab|prompt=false|1=hugetlbfs	/dev/hugepages	hugetlbfs	mode=1770,gid=999	0	0}}
Instead of rebooting, remount instead.
{{Console|1=sudo umount /dev/hugepages|2=mount /dev/hugepages}}
This can then be verified.
{{Console|1=sudo mount {{!}} grep huge|2=ls -FalG /dev/ {{!}} grep huge}}
Now to set the number of hugepages to use. For this one has to do a bit of math, for each gigabyte of the system RAM that you want to use for VMs you divide the size in megabytes by two.
{{Note|On my setup I will dedicated 12GB out of the 16GB of system RAM to VMs. This means {{mono|(12 * 1024) / 2}} or {{mono|6144}}}}
Set the number of hugepages.
{{Console|1=echo 6144 {{!}} sudo tee /proc/sys/vm/nr_hugepages}}
Also set this permanently by adding a file to {{mono|/etc/sysctl.d}}.
{{Console|title=/etc/sysctl.d/40-hugepages.conf|prompt=false|1=vm.nr_hugepages = 6144}}
Again verify the changes.
{{Console|1=grep HugePages_Total /proc/meminfo}}
Edit the libvirt QEMU config and turn hugepages on.
{{Console|title=/etc/libvirt/qemu.conf|prompt=false|1=…<br/>hugetlbfs_mount {{=}} "/dev/hugepages"<br/>…}}

== {{Icon|notebook}} KVM Group ==
Create a user for KVM.
{{Console|1=sudo useradd -g kvm -s /usr/bin/nologin kvm}}
Then modify the libvirt QEMU config to reflect this.
{{Console|title=/etc/libvirt/qemu.conf|prompt=false|1=user {{=}} "kvm"<br/>group {{=}} "kvm"}}
Fix permission on {{mono|/dev/kvm}}
{{Console|1=sudo groupmod -g 78 kvm}}
{{Note|systemd as of 234 assigns dynamic IDs to groups, but KVM expects 78}}
Add the current user to the {{mono|kvm}} group.
{{Console|1=sudo gpasswd -a kyau kvm}}
== {{Icon|notebook}} Kernel Modules ==
In order to mount directories from the host inside of a virtual machine, the {{mono|9pnet_virtio}} kernel module will need to be loaded.
{{Console|1=sudo modprobe 9pnet_virtio}}
Also load the module on boot.
{{Console|title=/etc/modules-load.d/virtio-9pnet.conf|prompt=false|1=9pnet_virtio}}
== {{Icon|notebook}} OVMF & IOMMU ==
The Open Virtual Machine Firmware (OVMF) is a project to enable UEFI support for virtual machines and enabling IOMMU will enable PCI pass-through among other things. This extends the possibilities for operating system choices significantly and also provides some other options.

Enable IOMMU on boot by adding an option to the kernel line in GRUB.
{{Console|title=/etc/default/grub|prompt=false|1=GRUB_CMDLINE_LINUX_DEFAULT{{=}}"intel_iommu{{=}}on"}}
Re-generate the GRUB config.
{{Console|1=sudo grub-mkconfig -o /boot/grub/grub.cfg}}
Reboot the machine and then verify IOMMU is enabled.
{{Console|1=sudo dmesg {{!}} grep -e DMAR -e IOMMU}}
If it was enabled properly, there should be a line similar to {{mono|[    0.000000] DMAR: IOMMU enabled}}.

Adding the OVMF firmware to libvirt.
{{Console|title=/etc/libvirt/qemu.conf|prompt=false|nvram {{=}} [<br/>	"/usr/share/ovmf/ovmf_code_x64.bin:/usr/share/ovmf/ovmf_vars_x64.bin"<br/>]}}

= {{Icon24|sitemap}} LVM =
During the [[ArchLinux:OVH|installation]] of the KVM host machine a {{mono|data}} volume group was created for VMs. Before carving out disk space for virtual machines, create the volume(s) that will exist outside of the virtual machines. These will be used for databases, web root directories and any other data that needs to persist between VM creation and destruction.
{{Console|1=sudo lvcreate -L 256G data --name http}}
{{Note|I am only using a single LVM volume and then creating directories inside of this for each machine}}
Create a directory for the volume.
{{Console|1=sudo mkdir /http}}
Format the new volume with {{mono|ext4}}.
{{Console|1=sudo mkfs.ext4 -O metadata_csum,64bit /dev/data/http|2=sudo mount /dev/data/http /http}}
Set proper permissions and mod the {{mono|http}} user's home directory.
{{Console|1=sudo chown http:http /http|2=sudo usermod -m -d /http http}}
Add the volume to {{mono|fstab}} so that it mounts upon boot.
{{Console|title=/etc/fstab|prompt=false|/dev/mapper/data-http	/http	ext4	rw,relatime,stripe{{=}}256,data{{=}}ordered,journal_checksum	0	0}}
Volumes will now need to be created for each virtual machine, for this an LVM thin pool can be utilized.
== {{Icon|notebook}} LVM Thin Provisioning ==
Thin provisioning creates another virtual layer on top of your volume group, in which logical thin volumes can be created. Thin volumes, unlike normal thick volumes, do not reserve the disk space for the volume on creation but instead do so upon write; to the operating system they are still reported as full size volumes. This means that when utilizing LVM directly for KVM it will perform similarly to a "dynamic disk" meaning it will only use what disk space it needs regardless of how big the virtual hard drive actually is. This can also be paired with LVM cloning (snapshots) to create some interesting setups, like running 1TB of VMs on a 128GB disk for example.
{{Warning|The one disadvantage to doing this is that without proper disk monitoring and management this can lead to over provisioning (overflow will cause volume drop)}}
Use the rest of the {{mono|data}} volume group for the thin pool.
{{Console|1=sudo lvcreate -l +100%FREE data --thinpool qemu}}
Pulling up {{mono|lvdisplay}} can verify that it created a thin pool.
{{Console|1=sudo lvdisplay data/qemu}}<br/>
{{Console|prompt=false|1={{Black|__}}LV Size                <1.50 TiB<br/>  Allocated pool data    0.00%}}
Finally {{mono|lvs}} should show the volume with the {{mono|t}} and {{mono|tz}} attributes as well as a data percentage.
{{Console|1=sudo lvs}}<br/>
{{Console|prompt=false|1={{Black|__}}LV   VG      Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert<br/>  http data    -wi-ao---- 256.00g<br/>  qemu data    twi-a-tz--  <1.50t             0.00   0.43<br/>  root neutron -wi-ao----  63.93g}}
Adding volumes to the thin pool is very similar to adding normal volumes, add one for the first VM.
{{Console|1=sudo lvcreate -V 20G --thin -n dns data/qemu}}
These volumes can be shrunk or extended at any point.
{{Console|1=sudo lvextend -L +15G data/dns}}
Or even removed entirely.
{{Console|1=sudo lvremove data/dns}}
Verify the new {{mono|base}} volume was added correctly to the thin pool.
{{Console|1=sudo lvs}}
The volume should be marked in pool {{mono|qemu}}, have a data of {{mono|0.00%}} and attributes {{mono|V}} and {{mono|tz}}.
{{Console|prompt=false|1={{Black|__}}LV   VG      Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert<br/>  dns  data    Vwi-a-tz--  20.00g qemu        0.00}}

= {{Icon24|sitemap}} Packer =
Packer is a tool for automating the creation of virtual machines, in this instance it will be used to automate the creation of Vagrant boxes. I have already taken the time to create a packer template for Arch Linux based off of my installation tutorials, but I encourage you to use this only as a basis and delve deeper to create your own templates. I could have very easily just have downloaded someone else's templates, but then I would lack understanding.
{{GitHub|[//github.com/kyau/packer-kvm-templates kyau/packer-kvm-templates]}}
== {{Icon|notebook}} Vagrant-Libvirt ==
The libvirt plugin installation for vagrant requires some cleanup first.
{{Console|1=sudo mv /opt/vagrant/embedded/lib/libcurl.so{,.backup}|2=sudo mv /opt/vagrant/embedded/lib/libcurl.so.4{,.backup}|3=sudo mv /opt/vagrant/embedded/lib/libcurl.so.4.4.0{,.backup}|4=sudo mv /opt/vagrant/embedded/lib/pkgconfig/libcurl.pc{,backup} }}
Then build the plugin.
{{Console|1=vagrant plugin install vagrant-libvirt}}
== {{Icon|notebook}} Templates ==
The Packer templates are in JSON format and contain all of the information needed to create the virtual machine image. Descriptions of all the template sections and values, including default values, can be found in the [//www.packer.io/docs/templates/index.html Packer docs]. For Arch Linux, the template file {{mono|archlinux-x86_64-base-vagrant.json}} will be used to generate an Arch Linux {{mono|qcow2}} virtual machine image.
{{Console|1=<nowiki>git clone https://github.com/kyau/packer-kvm-templates</nowiki>}}
To explain the template a bit, inside of the {{mono|builders}} section the template is specifying that it is a qcow2 image running on QEMU KVM. A few settings are being imported from user variables that are being set in the previous section, this includes the ISO url and checksum, the country setting, disk space for the VMs primary hard drive, the amount of RAM to dedicate to the VM, how many vCores to dedicated to the VM, whether or not it is a headless VM or not, and the login and password for the primary SSH user. These are all set as user variables and placed in a section at the top to be able to make quick edits. The template also specifies that the VM should use {{mono|virtio}} for the disk and network interfaces. Lastly the builtin web server in Packer and the boot commands; the {{mono|http_directory}} specifies which directory will be the main root of the builtin web server (this enables one to host files up for the VM to access during installation). The {{mono|boot_command}} is an array of commands that are to be executed upon boot in order to kick-start the installer. Finally, the {{mono|qemuargs}} should be rather apparent as they are the arguments passed to QEMU.
{{Console|1=cd packer-kvm-templates}}
Looking then at the {{mono|provisioners}} section which is executing three separate scripts after the machine has booted. These scripts are also being passed the required user variables that are set at the top of the file as shell variables. The {{mono|install.sh}} script is the one that installs Arch Linux, {{mono|hardnening.sh}} is the script that applies [[ArchLinux:Security|hardening]] the Arch Linux installation and finally {{mono|cleanup.sh}} is there for general cleanup after the installation is complete.
While the {{mono|README.md}} does have all of this information for the packer templates, it will also be detailed here.


For added security generate a new {{mono|moduli}} for your VMs (or copy from {{mono|/etc/ssh/moduli}}.
{{Console|1=ssh-keygen -G moduli.all -b 4096|2=ssh-keygen -T moduli.safe -f moduli.all|3=mv moduli.safe moduli && rm moduli.all}}
Enter the directory for the Arch Linux template and sym-link the moduli.
{{Console|1=cd archlinux-x86_64-base/default|2=ln -s ../../moduli . && cd ..}}
Build the base virtual machine image.
{{Console|1=./build archlinux-x86_64-base-vagrant.json}}
{{Note|This runs: {{mono|PACKER_LOG{{=}}1 PACKER_LOG_PATH{{=}}"packer.log" packer-io build archlinux-x86_64-base-vagrant.json}}, it logs to the current directory}}
Once finished, there should be a qcow2 vagrant-libvirt image for Arch Linux in the {{mono|box}} directory.

Add this image to Vagrant.
{{Console|1=vagrant box add box/archlinux-x86_64-base-vagrant-libvirt.box --name archlinux-x86_64-base}}

= {{Icon24|sitemap}} Vagrant-Libvirt =
Vagrant can be used to build and manage test machines. The [//github.com/vagrant-libvirt/vagrant-libvirt vagrant-libvirt] plugin adds a Libvirt provider to Vagrant, allowing Vagrant to control and provision machines via the Libvirt toolkit.

To bring up the first machine initialize Vagrant in a new directory first create a directory for the machine.
{{Console|1=cd|2=mkdir testmachine|3=cd testmachine}}
Init the machine the Vagrant.
{{Console|1=vagrant init archlinux-x86_64-base}}
Then bring up the machine.
{{Console|1=vagrant up}}
Then SSH into the machine directly.
{{Console|1=vagrant ssh}}
= {{Icon24|sitemap}} QEMU =
While Vagrant is a great tool for working with test and development environments, for the more permanent VMs on the system, utilizing QEMU directly will allow the VMs to run directly off of LVM thin volumes. Currently vagrant-libvirt cannot do this, due to it's own snapshotting interfering with it; thankfully LVM has snapshotting of its own.

For this a separate Packer template was created, one with all of the Vagrant stuff removed. To build one of these simply use the other {{mono|JSON}} file in the Arch Linux template directory.
{{Console|1=./build archlinux-x86_64-base.json}}
This can then be output directly to the LVM thin volume.
{{Console|1=sudo qemu-img convert -f qcow2 -O raw qcow2/archlinux-x86_64-base.qcow2 /dev/data/dns}}
Then because it copied a thick volume onto a thin volume it will be using all of the disk space.
{{Console|1=sudo lvs}}<br/>
{{Console|prompt=false|1={{Black|__}}LV   VG      Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert<br/>  dns  data    Vwi-a-tz--  20.00g qemu        100.00}}
The disk merely needs to be sparsified.
{{Console|1=sudo virt-sparsify --in-place /dev/data/dns}}
The disk should now be reading properly.
{{Console|1=sudo lvs}}<br/>
{{Console|prompt=false|1={{Black|__}}LV   VG      Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert<br/>  dns  data    Vwi-a-tz--  20.00g qemu        7.17}}
== {{Icon|notebook}} Network Bridge ==
Setting up a network bridge for KVM is simple with systemd. Replace {{mono|X.X.X.X}} with the host machine's IP address and update the Gateway and DNS if not using OVH.
{{Console|title=/etc/systemd/network/kvm0.netdev|prompt=false|1=[NetDev]<br/>Name{{=}}kvm0<br/>Kind{{=}}bridge}}
{{margin}}
{{Console|title=/etc/systemd/network/kvm0.network|prompt=false|1=[Match]<br/>Name{{=}}kvm0<br/><br/>[Network]<br/>DNS{{=}}213.186.33.99<br/>Address{{=}}{{MagentaBold|X.X.X.X}}/24<br/>Gateway{{=}}{{MagentaBold|Y.Y.Y}}.254<br/>IPForward{{=}}yes}}
{{margin}}
{{Console|title=/etc/systemd/network/eth0.network|prompt=false|1=[Match]<br/>Name{{=}}eth0<br/><br/>[Network]<br/>Bridge{{=}}kvm0}}
And finally restart networkd.
{{Console|1=sudo systemctl restart systemd-networkd}}
The bridge should now be up and running, this should be verified.
{{Console|1=ip a}}
Once the bridge is up and running QEMU can be directed to use it. Create a directory in {{mono|/etc/}} for QEMU and then make a {{mono|bridge.conf}}.
{{Console|1=sudo mkdir /etc/qemu|2=sudoedit /etc/qemu/bridge.conf}}
{{margin}}
{{Console|title=/etc/qemu/bridge.conf|prompt=false|1=allow kvm0}}
Then set {{mono|cap_net_admin}} on the binary helper.
{{Console|1=sudo setcap cap_net_admin=ep /usr/lib/qemu/qemu-bridge-helper}}
{{Warning|I had major issues using the bridge as a regular user, I actually had to remove the setuid bit to get it working: {{mono|sudo chmod u-s /usr/lib/qemu/qemu-bridge-help}}}}

== {{Icon|notebook}} NAT ==
To get NAT working inside of each VM IP forwarding will need to be enabled.
{{Console|title=/etc/sysctl.d/99-kvm.conf|prompt=false|1=net.ipv4.ip_forward {{=}} 1<br/>2 net.ipv6.conf.all.forwarding {{=}} 1<br/>net.ipv6.conf.all.proxy_ndp {{=}} 1<br/>net.ipv6.conf.default.forwarding {{=}} 1}}
Rules will also need to be appended to {{mono|nftables}}.
{{Console|title=/etc/nftables.conf|prompt=false|1=table inet filter {<br/>…<br/>  chain foward {<br/>    type filter hook forward priority 0;<br/>    oifname kvm0 accept<br/><nowiki>    </nowiki>iifname kvm0 ct state related, established accept<br/>    iifname kvm0 drop<br/>  }<br/>…<br/>} }}
Rebooting at this point to make sure all these networking settings were set correctly would be a wise idea.
{{Console|1=sudo systemctl reboot}}

== {{Icon|notebook}} Network Test ==
The network on the VM should now be fully tested, for this a connection can be made using the SPICE protocol. On a local client machine install {{mono|vinagre}}.
{{Console|1=pacaur -S vinagre}}

Using the OVH/SyS Manager setup two failover IP addresses to the same virtual MAC. The following arguments will launch the virtual machine. Be sure to input the proper virtual MAC so that it matches the one that OVH assigned.
{{Console|1=/usr/bin/qemu-system-x86_64 --enable-kvm -machine q35,accel=kvm -device intel-iommu {{GreenBold|\}}<br/>	-m 512 -smp 1 -cpu host -drive file=/dev/data/dns,cache=none,if=virtio,format=raw {{GreenBold|\}}<br/>	-net bridge,br=kvm0 -net nic,model=virtio,macaddr=00:00:00:00:00:00 -vga qxl {{GreenBold|\}}<br/>	-spice port=5900,addr=127.0.0.1,disable-ticketing {{GreenBold|\}}<br/>	-monitor unix:/tmp/monitor-dns.sock,server,nowait}}
Once launched, you should be able to connect to the KVM using a SPICE client such as Vinagre. Click {{mono|Connect}} in Vinagre, set the {{mono|Host:}} to {{mono|localhost}} and then make sure {{mono|Use host}} is checked with your KVM host server name filled in "as a SSH tunnel". Connect and enter your SSH key password.

The KVM virtual machine should now be visible through Vinagre.

Login as {{mono|root}}, if this was built using {{mono|packer-kvm-templates}} the default password is {{mono|password}}.

Edit the network interface configuration for systemd. This first VM is going to be acting as my DNS server, therefore it will be assigned two IP addresses.
{{Console|title=/etc/systemd/network/eth0.service|prompt=false|1=[Match]<br/>Name{{=}}eth0<br/><br/>[Network]<br/>Address{{=}}FAILOVER_IP_1/32<br/>Address{{=}}FAILOVER.IP.2/32<br/>DNS{{=}}213.186.33.99<br/>Peer{{=}}HOST_GATEWAY/32<br/><br/>[Gateway]<br/>Gateway{{=}}HOST_GATEWAY<br/>Destination{{=}}0.0.0.0/0}}
This is exactly how [//docs.ovh.ca/en/guides-network-bridging.html#networkd OVH] says it should be setup, however this was not enough as the VM still did not have a default route.
{{Note|TODO: Fix this section, this is an ugly hack}}
To fix the routing create a service on boot.
{{Console|title=/usr/lib/systemd/system/kvmnet.service|prompt=false|1=[Unit]<br/>Description{{=}}Start KVM Network<br/>After{{=}}network.target<br/>Before{{=}}multi-user.target shutdown.target<br/>Conflicts{{=}}shutdown.target<br/>Wants{{=}}network.target<br/><br/>[Service]<br/>ExecStart{{=}}/usr/local/bin/kvmnet<br/><br/>[Install]<br/>WantedBy{{=}}multi-user.target}}
And the script that does the routing.
{{Console|title=/usr/local/bin/kvmnet|prompt=false|1=#!/bin/bash<br/>ip route add {{MagentaBold|Y.Y.Y}}.254 dev eth0<br/>ip route add default via {{MagentaBold|Y.Y.Y}}.254 dev eth0}}
Don't forget to make it executable.
{{Console|1=sudo chmod +rx /usr/local/bin/kvmnet}}
Enable the service.
{{Console|1=sudo systemctl enable kvmnet}}
Reboot the VM and then verify it has internet access.
{{Console|1=sudo reboot|2=ping archlinux.org}}
Finally, verify it can be SSH into from the outside via '''BOTH''' IP addresses.

= {{Icon24|sitemap}} Libvirt =
To launch the virtual machines on boot there are two options. The first option involves importing the virtual machines into libvirt with {{mono|virsh}}. The second option is to setup a systemd service. Given that management will be loads easier with virt-manager I will opt for this option.

On the KVM host machine enable and start {{mono|libvirtd}}.
{{Console|1=sudo systemctl enable libvirtd|2=sudo systemctl start libvirtd}}
Then enable access to libvirtd to everyone in the {{mono|kvm}} group.
{{Console|title=/etc/polkit-1/rules.d/50-libvirt.rules|prompt=false|1={{BlackBold|/* Allow users in kvm group to manage the libvirt daemon without authentication */}}<br/>polkit.addRule(function(action, subject) {<br/>    if (action.id {{=}}{{=}} "org.libvirt.unix.manage" &&<br/>        subject.isInGroup("kvm")) {<br/>            return polkit.Result.YES;<br/>    }<br/>});}}
== {{Icon|notebook}} Virsh ==
Virsh is the command line interface for libvirt. It can be used to import the QEMU arguments into an XML format that libvirt will understand.

To make life easier it is suggested to make a shell alias for {{mono|virsh}}.
{{Console|1=alias virsh{{=}}"virsh -c qemu:///system"}}

Save the QEMU arguments used before to a temporary file.
{{Console|1=echo "/usr/bin/qemu-system-x86_64 --enable-kvm -machine q35,accel=kvm -device intel-iommu {{GreenBold|\}}<br/>	-m 512 -smp 1 -cpu Broadwell -drive file=/dev/data/dns,cache=none,if=virtio,format=raw {{GreenBold|\}}<br/>	-net bridge,br=kvm0 -net nic,model=virtio,macaddr=00:00:00:00:00:00 -vga qxl {{GreenBold|\}}<br/>	-spice port=5900,addr=127.0.0.1,disable-ticketing {{GreenBold|\}}<br/>	-monitor unix:/tmp/monitor-dns.sock,server,nowait" > kvm.args}}
{{Note|Temporarily changing the CPU because virsh cannot recognize {{mono|host}}}}
Convert this to XML format.
{{Console|1=virsh domxml-from-native qemu-argv kvm.args > dns.xml}}
Then open up the XML file in an editor and change the {{mono|name}}, {{mono|cpu}} and {{mono|graphics}} block.
{{Console|title=dns.xml|prompt=false|1=…<br/><nowiki>  <name>DNS (Arch64)</name></nowiki><br/>…<br/><nowiki>  <cpu mode='host-passthrough' /></nowiki><br/>…<br/><nowiki>  <graphics type='spice' port='5900' autoport='no' listen='127.0.0.1'></nowiki><br/>    <listen type='address' address='127.0.0.1' /><br/>  </graphics><br/>…}}
The last two {{mono|qemu:commandline}} arguments can also be removed as they were setting up the SPICE server which is done through the {{mono|graphics}} block.

The XML should now be in a similar state as to when it was executed with the QEMU binary.

Import the XML into libvirt.
{{Console|1=virsh define dns.xml}}
The VM can now be launched.
{{Console|1=virsh start DNS}}
SSH and SPICE over SSH should both now work and the machine should be running.
Use the following to start the machine on boot.
{{Console|1=virsh autostart DNS}}
{{Note|If you have issues auto-starting the machine, check the logfile {{mono|/var/log/libvirt/qemu/}}.}}
If libvirt complains about permission denied on mounting the LVM thin volume, set proper permissions by creating a udev rule.
{{Console|title=/etc/udev/rules.d/90-neutron.rules|prompt=false|1=ENV{DM_VG_NAME}{{=}}{{=}}"data" ENV{DM_LV_NAME}{{=}}{{=}}"*" OWNER{{=}}"kvm"}}
A reboot of the host machine at this point should yield the virtual machine DNS starting up automatically.

== {{Icon|notebook}} Virt-Manager ==
Virt-manager can be used to manage the virtual machines remotely.

Virt-manager can now be installed on the local machine (the one viewing this tutorial not the KVM host machine), this can be used to connect to libvirt remotely via SSH.
{{Console|1=pacaur -S virt-manager}}
Connect remotely to QEMU/KVM with virt-manager over SSH and the virtual machine should be shown as running.

= {{Icon24|sitemap}} Additional Notes =
These notes are here from my own install.
{{Console|1=cd ~/packer-kvm-templates/archlinux-x86_64-base|2=./build archlinux-x86_64-base.json}}
{{margin}}
{{Console|1=sudo lvcreate -V 20G --thin -n bind data/qemu|2=sudo lvcreate -V 20G --thin -n sql data/qemu|3=sudo lvcreate -V 20G --thin -n nginx data/qemu}}
{{margin}}
{{Console|1=sudo qemu-img convert -f qcow2 -O raw qcow2/archlinux-x86_64-base.qcow2 /dev/data/bind|2=sudo virt-sparsify --in-place /dev/data/bind}}
{{margin}}
{{Console|1=vim virshxml|2=./virshxml|3=virsh define ~/newxml-bind.xml}}
Then repeat this for {{mono|sql}} and {{mono|nginx}}.
{{Note|Don't forget about the notes {{mono|virshxml}} gives for replacing the networkd service}}
{{Console|1=virsh start bind|2=virsh start sql|3=virsh start nginx}}
{{margin}}
{{Console|1=virsh autostart bind|2=virsh autostart sql|3=virsh autostart nginx}}

= {{Icon24|sitemap}} DNS =
== {{Icon|notebook}} DNSSEC ==
Adding DNSSEC to BIND is always a good idea, first add the following lines to the {{mono|options}} inside of the BIND config.
{{Console|title=/etc/named.conf|prompt=false|dnssec-enable yes;<br/>dnssec-validation yes;<br/>dnssec-lookaside auto;}}
Install {{mono|haveged}} for key generation inside of VMs.
{{Console|1=pacaur -S haveged|2=haveged -w 1024}}
Gain root privileges.
{{Console|1=sudo -i|2=cd /var/named}}
Create zone signing keys for all domains.
{{Console|1=dnssec-keygen -a ECDSAP384SHA384 -n ZONE kyau.net|2=dnssec-keygen -a ECDSAP384SHA384 -n ZONE kyau.org}}
Create a key signing keys for all domains.
{{Console|1=dnssec-keygen -f KSK -a ECDSAP384SHA384 -n ZONE kyau.net|2=dnssec-keygen -f KSK -a ECDSAP384SHA384 -n ZONE kyau.org}}
Run the following for each domain to include the keys in the zone files.
{{Console|1=for key in `ls Kkyau.net*.key`; do echo "\$INCLUDE $key" >> kyau.net.zone; done|2=for key in `ls Kkyau.org*.key`; do echo "\$INCLUDE $key" >> kyau.org.zone; done}}
Run a check on each zone.
{{Console|1=named-checkzone kyau.net /var/named/kyau.net.zone|2=named-checkzone kyau.org /var/named/kyau.org.zone}}
Sign each zone with the {{mono|dnssec-signzone}}.
{{Console|1=dnssec-signzone -A -3 $(head -c 1000 /dev/random {{!}} sha1sum {{!}} cut -b 1-16) -N INCREMENT -o kyau.net -t kyau.net.zone|2=dnssec-signzone -A -3 $(head -c 1000 /dev/random {{!}} sha1sum {{!}} cut -b 1-16) -N INCREMENT -o kyau.org -t kyau.org.zone}}
To update a zone at any point just edit the zone, check the zone and then re-sign as root {{mono|sudo -i}}.
{{Console|1=cd /var/named|2=dnssec-signzone -A -3 $(head -c 1000 /dev/random {{!}} sha1sum {{!}} cut -b 1-16) -N INCREMENT -o kyau.net -t kyau.net.zone|3=systemctl restart named}}
{{Warning|DO NOT increment the zone file this will be done automatically!}}
Modify the bind config to read from the signed zone files.
{{Console|title=/etc/named.conf|prompt=false|1=zone "kyau.net" IN {<br/>    type master;<br/>    file "kyau.net.zone.signed";<br/>    allow-update { none; };<br/>    notify no;<br/>};<br/><br/>zone "kyau.org" IN {<br/>    type master;<br/>    file "kyau.org.zone.signed";<br/>    allow-update { none; };<br/>    notify no;<br/>};}}
Make sure all is in order.
{{Console|1=named-checkconf /etc/named.conf}}
Next visit the domain registrar for the domain.
= {{Icon24|sitemap}} SQL =
Create a directory on the host machine for the nginx and sql server.
{{Console|1=sudo mkdir /www/sql /www/nginx}}
Make sure it has the right permissions.
{{Console|1=sudo chown -R kvm:kvm /www}}
Edit the sql virtual machine to mount the folder inside of the VM.
{{Console|1=sudo virsh edit sql}}
{{margin}}
{{Console|prompt=false|1=…<br/>  <nowiki><filesystem type='mount' accessmode='passthrough'></nowiki><br/>    <nowiki><source dir='/www/sql'/></nowiki><br/>    <nowiki><target dir='neutron-sql'/></nowiki><br/>  <nowiki></filesystem></nowiki><br/>…}}
Shutdown the virtual machine, then restart it back up.

Login to the sql virtual machine and create a directory for SQL.
{{Console|1=mkdir /sql}}
Mount the partition from the HOST system.
{{Console|1=mount neutron-sql /sql -t 9p -o trans{{=}}virtio}}
Also set this to mount on boot.
{{Console|title=/etc/fstab|prompt=false|1=neutron-sql	 /sql	9p	trans{{=}}virtio	0	0}}
After the directory is mounted make sure it has the right permissions.
{{Console|1=sudo chown mysql:mysql /sql|2=sudo chmod 770 /sql}}
Install {{mono|mariadb}}.
{{Console|1=pacaur -S mariadb}}
Initialize the SQL database directory.
{{Console|1=sudo mysql_install_db --user{{=}}mysql --basedir{{=}}/sql/base --datadir{{=}}/sql/db}}
Modify the mysql global config to support different basedir.
{{Console|title=/etc/mysql/my.cnf|prompt=false|1=…<br/>[mysqld]<br/>port        {{=}} 3306<br/>socket      {{=}} /run/mysqld/mysqld.sock<br/>datadir     {{=}} /sql/db<br/>…}}
Enable and start the systemd service.
{{Console|1=sudo systemctl enable mariadb|2=sudo systemctl start mariadb}}
Run the mysql post-install security check.
{{Console|1=sudo mysql_secure_installation}}

= {{Icon24|book-brown}} References =
<references/>
https://www.digitalocean.com/community/tutorials/how-to-setup-dnssec-on-an-authoritative-bind-dns-server--2

[[Category:Arch Linux]]