Running Virtual Machines on Raspberry Pi 4 with Wireless Bridging using KVM

Running Virtual Machines on Raspberry Pi 4 with Wireless Bridging using KVM

Getting Started

  1. Install an operating system of your choice on the raspberry pi. For the commands that follow on this guide, I used a 64 bit version of Ubuntu server OS.
  2. Follow this guide if you're facing issues with the first step -

Install Qemu and dependencies

  1. $ sudo apt-get install qemu-kvm libvirt-daemon-system libvirt-clients bridge-utils
    This installs the packages you might need. The repository config in the case of Ubuntu server was already baked in the image.
  2. sudo adduser `id -un` kvm
    This adds your user to the kvm group. This is an optional step if you want to not use sudo.

Enable cgroups for memory

  1. Append cgroup_enable=cpuset cgroup_enable=memory cgroup_memory=1 to /boot/firmware/cmdline.txt (or equivalent in other OS's). It will look something like this:
root@Hypervisor:~# cat /boot/firmware/cmdline.txt
net.ifnames=0 dwc_otg.lpm_enable=0 console=serial0,115200 console=tty1 root=LABEL=writable rootfstype=ext4 elevator=deadline rootwait fixrtc cgroup_enable=cpuset cgroup_enable=memory cgroup_memory=1
  1. Reboot, and then run virt-host-validate. Make sure all the responses are [PASS]ing.

Almost ready to go

  1. Download the boot media image for your Virtual Machine. In my POC, I used an Ubuntu server image ISO to keep things simple.
  2. Run the VM using your tool of choice, rt-install --name test_1 --memory 2048 --disk size=5 --cdrom ubuntu-20.04.1-live-server-arm64.iso
    This will give you a ASCII based installation interface for your VM running on Qemu. NOTE: We are using the default host based networking here and we will get to the bridging later.

Install your VM and get it running

Follow on screen instructions to get the OS installed on the VM.

Setting up the Wireless bridge

Raspberry PI's Wireless driver does not support native wireless bridges. Which is a known issue. There are many workarounds to get past this. People have attempted using ARP Proxy to success before.
For our setup, we're going to use the Slack's open source project, Nebula. Nebula is a scalable overlay networking tool with a focus on performance, simplicity and security. It lets you seamlessly connect computers anywhere in the world.
In my set-up, I already had Nebula running on a VM with a public static IP. To use Nebula in the best capacity, it is recommended to have that.
Generate certificates for each VM that you are running on the Raspberry Pi with Nebula and proceed to install Nebula agent on it as a non lighthouse node.
Unfortunately, at the time of this write-up; there is no process to automate installation of Nebula (I'm planning to work on one).

Check that Nebula's interface is working with the following command:

pi@test_1:~$ ip a | grep nebula -A4
3: nebula1: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1300 qdisc fq_codel state UNKNOWN group default qlen 500
    inet scope global nebula1
       valid_lft forever preferred_lft forever
    inet6 fe80::a045:1afa:b700:96dd/64 scope link stable-privacy
       valid_lft forever preferred_lft forever


userx000@Hypervisor:~$ virsh list --all
 Id   Name     State
 1    test_1   running

If we attach to the VM with virsh attach test_1, we see that there's no one logged in except for the one with serial console.

pi@test_1:~$ w
 04:29:23 up 20:42,  1 user,  load average: 0.07, 0.02, 0.00
USER     TTY      FROM             LOGIN@   IDLE   JCPU   PCPU WHAT
pi       ttyAMA0  -                04:29    1.00s  0.20s  0.01s w

Now I try to SSH in using Nebula's private IP:

pi@testbed01:~$ w
 04:29:23 up 20:44,  2 users,  load average: 0.07, 0.02, 0.00
USER     TTY      FROM             LOGIN@   IDLE   JCPU   PCPU WHAT
pi       ttyAMA0  -                04:29    1.00s  0.20s  0.01s w
pi       pts/0    04:29   10.00s  0.09s  0.09s -bash

Hit me up with quesitons, comments and feedback. I'll be happy to address them! My current twitter username is @oswalpalash

Show Comments