Getting Started
- Install an operating system of your choice on the raspberry pi. For the commands that follow on this guide, I used a 64 bit version of Ubuntu server OS.
- Follow this guide if you're facing issues with the first step - https://ubuntu.com/tutorials/how-to-install-ubuntu-on-your-raspberry-pi#1-overview
Install Qemu and dependencies
$ sudo apt-get install qemu-kvm libvirt-daemon-system libvirt-clients bridge-utils
This installs the packages you might need. The repository config in the case of Ubuntu server was already baked in the image.sudo adduser `id -un` kvm
This adds your user to the kvm group. This is an optional step if you want to not use sudo.
Enable cgroups for memory
- Append
cgroup_enable=cpuset cgroup_enable=memory cgroup_memory=1to/boot/firmware/cmdline.txt(or equivalent in other OS's). It will look something like this:
root@Hypervisor:~# cat /boot/firmware/cmdline.txt
net.ifnames=0 dwc_otg.lpm_enable=0 console=serial0,115200 console=tty1 root=LABEL=writable rootfstype=ext4 elevator=deadline rootwait fixrtc cgroup_enable=cpuset cgroup_enable=memory cgroup_memory=1
- Reboot, and then run
virt-host-validate. Make sure all the responses are[PASS]ing.
Almost ready to go
- Download the boot media image for your Virtual Machine. In my POC, I used an Ubuntu server image ISO to keep things simple.
- Run the VM using your tool of choice,
rt-install --name test_1 --memory 2048 --disk size=5 --cdrom ubuntu-20.04.1-live-server-arm64.iso
This will give you a ASCII based installation interface for your VM running on Qemu. NOTE: We are using the default host based networking here and we will get to the bridging later.
Install your VM and get it running
Follow on screen instructions to get the OS installed on the VM.
Setting up the Wireless bridge
Raspberry PI's Wireless driver does not support native wireless bridges. Which is a known issue. There are many workarounds to get past this. People have attempted using ARP Proxy to success before.
For our setup, we're going to use the Slack's open source project, Nebula. Nebula is a scalable overlay networking tool with a focus on performance, simplicity and security. It lets you seamlessly connect computers anywhere in the world.
In my set-up, I already had Nebula running on a VM with a public static IP. To use Nebula in the best capacity, it is recommended to have that.
Generate certificates for each VM that you are running on the Raspberry Pi with Nebula and proceed to install Nebula agent on it as a non lighthouse node.
Unfortunately, at the time of this write-up; there is no process to automate installation of Nebula (I'm planning to work on one).
Check that Nebula's interface is working with the following command:
pi@test_1:~$ ip a | grep nebula -A4
3: nebula1: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1300 qdisc fq_codel state UNKNOWN group default qlen 500
link/none
inet 192.168.100.3/24 scope global nebula1
valid_lft forever preferred_lft forever
inet6 fe80::a045:1afa:b700:96dd/64 scope link stable-privacy
valid_lft forever preferred_lft forever
Profit?
userx000@Hypervisor:~$ virsh list --all
Id Name State
------------------------
1 test_1 running
If we attach to the VM with virsh attach test_1, we see that there's no one logged in except for the one with serial console.
pi@test_1:~$ w
04:29:23 up 20:42, 1 user, load average: 0.07, 0.02, 0.00
USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT
pi ttyAMA0 - 04:29 1.00s 0.20s 0.01s w
Now I try to SSH in using Nebula's private IP:
pi@testbed01:~$ w
04:29:23 up 20:44, 2 users, load average: 0.07, 0.02, 0.00
USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT
pi ttyAMA0 - 04:29 1.00s 0.20s 0.01s w
pi pts/0 192.168.100.2 04:29 10.00s 0.09s 0.09s -bash
Hit me up with quesitons, comments and feedback. I'll be happy to address them! My current twitter username is @oswalpalash