Building a K3S Cluster using Raspberry Pis

If you are going to build a Pi Cluster using K3S, the easiest way to get up and running is to use the pre-baked setup script from k3s.io

However, as with all things linux/pi… there’s a few steps you might want to do first.

Initially flash your pis all with the Raspberry Pi Imager using Raspberry Pi OS (Lite) 64Bit

Use the Settings cog to enable SSH on all the pis (I didn’t use wifi as I am cabling mine) and give them useful names like

kmaster
knodea
knodeb
knodec

That will make ssh-ing nice and easy (ssh pi@knodemaster.local etc)

In order (starting with master) plug in your network cable, then plug the pi in, so it is good to go. Give it a few second to start up, then ssh into the pi (again, start with master)

you will want to edit cmdline.txt before you try and install k3s or it will fail.

Run sudo nano /boot/cmdline.txt and then append cgroup_memory=1 cgroup_enable=memory to the end of the file. Now restart your pi with sudo shutdown -r now

Again, give it a few seconds to restart, then ssh back into it. On your MASTER node, run the script from k3s.io:

curl -sfL https://get.k3s.io | sh -
# Check for Ready node, takes ~30 seconds
k3s kubectl get node

That will take a few seconds to run. Once it completes you should have a k3s master node running. Next you need to find out the IP address of this node. Run ifconfig for this and not the IP address of this pi down.

Next we need to get the token for this master node. To do that run sudo cat /var/lib/rancher/k3s/server/node-token

Now, repeat these steps for each node.

1) SSH onto the Node Pi

2) Run sudo nano /boot/cmdline.txt and then append cgroup_memory=1 cgroup_enable=memory to the end of the file. Now restart your pi with sudo shutdown -r now

3) SSH back onto the Node Pi after thr restart

4) Replace your IP address and token in the below script and run it on the node

curl -sfL https://get.k3s.io | K3S_URL=https://YOUR_PI_IP_ADDRESS:6443 K3S_TOKEN=YOUR_REALLY_LONG_PI_TOKEN sh -

and then wait for this to complete.

Once that’s done for all nodes, ssh onto your master pi. Your kubeconfig isn’t world visible, but we need to see some information there, so run this:

mkdir ~/.kube 2> /dev/null
sudo k3s kubectl config view --raw > "$KUBECONFIG"
chmod 600 "$KUBECONFIG"

This will create you a .kube folder in your user root directory and you can see kubeconfig in there.

To keep this after a restart add KUBECONFIG=~/.kube/config to the bottom of your .bashrc file.

Now you can run cat .kube/config and get the values needed to populate your local kube config file so you can run kubectl commands on your pi cluster from your local machine. (You will have to update the IP address from the ‘default’ cluster to be the master node’s IP address instead of localhost.

NB: You can’t run the install script with https://kmaster.local:6443 for whatever reason, it NEEDS to be the IP address of the master node, not it’s network name.

Anyway, hope that helps.

Happy Hacking.