lucasteligioridis / raspbernetes Goto Github PK

Raspberry Pi Kubernetes Cluster

License: GNU General Public License v3.0

Makefile 35.66% Shell 64.34%

raspberry-pi kubernetes kubernetes-cluster kubernetes-setup immutable

raspbernetes's Introduction

Raspbernetes

Automated and repeatable method of deploying a headless Kubernetes stack onto a cluster of Raspberry Pis. Completely hands off experience from power on.

Detailed blog and guide posted onto my medium account:

Headless Kubernetes on 15 Raspberry Pis boot in under 8 minutes

Prerequisites

The following tools and pre-requisites must be available on the machine being used to build the SD cards:

Linux - Because of filesystem requirements
bash - 4.0+
make - 4.1+
kubectl - 1.20.4
4 Raspberry Pis (3 Masters and 1 Worker)

Applications

This stack is an opinionated way to deploy a home cluster with a HA design using haproxy and keepalived for cluster management.

See below for a list of the versions and applications used:

Raspbian - raspbian_lite-2020-02-14
Kubernetes - 1.20.4
Docker - 19.03.9
HA Proxy - 1.8.19
Keepalived - 2.0.10
Flannel - As per instructions on kubeadm installation page

Configuration

A short explanation of each environment variable that can be overridden.

Environment Settings

Mount device configuration:

MNT_DEVICE - Device name of SD slot on your local machine. (default: /dev/mmcblk0)

Raspberry Pi network and hostname configuration:

RPI_NETWORK_TYPE - Network option of choice. Either eth0 or wlan0. (default: wlan0)
RPI_HOSTNAME - Hostname for specific Raspberry Pi. (default: rpi-kube-master-01)
RPI_IP - Static IP to set. (default: 192.168.1.101)
RPI_GATEWAY - Generally your router ip. (default: 192.168.1.1)
RPI_DNS - Usually the same as your router unless you run a separate DNS (default: RPI_GATEWAY)
RPI_TIMEZONE - Local timezone. (default: Australia/Melbourne)

Kubernetes specific configuration:

KUBE_NODE_TYPE - Type of Kubernetes node. Either master or worker. (default: master)
KUBE_MASTER_VIP - Floating virtual IP (VIP) to use in keepalived. (default: 192.168.1.100)
KUBE_MASTER_IP_01 - IP of 1st master node to use in haproxy. (default: 192.168.1.101)
KUBE_MASTER_IP_02 - IP of 2nd master node to use in haproxy. (default: 192.168.1.102)
KUBE_MASTER_IP_03 - IP of 3rd master node to use in haproxy. (default: 192.168.1.103)

Network configuration if using Wifi. Accompanied with `RPI_NETWORK_TYPE` of `wlan0`:

WIFI_SSID - Local SSID to connect Wifi to. (default: n/a)
WIFI_PASSWORD - Password of above SSID to connect to Wifi using wpa_supplicant. (default: n/a)

Run

Be mindful of an SSH key that gets generated with this build that is stored in the output/ssh/ directory. This is not committed to git and should be treated safe like a normal private key. If you lose this key you will no longer be able to manage your stack. This is crucial for inter-cluster communication and remoting for management.

Build

Once appropriate above environment variables have been exported to suit your specific local environment, the below command will build an SD card with all automation scripts:

make build

Help

The help target is set as the default target, so display the above brief descriptions. Use the below command as is to print to the terminal:

make

raspbernetes's People

Contributors

Stargazers

Watchers

raspbernetes's Issues

Issues with keys for package repositories.

Had to manually go into each pi and pull down the correct signing keys for the package repositories.

Mount of filesystem fails (MNT_DEVICE)

I have an issue related to mount of devices.
This because of subname is not p1 and p2 but just 1 and 2. Refer lsblk print below.

LSBLK Printout

sdb 8:16 1 29,7G 0 disk
├─sdb1 8:17 1 256M 0 part /media/juhani/boot
└─sdb2 8:18 1 29,5G 0 part /media/juhani/rootfs

Could is be possible to add additional parameter (MNT_DEVICE_SUB or just SUB)?
I'm using external conf file with export commands and mount and umount issues
are only reasons why I have to modify the Makefile.

Content of the Makefile

mount: ## Mount the current SD device
sudo mount $(MNT_DEVICE)p1 $(MNT_BOOT)
sudo mount $(MNT_DEVICE)p2 $(MNT_ROOT)

.PHONY: unmount
unmount: ## Unmount the current SD device
sudo umount $(MNT_DEVICE)p1 || true
sudo umount $(MNT_DEVICE)p2 || true

Docker API version

Hi @lucasteligioridis,

Me again, so for the installation part its fixed with the export DOCKER_API_VERSION=1.39 pinning.
But now when you are on the boxes and try to run docker you get the same error and issue.

root@rpi-kube-master-01:/home/pi# docker ps
Error response from daemon: client version 1.40 is too new. Maximum supported API version is 1.39

after the export cmd the docker cmd works again.

We should add it in the users env by default, what do you think?

Kind regards,
Glenn

SD Card device name

My SD Card is detected as /dev/sda*

I have to modify the Makefile to change "p1" to "1" and "p2" to "2".

Using Pi4 with Multi-card reader connected via USB. OS is Raspbian Buster lite.

New docker images version, errors

In the last commit we changed the below versions for the docker installation

-docker_version="5:18.09.0~3-0~raspbian-stretch"
+docker_version="5:19.03.9~3-0~raspbian-stretch"

I have now tried it in combination of the project and it's now breaking the installation. I needed to reinstall all the raspberry PI's and used the project again.

I have retried it a couple of times with different issues but one that was interesting is that the Nodes overview only contained 1 Master Type node and the other Master nodes didn't have a type.

So I suggest to revert the change and add the export var again for the docker API so at least its working. Then from there we need to debug and see why its not working with the latest Docker version 5:19 but I think you have to do this as I'm not cable of understanding what's going wrong.