| title | author |
|---|---|
Just another way to setup k3s on Proxmox |
Christian ∆ Benstein |
This repo containers just another way to set up k3s on a Proxmox server in LXC containers using Terraform and Ansible. I have chosen not to use cool modules or automation and rely solely on out-of-the-box options to make this a bit future-proof.
This automation is heavily based on: https://betterprogramming.pub/rancher-k3s-kubernetes-on-proxmox-containers-2228100e2d13
The following is not included, and you need to set it up yourself on your local machine or on Proxmox.
Make sure to have the following tools installed:
- Terraform
- Ansible
- Kubectl
This repo sets up k3s on Proxmox, but you need to install Proxmox yourself. I'm using debian-12-standard_12.0-1_amd64 because it came the most ready out of the box, and it works with the k3s Traefik ingress. You will need to download the LXC image template on Proxmox yourself. See: https://pve.proxmox.com/wiki/Linux_Container
This repo consists of 3 parts:
- Installing the
infrawith Terraform - Creating a k3s
clusterwith Ansible - Creating a
testdeployment usingkubectl
You can use the included Makefile for quick setup. Follow along to create the cluster step by step. Please allow some time for each step to complete. Setting up the infra can take around 2 minutes, deploying k3s can take around 2Ð10 minutes (before the cluster is in a ready state).
This is taken care of by Terraform. Make sure to edit the following var's in terraform/vars.tf:
ssh_keyThe SSH key that will be used to connect to your containerspasswordthe password that will be used to connect to your containerstarget_nodename of your Proxmox nodeproxmox_serverthe API URL of your serverostemplatethe LXC container template you are usingproxmox_passwordroot password for Proxmox. This is needed because we are setting up nodes with elevated permissions and that is currently only possible by connecting with theroot@pamusermaster_vmidstarting number for thevmidof the containerworker_vmidstarting number for thevmidof the container
You can find the IP config for the nodes in the terraform/lcx_k3s.tf file. Edit to your own content. In this setup the master will be on 10.1.1.221 and the workers on 10.1.1.231-233
Now you can run make infra which will run:
$ terraform -chdir=terraform init
$ terraform -chdir=terraform applyMake sure the ansible/hosts is up-to-date with the IP's that you have chosen for your infra
Cluster creation is done by Ansible using the make cluster command, which will run:
$ ANSIBLE_CONFIG=ansible/ansible.cfg \
ansible-playbook ansible/playbook_configure_k3s.yaml \
--inventory ansible/hostsThe setup copies the kubeconfig of the server, and put's a modified copy in ansible/kubeconfig_k3s.yaml. At this point the cluster should be ready, and you can verify this by running:
$ KUBECONFIG=ansible/kubeconfig_k3s.yaml kubectl get nodes
NAME STATUS ROLES AGE VERSION
k3s-worker03 Ready <none> 5m v1.27.3+k3s1
k3s-worker01 Ready <none> 5m v1.27.3+k3s1
k3s-worker02 Ready <none> 5m v1.27.3+k3s1
k3s-master01 Ready control-plane,master 6m v1.27.3+k3s1I have included a simple test deployment to test your new cluster. This can be deployed with make test which will run:
$ KUBECONFIG=ansible/kubeconfig_k3s.yaml \
kubectl apply -f tests/test-deploymentAfter a short while this should serve a webpage on any of the worker IP's:
$ curl http://10.1.1.231
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>make destroy will burn down your cluster (:
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent