Code Monkey home page Code Monkey logo

docker-k3s-setup's Introduction

Using docker as backend for k3s

That is very easy. Just provide "--docker" as the argument. The only problem is that the default k3s-server.service loads an enviromentfile /etc/rancher/k3s/server.conf but it can not expand its values.

Since we do also want to ensure that the docker.service is started before, it is a good idea to copy k3s.service to a new file named k3s.service

Interestingly, when stopping the "k3s server" via systemctl-stop it does not kill the docker containers started by k3s. So we have added a script docker-stop-k3s which gets called automatically via ExecStopPost

Directly using kubectl as a non-root user

When installing kubectl via k3s it has a default KUBECONFIG being set to /etc/rancher/k3s/k3s.yaml. However the chmod bits for that file are 0400, so users can not access it.

Instead "k3s server" may get --write-kubeconfig-mode=0640

But even this is not working all the time, so some ExecStartPost are added

ExecStartPost=/bin/chgrp docker /etc/rancher/k3s/k3s.yaml
ExecStartPost=/bin/chmod 0640 /etc/rancher/k3s/k3s.yaml

Additionally, some /etc/sudoers.d/k3s is installed, so that users in the "docker" unix group can start/stop k3s containers.

Using nginx as the frontend for k3s

Forget about other how-to ... the real one comes from Suse.

Of course, make sure to have added --no-deploy=traefik to "k3s server".

But instead of kubectl, you can use a helm chart reference in k3s. Actually traefik is deployed via its helm chart in k3s, have a look at

sudo head /var/lib/rancher/k3s/server/manifests/traefik.yaml

When the no-deploy=traefik is in place, that file is ignored. Instead we deploy the ingress-nginx.yaml which comes from that suse ticket.

Using metallb to expose services

Again, make sure to have added --no-deploy=servicelb to "k3s server"

Since we are installing both nginx and metallb, we want to make sure that metallb gets run before nginx. That's why there are two files here:

manifests/setup-10-metallb-system.yaml
manifests/setup-20-ingress-nginx.yaml

Some kube tricks

There is an additional file kube being installed into local/bin

This is a wrapper around kubectl and helm/helmfile plus a few tricks.

  • run finalizers of a namespace of pvc ("kube fin ns" and "kube fin pvc")
    • this is being used when a "kube del ns" is blocked somehow
  • exec a command in a container as root user ("kube root -it ...")
    • this works for both docker and container backend
  • and just find the container names ("kube get con" or "kube con -A")
    • just a thin wrapper around get-pod-yaml
  • find and update ip addresses ("kube get ip" and "kube get ipp")
    • ipp writes /etc/hosts but the service name must be there already
  • find and del volumes by namespace ("kube get vol" and "kube del vol")
    • volume is a "pv" but the pv-list is not global anymore
  • shorthand "kube chart" for "kube helm search repo"

How to install?

make install
# make uninstall

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.