costela / hcloud-ip-floater Goto Github PK

View Code? Open in Web Editor NEW

52.0 6.0 9.0 187 KB

k8s controller for Hetzner Cloud floating IPs

License: GNU General Public License v3.0

Dockerfile 1.35% Go 98.65%

kubernetes hcloud hetzner floating-ip k8s k8s-controller golang go failover

hcloud-ip-floater's Introduction

Hetzner Cloud™ IP Floater

This small kubernetes controller manages the attachment of hetzner cloud ("hcloud") floating IPs to kubernetes nodes.

It watches for changes to kubernetes LoadBalancer services, chooses one of the nodes where its pods are scheduled and attaches its assigned floating IP to the selected node.

The service IP assignment is left to a separate component, like MetalLB.

Installation

The controller can be installed to a cluster using e.g. kustomize. Simply kubectl apply -k the following kustomization.yaml:

namespace: hcloud-ip-floater
bases:
  - github.com/costela/hcloud-ip-floater/deploy?ref=v0.1.6
secretGenerator:
  - name: hcloud-ip-floater-secret-env
    literals:
      - HCLOUD_IP_FLOATER_HCLOUD_TOKEN=<YOUR HCLOUD API TOKEN HERE>

The provided deployment manifest expects a secret named hcloud-ip-floater-secret-env to exist, which is the recommended location for storing the hcloud API token.

It's also possible to provide a configMapGenerator called hcloud-ip-floater-config-env with the non-secret options listed in the configuration options section below.

⚠ in order for the controller to attach IPs to the hcloud nodes, the k8s nodes must use the same names as in hcloud.

Configuration options

Either as command line arguments or environment variables.

`--hcloud-token` or `HCLOUD_IP_FLOATER_HCLOUD_TOKEN` (required)

API token for hetzner cloud access.

`--service-label-selector` or `HCLOUD_IP_FLOATER_SERVICE_LABEL_SELECTOR`

Service label selector to use when watching for kubernetes services. Any services that do not match this selector will be ignored by the controller.

Default: hcloud-ip-floater.cstl.dev/ignore!=true

`--floating-label-selector` or `HCLOUD_IP_FLOATER_FLOATING_LABEL_SELECTOR`

Label selector for hcloud floating IPs. Floating IPs that do not match this selector will be ignored by the controller.

Default: hcloud-ip-floater.cstl.dev/ignore!=true

`--log-level` or `HCLOUD_IP_FLOATER_LOG_LEVEL`

Log output verbosity (debug/info/warn/error)

Default: warn

hcloud-ip-floater's People

Stargazers

Watchers

Forkers

mstarostik simonsystem rbnis kalkspace eplightning truegoric o-be-one guettli

hcloud-ip-floater's Issues

Supporting multiple floating ips in a cluster

I was thinking it would be VERY nice if this could handle "ALL" public ips assigned to a cluster (so we have HA on them).
f.ex. We have 1 public ip - which lands on traefik (a daemonset on all nodes).. this serves all ingress http/https..
BUT we also have a few services that can't be served through traefik.. so we need an extra public ip for those.

I was thinking that if one could instead give it a "list of floating ips"..
to also work with floating ips on dedicated servers (robot api) - it could be done by f.ex. having a label with ip as value (on loadbalancer service objects) - so if one defines such a label - it should ONLY "sync that ip" to the node THAT loadbalancer-service's pod is on.. WDYT ?

That approach should work for both hcloud and dedicated servers AFAIK - and without such a label on the object - the would do "as it does not" - which means point all floating ips it manages to the same hcloud-server.

support for kubernets 1.20

Im unable to get this to work on kubernetes 1.20, its unable to assign the ip's. is support for this coming in future?
here is the current error:

W1230 10:06:57.131557       1 reflector.go:289] k8s.io/[email protected]+incompatible/tools/cache/reflector.go:94: watch of *v1.Pod ended with: too old resource version: 3584 (4248)
W1230 10:27:05.142240       1 reflector.go:289] k8s.io/[email protected]+incompatible/tools/cache/reflector.go:94: watch of *v1.Pod ended with: too old resource version: 7132 (11402)
[root@prod-ghorofa-124-fra1o62b7tq1tp relations]# kubectl  logs -n hcloud-ip-floater hcloud-ip-floater-6cb5d5c58b-wtvkd 
W1230 10:06:57.131557       1 reflector.go:289] k8s.io/[email protected]+incompatible/tools/cache/reflector.go:94: watch of *v1.Pod ended with: too old resource version: 3584 (4248)
W1230 10:27:05.142240       1 reflector.go:289] k8s.io/[email protected]+incompatible/tools/cache/reflector.go:94: watch of *v1.Pod ended with: too old resource version: 7132 (11402)

Swarm and the old floater ip

Hi there,

I noticed your other now deprecated project:
https://github.com/costela/hetzner-ip-floater

I actually preferred the old approach.

I just wrote this for swarm, and it allows me to setup clusters intuitively in seconds on swarm, that I much prefer over using the patched together k8 approach:
https://github.com/sfproductlabs/roo

I was wondering if:

you added the floating address to each machine (so it was pre-set with all your floating IPs like using sudo ip addr add xxx.xxx.183.82 dev eth0)
if you did, whether you noticed irregularities in performance
if you didn't what else you did? as the code just assigned the floating cloud on the hetzner api, not on the machine (did you use an additional bash script?) would love to know your thoughts....

Thanks for your help!
Andrew

Installation with kubectl failes

Hi,

when trying to install as described in you Readme file with

echo "namespace: hcloud-ip-floater
 bases:
   - github.com/costela/hcloud-ip-floater/deploy?ref=v0.1.4
 secretGenerator:
   - name: hcloud-ip-floater-secret-env
     literals:
       - HCLOUD_IP_FLOATER_HCLOUD_TOKEN=...." > fip.yml
kubectl apply -f fip.yml

I get the following error:

error: error validating "test.yml": error validating data: [apiVersion not set, kind not set]; if you choose to ignore these errors, turn validation off with --validate=false

Any ideas how to fix this?

add optional MetalLB integration

We could optionally export the floating IPs under our control to MetalLB, to automate its configuration.

Dependabot can't resolve your Go dependency files

Dependabot can't resolve your Go dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

github.com/costela/hcloud-ip-floater/internal/config: cannot find module providing package github.com/costela/hcloud-ip-floater/internal/config
	github.com/costela/hcloud-ip-floater/internal/fipcontroller: cannot find module providing package github.com/costela/hcloud-ip-floater/internal/fipcontroller
	github.com/costela/hcloud-ip-floater/internal/servicecontroller: cannot find module providing package github.com/costela/hcloud-ip-floater/internal/servicecontroller

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

Could not sync Floating IPs...

Hi,
my pod didn't sync floating IP, some clue to solve it ?

kubectl logs hcloud-ip-floater-74f496fdb7-qpjl6 -n hcloud-ip-floater
time="2022-03-13T11:14:55Z" level=error msg="could not sync floating IPs" component=fipcontroller error="Get https://api.hetzner.cloud/v1/floating_ips?page=1: dial tcp 213.239.246.1:443: i/o timeout"
time="2022-03-13T11:20:25Z" level=error msg="could not sync floating IPs" component=fipcontroller error="Get https://api.hetzner.cloud/v1/floating_ips?page=1: dial tcp 213.239.246.1:443: i/o timeout"
time="2022-03-13T11:25:55Z" level=error msg="could not sync floating IPs" component=fipcontroller error="Get https://api.hetzner.cloud/v1/floating_ips?page=1: dial tcp 213.239.246.1:443: i/o timeout"
time="2022-03-13T11:31:25Z" level=error msg="could not sync floating IPs" component=fipcontroller error="Get https://api.hetzner.cloud/v1/floating_ips?page=1: dial tcp 213.239.246.1:443: i/o timeout"
time="2022-03-13T11:36:55Z" level=error msg="could not sync floating IPs" component=fipcontroller error="Get https://api.hetzner.cloud/v1/floating_ips?page=1: dial tcp 213.239.246.1:443: i/o timeout"
time="2022-03-13T11:42:25Z" level=error msg="could not sync floating IPs" component=fipcontroller error="Get https://api.hetzner.cloud/v1/floating_ips?page=1: dial tcp 213.239.246.1:443: i/o timeout"
time="2022-03-13T11:47:55Z" level=error msg="could not sync floating IPs" component=fipcontroller error="Get https://api.hetzner.cloud/v1/floating_ips?page=1: dial tcp 213.239.246.1:443: i/o timeout"

Thanks !!

example on a service/ip

care to provide a concrete example of how the service and floating ip should be labelled?

Runtime error: Invalid memory address or nil pointer dereference

Hello and many thanks for this useful application! Recently I upgraded to the latest version and now I got this error message followed by a stack trace and the pod keeps CrashLooping:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x6e7eba]

Let me know, if you need any further informations.

Floating Ip Configuration

Hey @costela Is there additional configuration that one needs to make on the nodes before routing works correctly or does the hcloud-ip-floater 'just work'
The additional configuration in this case is assigning the floating ip to eth0 for all the nodes

Add support for other cloud providers

Hey @costela
I would like to propose extending ip-floater to support other clouds.
This could be done gradually based on provider support and ease of implementation.
We could probably start with the following clouds;

DigitalOcean
Linode
Vultr
Maybe:
OVH
Scaleway

Digitalocean could provide the quickest POC as it's API is most similar to Hetzner's.
Based on code structure,
I think most changes will be made here https://github.com/costela/hcloud-ip-floater/tree/master/internal/fipcontroller with an additional provider flag (default: hcloud)
DigitalCean, Linode & Vultr also use an api_token similar to OVH cloud.

wrong Node assigned floating IP

So with

costela/hcloud-ip-floater:latest

and weave-net, 2.8.1, and metallb v0.13.7:

in the service description:

  Normal  nodeAssigned  37m (x2 over 37m)  metallb-speaker     announcing from node "prod-ghorofa-ks8-hundredeighteenszcdrp2jdc" with protocol "layer2"

but with hcloud, the wrong node is set:

time="2022-11-27T10:45:59Z" level=info msg="attached floating IP" component=fipcontroller fip=XXX.XXX node=prod-ghorofa-ks8-hundredeighteeny0l5mpmxff

what is going on here? it seems to work fine with other CNI's i think, since access from outside is fine.

implement service controller

After the discussion on metallb/metallb#637, it seems the dependency on metalLB is unnecessary if some requirements are met, namely:

the cluster should run kube-proxy in IPVS mode (kube-router with service-proxy enable should also work, but is currently untested).
hcloud-ip-floater should implement a service controller for IP assignment. This can bypass metalLB IP pools and assign IPs to LoadBalancer services directly from those known to hcloud-ip-floater.

If this works, we can probably deprecate #2.

Add support for dedicated hetzner servers (robot api)

It would be great if this code also worked for dedicated servers with a floating ip.. It would need to implement this API instead https://docs.hetzner.com/robot/dedicated-server/robot-interfaces/ - an example of code doing this: https://github.com/ahes/hetzner-api-failover

But this project with following externalip handed out by metallb "is just what I need".. :)

Integration with nginx ingress?

Hi, and thanks for your work.

I see you mention metallb, would this work with nginx ingress also?

If I understand correctly your controller, we just need to publish a svc kind LoadBalancer, and your controller takes care of the rest :)

If it is true, then I'll update the documentation with this possibility.

arm64 image

will there be some ready made arm64 images? CAX servers are available in falkenstein or so.

no matches for kind "ClusterRole" in version "rbac.authorization.k8s.io/v1beta1"

Hi,

got an issue deplyoing it:

kubectl apply -k .
namespace/hcloud-ip-floater created
serviceaccount/hcloud-ip-floater created
secret/hcloud-ip-floater-secret-env-6c674bdb66 created
deployment.apps/hcloud-ip-floater created
unable to recognize ".": no matches for kind "ClusterRole" in version "rbac.authorization.k8s.io/v1beta1"
unable to recognize ".": no matches for kind "ClusterRoleBinding" in version "rbac.authorization.k8s.io/v1beta1"

kubectl version

Client Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.4", GitCommit:"e6c093d87ea4cbb530a7b2ae91e54c0842d8308a", GitTreeState:"clean", BuildDate:"2022-02-16T12:38:05Z", GoVersion:"go1.17.7", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.7+k3s1", GitCommit:"8432d7f239676dfe8f748c0c2a3fabf8cf40a826", GitTreeState:"clean", BuildDate:"2022-02-24T23:03:47Z", GoVersion:"go1.16.10", Compiler:"gc", Platform:"linux/amd64"}

kubectl api-versions

acid.zalan.do/v1
admissionregistration.k8s.io/v1
agent.k8s.elastic.co/v1alpha1
apiextensions.k8s.io/v1
apiregistration.k8s.io/v1
apm.k8s.elastic.co/v1
apm.k8s.elastic.co/v1beta1
apps/v1
authentication.k8s.io/v1
authorization.k8s.io/v1
autoscaling/v1
autoscaling/v2beta1
autoscaling/v2beta2
batch/v1
batch/v1beta1
beat.k8s.elastic.co/v1beta1
certificates.k8s.io/v1
coordination.k8s.io/v1
discovery.k8s.io/v1
discovery.k8s.io/v1beta1
elasticsearch.k8s.elastic.co/v1
elasticsearch.k8s.elastic.co/v1beta1
enterprisesearch.k8s.elastic.co/v1
enterprisesearch.k8s.elastic.co/v1beta1
events.k8s.io/v1
events.k8s.io/v1beta1
flowcontrol.apiserver.k8s.io/v1beta1
helm.cattle.io/v1
k3s.cattle.io/v1
kibana.k8s.elastic.co/v1
kibana.k8s.elastic.co/v1beta1
maps.k8s.elastic.co/v1alpha1
metrics.k8s.io/v1beta1
networking.k8s.io/v1
node.k8s.io/v1
node.k8s.io/v1beta1
policy/v1
policy/v1beta1
rabbitmq.com/v1beta1
rbac.authorization.k8s.io/v1
scheduling.k8s.io/v1
storage.k8s.io/v1
storage.k8s.io/v1beta1
traefik.containo.us/v1alpha1
upgrade.cattle.io/v1
v1

I did manage to deploy it, by changing rbac.authorization.k8s.io/v1beta1 to rbac.authorization.k8s.io/v1, but not so sure if that was healthy to do?

Cheers
Michael

costela / hcloud-ip-floater Goto Github PK

hcloud-ip-floater's Introduction

Hetzner Cloud™ IP Floater

Installation

Configuration options

--hcloud-token or HCLOUD_IP_FLOATER_HCLOUD_TOKEN (required)

--service-label-selector or HCLOUD_IP_FLOATER_SERVICE_LABEL_SELECTOR

--floating-label-selector or HCLOUD_IP_FLOATER_FLOATING_LABEL_SELECTOR

--log-level or HCLOUD_IP_FLOATER_LOG_LEVEL