Installation with kubectl failes about hcloud-ip-floater HOT 5 CLOSED

from hcloud-ip-floater.

Ok, there was a misunderstanding. I though kubectl apply -k fip.yml might work but this only gives error: got file 'fip.yml', but 'fip.yml' must be a directory to be a root.

Renaming fip.yml to kustomization.yml, placing it into an empty directory and then doing kubectl apply -k directory/ did the trick!

Is the use of MetalLB really neccessary, or can the floating IP also be somehow manually assigned to a service or load balancer? I am currently wondering, how MetalLB will to the magic. To configure DNS accordingly, the assignment of IP address to load balancers should be deterministic and not random...

from hcloud-ip-floater.

MetalLB is not required, but AFAIK it's not possible to add IPs to services manually. It's possible to request a specific IP via .Spec.LoadBalancerIP (see here), but k8s still needs some component to respond to these requests.
Even if it were possible to manually assign IPs to services like this, you'd still have the problem that the nodes in your cluster don't know this IP, so traffic wouldn't be correctly forwarded. To fix this you could manually add the IP to all nodes. But this is all hacky.

That's basically what MetalLB does for you (if you ignore its BGP features). You give it a list of available IPs, it assigns them to requesting services and anounces them on the nodes.

As for DNS, that's what external-dns is for.

I'm also working on integrating directly with MetalLB (see #2) to make its use even easier, but for now you'd still have to configure it minimally.

from hcloud-ip-floater.

Thanks for clarification! I got this working with MetalLB and IP adresses are assigned correcly. Draining nodes triggers re-sheduling of contrainers and IP adresses to a different node as well, so this looks quite good :-)

Wouldn't it make more sense though if your component would automatically create floating IPs and assign them to load balancers when needed? Of course, this would only make sense if an integration with MetalLB works in a way where MetalLB does not have to be configured manually anymore (as suggested in #2).

And a last question about best practices: Are you using a new floating IP for every service you want the cluster to expose or are you using one floating IP which targets at a, e.g., traefik container, from where all further traffic is routed? Both would for sure work, but I am a bit unsure about what will be the best solution from a technical perspective.

from hcloud-ip-floater.

Wouldn't it make more sense though if your component would automatically create floating IPs and assign them to load balancers when needed?

This is more complicated than it sounds, for a couple of reasons:

1. AFAIK there is no portable way to detect a service stuck in pending state caused by running out of IPs. MetalLB does log this as an k8s Event, but checking this is not a portable solution because it would only work with MetalLB (and would be error-prone: the event only has the text "no available IPs", which we would have to match and could easily break with a future MetalLB release).
2. It would mean more responsibility: a bug in hcloud-ip-floater could incur real financial costs to the user. Hetzner's resource quotas would cap this, but it still wouldn't be something I could easily brush off.

That being said, this is something I'm looking into. So this might still happen in a future version.

The IP assignment, OTOH, should be left to some other tool (like MetalLB), because it requires more than just saying "here, take this IP", and would make this little tool not so little anymore.

And a last question about best practices: (...)

This is definitely beyond the scope of this project (let alone issue): it depends a lot on who's using and managing your cluster, what kind of services you are running and how the services relate to one another. There's no single answer. You should probably search the kubernetes forum or maybe stackoverflow for more info.

from hcloud-ip-floater.