kubevirt / cloud-provider-kubevirt Goto Github PK
View Code? Open in Web Editor NEWKubernetes cloud-provider for KubeVirt
License: Apache License 2.0
Kubernetes cloud-provider for KubeVirt
License: Apache License 2.0
We should add a better documentation for the cloud provider with architecture diagrams.
The kubvirt-cloud-provider currently tries to set the external IP of a LoadBalancer service only at creation time (see https://github.com/kubevirt/cloud-provider-kubevirt/blob/master/pkg/cloudprovider/kubevirt/loadbalancer.go#L94,L108). In case that fails or the external IP changes, the cloud-provider will not reconcile the service.
A mechanism in the EnsureLoadbalancer method (https://github.com/kubevirt/cloud-provider-kubevirt/blob/master/pkg/cloudprovider/kubevirt/loadbalancer.go#L81,L87) is needed to update the external IP.
The KubeVirt CCM deployment should be able to mount a KubeVirt kubeconfig as a secret and a CCM configuration as a config map.
Hi Team,
The last release for this repository is in the month of Oct 2022. Will there be a new release of cloud-provider-kubevirt in the near future ?
Is there a document that refers to the timelines for the release of this repository.
Thanks
Sharath
As mentioned in this ticket: #15, EnsureLoadBalancer can only support an ExternalTrafficPolicy=Cluster
.
It would also require:
Is there a plan to fix this issue to have ExternalTrafficPolicy=Local
work ?
We for example have this ticket opened kubermatic/kubermatic#9022 by our customer that would like to use the Local
externalTrafficPolicy.
In order to gain more code maturity, need to add integration tests
Those tests would be done using cluster-api
clusters
For more info about cluster-api
, see:
clusterctl init --infrastructure kubevirt
#16 copies annotations from proxied service to proxy. It was primarily created for load balancers that use service annotations to provide metadata for endpoint wiring, for instance an external IP address.
This works in general, but the allow-shared-ip
of MetalLB is a special case. There are likely to be many of this nature. The value provided in the annotation is a lookup key for other services that an IP address should be allowed to be shared with. If two services have the same IP address request but do not contain the same sharing key, the services will not be allowed to share the address.
A nuance of this check is the services must also be managed by the same load balancer. This only makes sense.
When deployed in Gardener, the load balancer of a shoot is not the same as the external load balancer of the cluster. So blindly copying allow-shared-ip
will fail because there are two load balancers using the key.
There are several ways to make the key unique, what I am concerned with is how to do this without adding special cases for unrelated projects to the code.
Any thoughts, @gonzolino or @stoyanr?
once kubevirt implements kubevirt/kubevirt#8063 we should have an alternative way of retrieving the zone/region information that doesn't require access to the VMI's node. We'll be able to read the zone/region off of the VMI itself.
https://github.com/kubernetes/cloud-provider-gcp/blob/c2a51fc35cbf856dc4bab2a0d83ae46993ebe7f2/WORKSPACE#L78
https://github.com/kubernetes/cloud-provider-aws/blob/master/Dockerfile#L25
https://github.com/kubernetes/kubernetes/blob/7c46f40bdf89a437ecdbc01df45e235b5f6d9745/CHANGELOG/CHANGELOG-1.22.md#no-really-you-must-read-this-before-you-upgrade
Many of Kubernetes community folks decided to use Distroless as their base image. Alpine is a great distro but what about using Distroless? :)
As it stands, EnsureLoadBalancer can only support an ExternalTrafficPolicy of Cluster
. This is unusable when the client IP address of external traffic must be the actual client address and not the masquerade address of the CNI.
Supporting this will require:
Open to suggestions on what information is available to the CCM to more selectively label the proxies.
Currently we pass to node the interface name "default". We have to pass all reported ip addresses with appropriate types.
https://github.com/kubevirt/cloud-provider-kubevirt/blob/main/pkg/provider/instances_v2.go#L138
We should add information about supported k8s versions or policy to the main README document
Add integration covering al the use cases, one solution would be to copy mechanism from capk.
In cases the cloud-controller-manager can't access the nodes, the user would need to have option to disable this functionality
Like described in README
for our set up we use infrastructure cluster that hosts KubeVirt VMs (previously known as UnderKube) and tenant cluster that uses those VMs as kubernetes nodes (OverKube).
Nodes that host UnderKube VMs are located in two different availability zones and labeled accordingly.
Recently we were testing VM migration scenarios and found out that labels (such as zone) on OverKube nodes always remain the same despite VMs were migrated between racks, meaning different zones.
What we expect: labels are being changed dynamically when KubeVirt VMs on an UnderKube cluster are migrated between availability zones and correctly propagated to kubernetes nodes on a tenant cluster.
I can provide more info if needed about migration steps etc
We encountered the following error when attempting to add a new node to a tenant cluster:
Node "xxx" is invalid: spec.providerID: Forbidden: node updates may not change providerID except from "" to valid, requeuing
Initially, we used another cloud provider to add nodes to the cluster. However, when we attempted to re-add them, we received the above error message. It appears that when updating the node manifest, the cloud-provider-kubevirt attempts to update the providerID field value with providerID: kubevirt://nodeName
Is it possible to disable updates to this field if it already exists?"
In order to support adding the new topology
labels to overkube nodes, as opposed to the deprecated failure-domain
labels (see also #11), the CCM should be revendored to a version of kubernetes newer than 1.16. Unfortunately, this is hard to do since kubevirt.io/client-go is still based on 1.16 and there are conflicts, see also #10.
There are two ways this issue could be resolved:
kubevirt.io/client-go
is revendored to a newer kubernetes (or actively contribute to this).kubevirt.io/client-go/kubecli
and use a different client for reading / writing Kubevirt resources. A good alternative could be sigs.k8s.io/controller-runtime/pkg/client
. In this case, only the dependency to kubevirt.io/client-go/api/v1
would remain, which should not be an issue.The second approach requires more changes to the CCM itself, but can be done without depending on any changes to kubevirt.io/client-go
. As an additional bonus, this would remove the dependency to glog described in #10 and would enable vendoring of kubevirt.io/client-go
newer than 0.26.5.
@afritzler @gonzolino What do you think?
The current implementation doesn't work as expected, it fails with:
that match label selector "", field selector "spec.Hostname=testmachine": field label not supported: spec.Hostname
A ticket to track an upstream work in our Zenhub:
#ref: #78
By default a namespace is passed to the cloud provider from a kubeconfig. We should allow to read namespace from the CCM configuration as well.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.