cisco-open / cluster-registry-controller Goto Github PK

Description:
Provider & distribution info about Rancher k8s cluster is missing.

k get clusters.clusterregistry.k8s.cisco.com -A -o wide
NAME                            ID                                     STATUS   TYPE    SYNCED   VERSION         PROVIDER   DISTRIBUTION   REGION   STATUS MESSAGE   SYNC MESSAGE
k3sandcalisti-managed-cluster   abcdfgew-123-1234-1234-abdpqra65e7e   Ready    Local            v1.24.14+k3s1                                                                                          
 

Feature request:
Add provider and distribution info for Rancher type cluster.

Feel free to reach out to me for any clarifications.

Is your feature request related to a problem? Please describe.
I am trying to sync VirtualServices for our istio MultiCluster mesh.
We need to mutate the namespace of the virtualService while synchronising.

Describe the solution you'd like to see
Currently, if we mutate object's namespace like so:

apiVersion: clusterregistry.k8s.cisco.com/v1alpha1
kind: ResourceSyncRule
metadata:
  name: istio-sync
spec:
  groupVersionKind:
    group: networking.istio.io
    kind: VirtualService
    version: v1beta1
  rules:
    - match:
      - labels:
        - matchLabels:
            published: "true"
      mutations:
        overrides:
          # Can't handle namespace change - controller deletes it. TODO: Raise the issue
          - path: /metadata/namespace
            type: replace
            value: edge
          - path: /spec/gateways/0
            type: replace
            value: edge/ingressgateway

the object is created and then immediately deleted.

Describe alternatives you've considered
The workaround could be to mirror all the namespaces, but then we have a policy that prevents istio gateways to consider virtualServices from foreign namespaces, so it's a no-go

Additional context
Not sure if this a bug or a feature request. I am planning to link PR with potential solution to this issue

Describe the bug

Bug on v0.2.7 version seen after my commit #33

A scenario that I want to discuss further. (I found this today)

Legend:
v0.2.2 old cluster registry - OCR
v0.2.7 new cluster registry - NCR

When a NCR is deployed OCR is currently a leader but NCR will not be ready unless a CA Bundle is generated and injected into ValidatingWebhook which leads to NCR never being ready(not getting leaderElection)

By default leaderElection is true in CR - https://github.com/cisco-open/cluster-registry-controller/blob/master/deploy/charts/cluster-registry/values.yaml#L54

This can be solved by
A) force leaderElection on NCR (dont know how to)
B) disable leaderElection as Webhook check is validated periodically here - https://github.com/cisco-open/cluster-registry-controller/blob/master/pkg/cert/renewer.go#L117

for example: while upgrading from OCR to NCR there a short window where two cluster-registries will be deployed, one will be a leader and other waiting to be a leader

v0.2.2 cluster-registry: /metrics as readiness probe comes up without webhook validation and marks as ready there by terminating the old pod of cluster-registry

v0.2.7 cluster-registry: /readyz as readiness probe is not marked as ready unless wehook is ready which is where the webhook awaits for leaderelection to generate ca Bundle and mark new pod of cluster registry as ready and there by kill the old pod

Steps to reproduce the issue

deploy helm chart with replicaset:2 and leaderelection enabled and check if both pods are ready

helm install --set replicaset=2 -n cluster-registry cluster-registry deploy/charts/cluster-registry

Expected behavior
to set readiness probe to ready(with webhook CA Bundle ready) before leaderElection

Screenshots

Continues to be in this state until cluster-registry-controller-controller-b8b499b68-llg4r is killed so that leaderElection is transferred to cluster-registry-controller-controller-b8b499b68-nkvcp

Additional context
doesn't work if there are two v0.2.7 cluster registries, one pod waits for another to hand over the lease. Sorry I missed checking this before committing into cluster-registry.

Quick Solution
If leaderElection: false then the above issue is not seen

This issue was automatically created by Allstar.

Security Policy Violation
Dismiss stale reviews not configured for branch master

This issue will auto resolve when the policy is in compliance.

Issue created by Allstar. See https://github.com/ossf/allstar/ for more information. For questions specific to the repository, please contact the owner or maintainer.