Code Monkey home page Code Monkey logo

controller-runtime-example's Issues

make deploy doesn’t work b/c of services

with a virgin kcp cluster

$ make deploy
/tmp/controller-runtime-example/bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases
cd config/manager && /tmp/controller-runtime-example/bin/kustomize edit set image controller=controller:latest
/tmp/controller-runtime-example/bin/kustomize build config/default | kubectl apply -f -
namespace/controller-runtime-example-system created
serviceaccount/controller-runtime-example-controller-manager created
role.rbac.authorization.k8s.io/controller-runtime-example-leader-election-role created
clusterrole.rbac.authorization.k8s.io/controller-runtime-example-kcp-manager-role created
clusterrole.rbac.authorization.k8s.io/controller-runtime-example-manager-role created
clusterrole.rbac.authorization.k8s.io/controller-runtime-example-metrics-reader created
clusterrole.rbac.authorization.k8s.io/controller-runtime-example-proxy-role created
rolebinding.rbac.authorization.k8s.io/controller-runtime-example-leader-election-rolebinding created
clusterrolebinding.rbac.authorization.k8s.io/controller-runtime-example-kcp-manager-rolebinding created
clusterrolebinding.rbac.authorization.k8s.io/controller-runtime-example-manager-rolebinding created
clusterrolebinding.rbac.authorization.k8s.io/controller-runtime-example-proxy-rolebinding created
configmap/controller-runtime-example-manager-config created
apiexport.apis.kcp.dev/controller-runtime-example-data.my.domain created
apiresourceschema.apis.kcp.dev/controller-runtime-example-today.widgets.data.my.domain created
unable to recognize "STDIN": no matches for kind "Service" in version "v1"
unable to recognize "STDIN": no matches for kind "Deployment" in version "apps/v1"
make: *** [Makefile:112: deploy] Error 1

When compute is being added the Deployment API is added per default by the syncer. This is not the case of the Service API. This is not required by the core functionalities of the controller but referenced in the manifests for monitoring, auth-proxy and webhook. Of the three only auth-proxy is activated by default. As Robin mentioned this could get commented out.
Possibly the readme or the kustomization.yaml could be enhanced for making the user aware of it.

controller runtime client list fails with error "Index with name cluster does not exist"

hi,
When I use latest https://github.com/kcp-dev/controller-runtime, the controller runtime client List is fails with error "Index with name cluster does not exist".

Steps to reproduce:
point to latest kcp-dev/controller-runtime, make change to compile the code and test the controller-runtime-example. you can use this branch to test https://github.com/prembhaskal/kcp-dev-controller-runtime-example/tree/client-list-issue


% export KUBECONFIG=/tmp/kcp.kubeconfig
% kubectl ws create prem-org --type organization
% kubectl ws prem-org

 % cat /tmp/data-api-binding.yaml 
apiVersion: apis.kcp.io/v1alpha1
kind: APIBinding
metadata:
  name: data.my.domain
spec:
  reference:
    export:
      name: data.my.domain
      path: root
  permissionClaims:
    - resource: "secrets"
      all: true
      state: Accepted
    - resource: "configmaps"
      all: true
      state: Accepted
    - resource: "namespaces"
      all: true
      state: Accepted
% kubectl apply -f /tmp/data-api-binding.yaml

% # create configmap with needed labels and data, check the output of the controller and see if anything wierd happens there.
% cat /tmp/test-cm.yaml 
apiVersion: v1
kind: ConfigMap
metadata:
  name: syncer-config
  labels:
    name: syncer-config
data:
  agent-target-cluster: "root:045aa3c9-017c-4824-a410-561b1e2ddef3:soumik-demo:cg-1"
  agent-target-name: "downstream"
  agent-target-namespace: "default"
  upstream-context: base
  secretData: syncer-config

% kubectl apply -f /tmp/test-cm.yaml

error seen

2024-01-29T23:23:54+05:30	INFO	Update: updated configMap	{"controller": "configmap", "controllerGroup": "", "controllerKind": "ConfigMap", "ConfigMap": {"name":"syncer-config","namespace":"default"}, "namespace": "default", "name": "syncer-config", "reconcileID": "38ae15cb-9d1c-433d-a3d6-645970009745", "cluster": "29qso51vovx9kz1x"}
2024-01-29T23:23:54+05:30	INFO	Get: retrieved configMap	{"controller": "configmap", "controllerGroup": "", "controllerKind": "ConfigMap", "ConfigMap": {"name":"syncer-config","namespace":"default"}, "namespace": "default", "name": "syncer-config", "reconcileID": "2b6493be-d719-4b86-9870-2ee1aa72fcea", "cluster": "29qso51vovx9kz1x"}
2024-01-29T23:23:54+05:30	ERROR	unable to list configmaps	{"controller": "configmap", "controllerGroup": "", "controllerKind": "ConfigMap", "ConfigMap": {"name":"syncer-config","namespace":"default"}, "namespace": "default", "name": "syncer-config", "reconcileID": "2b6493be-d719-4b86-9870-2ee1aa72fcea", "cluster": "29qso51vovx9kz1x", "error": "Index with name cluster does not exist"}
github.com/kcp-dev/controller-runtime-example/controllers.(*ConfigMapReconciler).Reconcile
	/Users/bpremkumar/code/github.com/prembhaskal/kcp-dev-controller-runtime-example/controllers/configmap_controller.go:87
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
	/Users/bpremkumar/go/pkg/mod/github.com/kcp-dev/[email protected]/pkg/internal/controller/controller.go:119
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
	/Users/bpremkumar/go/pkg/mod/github.com/kcp-dev/[email protected]/pkg/internal/controller/controller.go:316
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
	/Users/bpremkumar/go/pkg/mod/github.com/kcp-dev/[email protected]/pkg/internal/controller/controller.go:266
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
	/Users/bpremkumar/go/pkg/mod/github.com/kcp-dev/[email protected]/pkg/internal/controller/controller.go:227

Server rejected event errors when leader election is enabled

When the leader election is enabled:

restConfig := ctrl.GetConfigOrDie()
...
mgr, err := kcp.NewClusterAwareManager(cfg, ctrl.Options{
	Scheme:                 scheme,
	MetricsBindAddress:     metricsAddr,
	Port:                   9443,
	HealthProbeBindAddress: probeAddr,
	LeaderElection:         true,
	LeaderElectionConfig:   restConfig,
	LeaderElectionID:       "68a0532d.my.domain",
})

(NOTE: I specify the original rest config so it doesn't use the VirtualWorkspace URL, otherwise, it would fail because the client used in leader election doesn't use the right round tripper.)

then the leader election code tries to create an event in the VirtualWorkspace URL, but it fails because it's using a wrong path (does not contain the expected /clusters/* section)

I0707 11:39:21.287347       1 round_trippers.go:553] POST https://192.168.1.133:6443/services/apiexport/root:plane:myapp/controller-runtime-example-data.my.domain/api/v1/namespaces/controller-runtime-example-system/events 403 Forbidden in 0 milliseconds
E0707 11:39:21.287479       1 event.go:267] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"68a0532d.my.domain.16ff8920aa62aeb2", GenerateName:"", Namespace:"controller-runtime-example-system", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"ConfigMap", Namespace:"controller-runtime-example-system", Name:"68a0532d.my.domain", UID:"bc0f0a11-21c8-4a6a-b632-6d5abe3a7532", APIVersion:"v1", ResourceVersion:"27429", FieldPath:""}, Reason:"LeaderElection", Message:"controller-runtime-example-controller-manager-65cd9fdd6c-5b58t_61618914-f476-4a88-86ce-4cfcbdf88ea4 became leader", Source:v1.EventSource{Component:"controller-runtime-example-controller-manager-65cd9fdd6c-5b58t_61618914-f476-4a88-86ce-4cfcbdf88ea4", Host:""}, FirstTimestamp:time.Date(2022, time.July, 7, 11, 39, 21, 286184626, time.Local), LastTimestamp:time.Date(2022, time.July, 7, 11, 39, 21, 286184626, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'forbidden: User "system:serviceaccount:controller-runtime-example-system:controller-runtime-example-controller-manager" cannot post path "/services/apiexport/root:plane:myapp/controller-runtime-example-data.my.domain/api/v1/namespaces/controller-runtime-example-system/events": Path not resolved to a valid virtual workspace' (will not retry!)
I0707 11:39:21.289195       1 round_trippers.go:553] POST https://192.168.1.133:6443/services/apiexport/root:plane:myapp/controller-runtime-example-data.my.domain/api/v1/namespaces/controller-runtime-example-system/events 403 Forbidden in 1 milliseconds
E0707 11:39:21.289449       1 event.go:267] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"68a0532d.my.domain.16ff8920aa62d112", GenerateName:"", Namespace:"controller-runtime-example-system", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), DeletionTimestamp:<nil>, DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"Lease", Namespace:"controller-runtime-example-system", Name:"68a0532d.my.domain", UID:"b5d779ea-914b-41e5-aa3a-71ff06ee5349", APIVersion:"coordination.k8s.io/v1", ResourceVersion:"27430", FieldPath:""}, Reason:"LeaderElection", Message:"controller-runtime-example-controller-manager-65cd9fdd6c-5b58t_61618914-f476-4a88-86ce-4cfcbdf88ea4 became leader", Source:v1.EventSource{Component:"controller-runtime-example-controller-manager-65cd9fdd6c-5b58t_61618914-f476-4a88-86ce-4cfcbdf88ea4", Host:""}, FirstTimestamp:time.Date(2022, time.July, 7, 11, 39, 21, 286193426, time.Local), LastTimestamp:time.Date(2022, time.July, 7, 11, 39, 21, 286193426, time.Local), Count:1, Type:"Normal", EventTime:time.Date(1, time.January, 1, 0, 0, 0, 0, time.UTC), Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'forbidden: User "system:serviceaccount:controller-runtime-example-system:controller-runtime-example-controller-manager" cannot post path "/services/apiexport/root:plane:myapp/controller-runtime-example-data.my.domain/api/v1/namespaces/controller-runtime-example-system/events": Path not resolved to a valid virtual workspace' (will not retry!)

Instead of using the VirtualWorkspace URL, it should use the URL of the actual workspace that was provided as part of LeaderElectionConfig

Update README to note that both REGISTRY and IMG should be provided

The README indicates:

make docker-build docker-push IMG=<some-registry>/controller-runtime-example:tag

but it should be

make docker-build docker-push REGISTRY=<some-registry> IMG=controller-runtime-example:tag

because the makefile is using a default REGISTRY ?= localhost and is using ${REGISTRY}/${IMG}

Same for other commands using IMG

Raise exception: failed to wait for widget caches to sync: timed out waiting for cache to be synced

I deployed the controller to my virtual workspace: root:gold:dogs , it successed to sync to a KinD physics cluster.
but the pod keep CrashLoopBackOff, there are error raised in the logs:

290 1.6632054553562376e+09  ERROR   Could not wait for Cache to sync        {"controller": "configmap", "controllerGroup": "", "controllerKind": "ConfigMap", "error": "failed to wait for configmap caches to sync:     timed out waiting for cache to be synced"}

300 1.6632054553565369e+09  ERROR   Could not wait for Cache to sync        {"controller": "widget", "controllerGroup": "data.my.domain", "controllerKind": "Widget", "error": "failed to wait for widget caches to     sync: timed out waiting for cache to be synced"}

Normally, it's caused by RBAC issue, I checked the clsuterrole, clusterrolebinding something, I think these are all as expectd, may I missing something?

The controller cannot work

Deployd controller follow the guide, seems it start up and start the watching:

I0926 08:31:12.492757       1 round_trippers.go:553] GET https://172.16.251.127:6443/services/apiexport/root:gold:dogs/controller-runtime-example-data.my.domain/clusters/%2A/api/v1/secrets?allowWatchBookmarks=true&resourceVersion=105000&timeoutSeconds=460&watch=true 200 OK in 3 milliseconds
I0926 08:31:12.492971       1 round_trippers.go:553] GET https://172.16.251.127:6443/services/apiexport/root:gold:dogs/controller-runtime-example-data.my.domain/clusters/%2A/apis/data.my.domain/v1alpha1/widgets?allowWatchBookmarks=true&resourceVersion=104997&timeoutSeconds=495&watch=true 200 OK in 3 milliseconds
I0926 08:31:12.494016       1 round_trippers.go:553] GET https://172.16.251.127:6443/services/apiexport/root:gold:dogs/controller-runtime-example-data.my.domain/clusters/%2A/api/v1/configmaps?allowWatchBookmarks=true&resourceVersion=104996&timeoutSeconds=381&watch=true 200 OK in 4 milliseconds

But the the reconcile do nothing event if I create cm or widget:

[root@experimental experimental]# kubectl get widget -A
NAMESPACE   NAME            AGE
default     widget-sample   11m
test        widget-sample   22m

Even more I query by the curl, still get nothing:

[root@experimental controller-runtime-example]# kubectl get --raw https://172.16.251.127:6443/services/apiexport/root:gold:dogs/controller-runtime-example-data.my.domain/clusters/\*/apis/data.my.domain/v1alpha1/namespaces/default/widgets
{"apiVersion":"data.my.domain/v1alpha1","items":[],"kind":"WidgetList","metadata":{"continue":"","resourceVersion":"105245"}}

am I missing something?

Controller not finding the widget resource when no apibinding created

It is a know issue and not directly related to the example that at least one APIBinding has first to be created for a controller to be able to watch a resource type through a virtual workspace.

As I have seen a couple of people loosing time in the troubleshooting of it in the context of simply running the controller runtime example I am proposing:

  • to have a word about it in the readme
  • to have an APIBinding manifest in the config/kcp directory that gets automatically created when the example is deployed (there is an open PR for that)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.