What problem are you facing?

I want to enable users to provide their own ProviderConfig using their namespaced ServiceAccount.
Currently, the only way of doing that is by providing a kubeconfig inside a secret. Then use that secret in the ProviderConfig.
This works but would be amazing to have another simpler mechanism to provide inner kubernetes credentials per team.

How could Crossplane help solve your problem?

Adding a new CredentialsSource option: ServiceAccount to the ProviderConfig

What problem are you facing?

since latest 0.1.0 release we have a few more comments - how about timeline to cut new release 0.2.0 ?

How could Crossplane help solve your problem?

What problem are you facing?

We are provisioning Azure AKS Kubernetes Clusters using the Azure Provider by Upbound. After provisioning, we install software inside of the clusters using this Kubernetes Provider as well as the related Helm Provider. When using Local Accounts for authentication with the clusters API servers, this works out of the box. However, we would like to switch to authentication based on Azure AD Service Principals and Managed Identities, as this is the best practice and recommended by Azure. The Kubernetes Go Client which is used by this provider does not support this authentication method by default, so currently we can't use this provider with AKS clusters configured with Azure AD Authentication. As the same problem arises with standard kubectl usage and Terraform providers as well (see e.g. Azure/kubelogin#114), Azure provides the kubelogin client-go credential plugin which can be used to retrieve the user credentials and pass it to the Kubernetes Go Client.

How could Crossplane help solve your problem?

I propose to integrate the official kubelogin Go package into this provider (as well as the Helm Provider), so that it's possible to use it with AKS clusters configured with Azure AD Authentication. I'm open to provide an implementation for this and open a PR. Also, I would be happy to hear the maintainer's thoughts about this. Are you open to such contribution? @turkenh @morningspace

What problem are you facing?

I wanted to make a Crossplane Kubernetes Object like this:

apiVersion: kubernetes.crossplane.io/v1alpha1
kind: Object
metadata:
  name: federate-list-of-stuff
spec:
  deletionPolicy: Delete
  forProvider:
    manifest:
      apiVersion: v1
      kind: List
      metadata:
        name: list-of-stuff
      items:
        - apiVersion: v1
          kind: ConfigMap
          metadata:
            name: feds-one
            namespace: default
          data:
            foo: bar
        - apiVersion: v1
          kind: ConfigMap
          metadata:
            name: feds-two
            namespace: default
          data:
            peanutbutter: and jelly

How could Crossplane help solve your problem?

Send all items in the list to specified cluster. Another way to encapsulate with yaml in yaml out.
In the status would be the ever familiar references array you would see on an XRD resource.
Like so:

resourceRefs:
    - apiVersion: v1
      kind: ConfigMap
      name: feds-one
      namespace: default
    - apiVersion: v1
      kind: ConfigMap
      name: feds-two
      namespace: default

What problem are you facing?

I install kubernetes-provider by kubectl crossplane install xxx. Apply providerConfig and try to manage another bussiness k8s cluster.

First, kubectl get providerconfig / provider doesn't show anything. So I want to delete this provider, but it doesn't support something like uninstall cli ??? besides. I read the guide, it shows how to create/manage k8s resources for in-cluster or another cluster. But, for another cluster case, it only support kind ??? can you support an example for ordinary cluster ? like kubectl config view|sed -e ‘xxxxx’

What problem are you facing?

This is a follow-up issue based on review comments from @turkenh in PR #21, where we introduced a way to add/remove finalizer to referred resource in Object, so that can control the create/delete order when Object has dependent resource which needs to be created before Object gets created and deleted after Object gets deleted.

One problem with adding finalizers to referred resources is how to handle changes there. For example, an Object depends on secret-a, but then changed to secret-b with a kubectl apply, in that case, we would need to remove finalizer from secret-a and add secret-b.

How could Crossplane help solve your problem?

This could be handled by checking lastApplied in Observe(), and make corresponding switch when needed.

I have run grype on this repo and get the following output:

NAME                                 INSTALLED                           FIXED-IN                           TYPE       VULNERABILITY        SEVERITY   
github.com/emicklei/go-restful       v2.15.0+incompatible                2.16.0                             go-module  GHSA-r48q-9g5r-8q2h  Critical    
github.com/prometheus/client_golang  v1.11.0                                                                go-module  CVE-2022-21698       High        
github.com/prometheus/client_golang  v1.11.0                             1.11.1                             go-module  GHSA-cg3q-j54f-5p7p  High        
golang.org/x/net                     v0.0.0-20220516155154-20f960328961  0.0.0-20220906165146-f3363e06e74c  go-module  GHSA-69cg-p879-7622  High        
golang.org/x/net                     v0.0.0-20220516155154-20f960328961  0.7.0                              go-module  GHSA-vvpx-j8f3-3w6h  Low         

It would be great to bump the dependencies here and get rid of some of these. Even if they aren't exploitable, it is a bad look in regulated environments. Something similar to crossplane/crossplane#3562 in core Crossplane.

For the default crossplane providers you can see the provider configs via

kubectl get provider

The kubernetes provider doesn't show up in that list as of know.

That's because in the CRD the categories section of the providerconfigs.kubernetes.crossplane.io crd is missing:

  names:
    categories:
    - crossplane
    - provider

This probably has to be generated through some kubebuilder annotations.

I created a composition as attached that is just supposed to create a namespace when a project.example.com resource is created.

The resulting namespace is created as expected, however it doesn't have an owner reference set. That means when I delete my project.example.com resource, the provisioned namespace remains on the cluster.

It should have the owner reference set so that it gets cleaned up by the kubernetes garbage collector once the composite objects.kubernetes.crossplane.io resource is gone.

apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: compositeprojects.example.com
  labels:
    provider: me
spec:
  compositeTypeRef:
    apiVersion: example.com/v1alpha1
    kind: CompositeProject
  patchSets:
  - name: metadata
    patches:
    - fromFieldPath: metadata.labels
  resources:
    - name: namespace
      base:
        apiVersion: kubernetes.crossplane.io/v1alpha1
        kind: Object
        spec:
          forProvider:
            manifest:
              apiVersion: v1
              kind: Namespace
            metadata:
              name: ""
          providerConfigRef:
            name: kubernetes-provider
      patches:
        - type: PatchSet
          patchSetName: metadata
        - fromFieldPath: "spec.claimRef.name"
          toFieldPath: "spec.forProvider.manifest.metadata.annotations[paas.getcruise.com/repos]"
        - fromFieldPath: "spec.claimRef.name"
          toFieldPath: "spec.forProvider.manifest.metadata.name"

What problem are you facing?

From https://crossplane.slack.com/archives/CEG3T90A1/p1647335790153809?thread_ts=1647335364.461669&cid=CEG3T90A1:

I'm observing that a single XRC with a composition provisionning 4 kubernetes provider Objects is taking ~30-60s to become ready on a fresh empty K8S cluster. When there is an set of existing 300 XRCs already reconciled, then a new XRC reconciles in 30 minutes on my cluster.
I'm running latest versions (crossplane 1.6.4 with helm provider 0.10 and kubernes provider 0.10.0) with the --max-reconcile-rate=100 arg in the helm chart args.

How could Crossplane help solve your problem?

From crossplane/crossplane#2595 (comment), the helm and kubernetes providers don't yet support the max reconcile rate option.

Support for the --max-reconcile-rate would enable concurrent parallel processing of kubernetes provider Object MRs

What problem are you facing?

I'm trying to use provider-kubernetes to create Kubernetes Jobs, however the RBAC settings for the used ServiceAccount won't allow it.
I can specify a ServiceAccount to use but then I also need to manually add the RBAC settings for my resources as well as the CRDs from this package.

How could Crossplane help solve your problem?

Ability to somehow configure RBAC settings for the provider without having to create an extra ServiceAccount.
I suppose it would also be okay if the created ServiceAccount was deterministic (for example by specifying the name).
Then we could just add extra Roles on top of the pre-existing ones.

What problem are you facing?

GKE is moving to a dedicated gke-gcloud-auth-plugin for cluster authentication. From the announcement:

This new binary, gke-gcloud-auth-plugin, uses the Kubernetes Client-go Credential Plugin mechanism to extend kubectl’s authentication to support GKE. Because plugins are already supported by kubectl, you can switch to the new mechanism now, before v1.26 becomes available.

How could Crossplane help solve your problem?

Ensure that our gke authentication flow will still be compatible with GKE for versions 1.26+

Vulnerabilities in the kubernetes provider image that possibly should be patched for the next release.
screenshot from docker-desktop below.

What problem are you facing?

We currently use GKE workload identity for creating resources with the gcp-provider. We would like to also use workload identity to authenticate to new GKE clusters that are created in order to use the kubernetes-provider (and helm-provider) with the identity GoogleApplicationCredentials type. I would assume that the identity source would be InjectedIdentity, but that seems to not be supported for the ProviderConfig identity.

How could Crossplane help solve your problem?

Allow InjectedIdentity identity source to be used with GoogleApplicationCredentials identity type. When InjectedIdentity identity source is used perform an authentication flow using the Google Application Default Credentials that is provided by the service account's workload identity. This would also be required in the helm-provider as well.

If there are any other alternatives to using workload identity from the crossplane cluster to authenticate to a new GKE cluster, please let me know. We are not allowed to use service account keys in our GCP org and need to use workload identity for service account authentication.

What happened?

I try to create an ArgoCD application using Kubernetes crossplane provider (through a composition) but the application is never created. If I take the manifest and apply it directly it works so it doesn't seems related to a missconfiguration on ArgoCD side.

Nothing is logged neither on the kubernetes events nor on the provider container

How can we reproduce it?

• create an argocd application from a kubernetes Object resource

apiVersion: kubernetes.crossplane.io/v1alpha1
kind: Object
spec:
  forProvider:
    manifest:
      apiVersion: argoproj.io/v1alpha1
      kind: Application
      metadata:
        labels:
          instance: my-application
        name: my-application
        namespace: argocd
      spec:
        destination:
          name: cluster-name
          namespace: my-application
        project: project-name
        source:
          path: my-application
          plugin:
            env:
              [...]
          repoURL: https://[....].git
          targetRevision: revision-name
        syncPolicy:
          automated:
            prune: true
            selfHeal: true
          retry:
            backoff:
              duration: 30s
              factor: 2
              maxDuration: 2m
            limit: 5
  managementPolicy: Observe
  providerConfigRef:
    name: provider-kubernetes

• application never got created. the object sync seems to be stuck:

  Normal  CreatedExternalResource  66s (x9 over 10m)   managed/object.kubernetes.crossplane.io  Successfully requested creation of external resource
  Normal  PendingExternalResource  65s (x17 over 10m)  managed/object.kubernetes.crossplane.io  Waiting for external resource existence to be confirmed

What environment did it happen in?

Crossplane version: v1.10.2
Kubernetes provider version: v0.6.0
Cloud provider: AWS
Kubernetes version: v1.23-9
Kubernetes distribution: EKS
OS: Amazon Linux"
Kernel: Linux 5.4.209-116.367.amzn2.x86_64 #1 SMP Wed Aug 31 00:09:52 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

When using k8s provider (pointing to main currently)

apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
  name: provider-kubernetes
spec:
  package: "crossplane/provider-kubernetes:main"

and attempting to create a k8s object from another object, The toFieldPath is not respected and the object gets created using the value of field path without the rewrite.

  example manifest 
  
  apiVersion: kubernetes.crossplane.io/v1alpha1
kind: Object
metadata:
  name: somename
  namespace: somenamespace
spec:
  references:
  - patchesFrom:
      apiVersion: v1
      kind: Secret
      name: "cloudsql-instance-credentials"
      namespace: "value"
      toFieldPath: spec.forProvider.manifest.data.CLOUDSQL_INSTANCE
      fieldPath: data.cloudsql_instance_connection
  forProvider:
    manifest:
      apiVersion: v1
      kind: ConfigMap
      data: 
        CLOUDSQL_INSTANCE: ""
      metadata:
        name: cloudsql
        namespace: {{ .Release.Namespace }}
  providerConfigRef:
    name: provider-kubernetes

example config map created : 

apiVersion: v1
data:
  CLOUDSQL_INSTANCE: ""
  cloudsql_instance_connection:BASE64ENCODEDVALUE==
kind: ConfigMap
metadata:
  name: cloudsql
  namespace: infra

when I attempt to add the toFieldPath directly to the object endpoint under Kubernetes.crossplane.io as it appears to be redacted I get

 objects.kubernetes.crossplane.io "nameofmyobject" was not valid:
# * <nil>: Invalid value: "The edited file failed validation": ValidationError(Object.spec.references[0].patchesFrom): unknown field "toFieldPath" in io.crossplane.kubernetes.v1alpha1.Object.spec.references.patchesFrom

For our case, it's critical to be able to perform health assessment of resources created from Objects. Specifically, for custom resources compatible with kstatus, which essentially means "wait till the resource's status has Ready = True condition".

I would assume the correct implementation means the following:

• there's some way to enable health check for arbitrary Object
• if enabled, Object doesn't become ready until underlying resource reports Ready status
• Objects with dependencies (ones that have references[].dependsOn) can honor status of their dependency, regardless of whether it's another Object or an arbitrary k8s resource

I apologize if it's already possible, I didn't realize how to do it.

What happened?

KUBERNETES provider controller container creation fails with error
Error: failed to start container "provider-kubernetes": Error response from daemon: OCI runtime create failed: container_linux.go:367: starting container process caused: chdir to cwd ("/home/nonroot") set in config.json failed: permission denied: unknown

Similar issue for provider-aws and for provider-helm

The last working release is v0.4.0.

How can we reproduce it?

1. create Minikube cluster
2. install Crossplane
3. add Kubernetes provider

What environment did it happen in?

Crossplane version: 1.9.0, installed by Helm chart
k8s: Minikube with k8s v1.20.2

What problem are you facing?

Provider kubernetes is designed to provision and manage Kubernetes objects on (remote) Kubernetes clusters. For a normal Kubernetes object such as a ConfigMap or whatever else, it does not have to define a connection secret, since it does not make sense to use a connection string in order to connect to these Kubernetes objects. For the kubeconfig used to connect to the cluster that these Kubernetes objects reside in, it is handled separately in ProviderConfig using spec. credentials.

However, if the Kubernetes object managed by the provider is something driven by some Kubernetes controller at backend which has more complicated scenario, such as to drive a provisioning of a Kafka instance on the target cluster, then to connect to what is actually provisioned by the Kubernetes object may need some connection string, e.g. the connection string to access the Kafka instance.

Right now, the provider does not write any connection secret by itself. So, if I specify spec.writeConnectionSecretToRef, a connection secret with empty data will be created, which is not expected.

How could Crossplane help solve your problem?

To allow provider kubernetes to respect the connection settings by creating the corresponding secret would help in this case. Not like other providers, the provider kubernetes itself may not know how to build such connection string, but in case the whatever backend that drives the actual workload provisioning knows that and create the secret, then the provider may have a way that can allows user to specify where to retrieve this string, e.g.: via an optional field called spec.readConnectionSecretToRef, then expose it to Object resource using spec.writeConnectionSecretToRef.

Scenario:
I have EKS cluster and I need to update aws-node daemonset environment variable.
I try to do it with following Object definition, this is part of composition:

  - name: aws-node-daemonset
    base:
      apiVersion: kubernetes.crossplane.io/v1alpha1
      kind: Object
      spec:
        managementPolicy: ObserveCreateUpdate
        providerConfigRef:
          name: kubernetes-provider
        forProvider:
          manifest:
            apiVersion: apps/v1
            kind: DaemonSet
            metadata:
              name: aws-node
              namespace: kube-system
            spec:
              template:
                spec:
                  containers:
                  - env:
                    - name: ENABLE_PREFIX_DELEGATION
                      value: "true"
                    name: aws-node

Result is that I get error update failed: cannot apply object: cannot patch object: DaemonSet.apps "aws-node" is invalid: spec.template.spec.containers[0].image: Required value
So it looks like I have to add complete daemonset definition to make it work but that's not optimal, I just want to modify existing a little bit.
Is there anyway to do some kind of "merge patch" to existing kubernetes object? I think that way this could work

What happened?

i tried to create a namespace on a k8s cluster and got following message

observe failed: cannot get object: no matches for kind "Namespace" in version "v1"

How can we reproduce it?

mu current steps are

1. spin a new gke cluster
2. install crossplane and this providers
3. use crossplane to create a second gke cluster
4. use this provider to try to create a namespace on the second cluster

What environment did it happen in?

Crossplane version: 1.6.x

• Cloud provider or hardware configuration: gcloud dke

• Kubernetes version (use kubectl version)
  Client Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.1", GitCommit:"86ec240af8cbd1b60bcc4c03c20da9b98005b92e", GitTreeState:"clean", BuildDate:"2021-12-17T02:23:27Z", GoVersion:"go1.17.5", Compiler:"gc", Platform:"linux/amd64"}
  Server Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.3-gke.700", GitCommit:"e21de5d6cf1b351bc99285c3c762098b8617b773", GitTreeState:"clean", BuildDate:"2021-11-05T09:25:32Z", GoVersion:"go1.16.7b7", Compiler:"gc", Platform:"linux/amd64"}

• Kubernetes distribution GKE

What problem are you facing?

One of the actual limitations of Crossplane is that existing Kubernetes operator (i.e. check operatorhub.io) cannot be integrated within a Composition because the provider-kubernetes - in particular, the Kind Object - is not aware of the Readiness of the applied manifest. The actual implementation of Kind Object has status Ready when the Kubernetes manifest is applied.

How could Crossplane help solve your problem?

The Ready status of the Kind Object should be aligned with the status of the applied Manifest. This would expand the compatibility of the Composition with operators that are not implement with the Crossplane runtime.

Motivation

I want to give a simple API for a complicated kubernetes resource manifest.
For better demonstration, the "complicated" k8s resource will be a Secret.

What happened?

I created the following XRD and Composition

---
apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
  name: compositeexternalsecrets.my-domain.io
spec:
  group: my-domain.io
  names:
    kind: CompositeSecret
    plural: compositesecrets
  claimNames:
    kind: MySecret
    plural: mysecrets
  versions:
    - name: v1alpha1
      served: true
      referenceable: true
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              properties:
                dataInject:
                  type: string
---
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: mysecret
spec:
  compositeTypeRef:
    apiVersion: my-domain.io/v1alpha1
    kind: CompositeSecret
  resources:
    - base:
        apiVersion: kubernetes.crossplane.io/v1alpha1
        kind: Object
        spec:
          providerConfigRef:
            name: crossplane-provider-kubernetes-config
          forProvider:
            manifest:
              apiVersion: v1
              kind: Secret
              type: Opaque
              data:
                default: default
      patches:
        - fromFieldPath: "metadata.namespace"
          toFieldPath: "spec.forProvider.manifest.metadata.namespace"
        - fromFieldPath: "spec.dataInject"
          toFieldPath: "spec.forProvider.manifest.data[injected]"

And claim it with

---
apiVersion: my-domain.io/v1alpha1
kind: MySecret
metadata:
  name: secret
  namespace: some-namespace
spec:
  dataInject: injected

The idea is to created a k8s resource of kind MySecret with as little data as possible and by that create a more complex and importantly a different k8s resource.
In this example I would like to see a Secret to be created that looks like this:

apiVersion: v1
kind: Secret
type: Opaque
metadata:
  name: secret-hashed-12379 # generated
  namespace: some-namespace # identical to MySecret's namespace
data:
  default: default
  injected: injected

What happens instead is that the Secret does not get created. Instead I see the following error
an empty namespace may not be set when a resource name is provided.

A kubectl describe object secret-f7jbk-n52l5 returns:

Name:         secret-f7jbk-n52l5
Namespace:
Labels:       crossplane.io/claim-name=secret
              crossplane.io/claim-namespace=sms-infra
              crossplane.io/composite=secret-f7jbk
Annotations:  crossplane.io/external-name: secret-f7jbk-n52l5
API Version:  kubernetes.crossplane.io/v1alpha1
Kind:         Object
Metadata:
  ...
  Owner References:
    API Version:     my-domain.io/v1alpha1
    Controller:      true
    Kind:            CompositeSecret
    Name:            secret-f7jbk
    UID:             c48be853-eb31-4f60-9f78-13ade20dc7c8
  Resource Version:  1267707672
  UID:               c7b56c56-8578-4095-a2d1-6d6b165ff750
Spec:
  For Provider:
    Manifest:
      API Version:  v1
      Data:
        Default:      default
        Injected:     injected
      Kind:           Secret
      Type:           Opaque
  Management Policy:  Default
  Provider Config Ref:
    Name:  crossplane-provider-kubernetes-config
Status:
  At Provider:
  Conditions:
    Last Transition Time:  2022-04-19T13:51:05Z
    Message:               observe failed: cannot get object: an empty namespace may not be set when a resource name is provided
    Reason:                ReconcileError
    Status:                False
    Type:                  Synced
Events:
  Type     Reason                         Age                  From                                     Message
  ----     ------                         ----                 ----                                     -------
  Warning  CannotObserveExternalResource  92s (x11 over 7m7s)  managed/object.kubernetes.crossplane.io  cannot get object: an empty namespace may not be set when a resource name is provided

I also tried other resources instead of Secrets. Patching the namespace does not work. I am using provider-kubernetes for this because of Crossplane's Composite Resource Limitation for cluster scoped resources crossplane/crossplane#1730

What environment did it happen in?

Crossplane version: v1.6.0
provider-kubernetes version: v0.3.0

kubectl version

Client Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.5", GitCommit:"c285e781331a3785a7f436042c65c5641ce8a9e9", GitTreeState:"clean", BuildDate:"2022-03-16T15:51:05Z", GoVersion:"go1.17.8", Compiler:"gc", Platform:"darwin/arm64"}
Server Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.5", GitCommit:"aea7bbadd2fc0cd689de94a54e5b7b758869d691", GitTreeState:"clean", BuildDate:"2021-09-15T21:04:16Z", GoVersion:"go1.16.8", Compiler:"gc", Platform:"linux/amd64"}
```

What problem are you facing?

I would like to install an operator with provider-kubernetes but the yaml is too long and complex to copy+paste it. This raw yaml file is available online on GitHub.

How could Crossplane help solve your problem?

It would be great if provider-kubernetes could allow patching from online yaml files.

I'm trying to create a k8s secret patching from another secret.

I'm trying to achieve this by applying the following object inside of a composition:

- name: argocd-creds-secret
      base:
        apiVersion: kubernetes.crossplane.io/v1alpha1
        kind: Object
        metadata:
          name: cluster-bearer-token
        spec:
          references:
          - patchesFrom:
              apiVersion: v1
              kind: Secret
              name: connection-string
              namespace: default
              fieldPath: data.kubeconfig
            toFieldPath: data.bearer-token
            transforms:
              - type: string
                string:
                  type: Regexp
                  regexp:
                    match: '(?<=token:).*$'
          forProvider:
            manifest:
              apiVersion: v1
              kind: Secret
              metadata:
                namespace: default
              data:
                sample-key: #to be patched
          providerConfigRef:
            name: local-kubernetes-provider

It looks like it completely ignores the transformation.
Is the transformation taken into account at the current implementation=

What problem are you facing?

We are currently using crossplane as our control plane deployed in a control k8s cluster. we also deployed aws-provider, provider-kubernetes. we intend to deploy application configuration composed cloud resources created by aws-provider in bussiness k8s cluster which is piloted by provider-kubernetes.

we hope to compose our application k8s resources and aws resources. Say a redis created by aws-provider from control k8s cluster,
we hope to use it directly in our "object" or "composition" resource for provider-kuberntes.

How could Crossplane help solve your problem?

Does provider-kubernetes currently support such feature ? I think it would be cool if we could manage business k8s cluster and cloud resources from the same control plane, and compose those cloud resources like mysql redis to applicatioin in business k8s cluster .

What happened?

root@gyliu-dev21:~/go/src/github.com/crossplane/provider-kubernetes# make local-dev
make: *** No rule to make target 'local.up', needed by 'local-dev'.  Stop.

How can we reproduce it?

What environment did it happen in?

Crossplane version:

Hi all,
I don't know what i miss but the provider-kubernetes pod doesn't start. The event is that no command speicified.

Error: Error response from daemon: No command specified                                                                
Pod sandbox changed, it will be killed and re-created.                                                                 
Container image "PRIVATE-REPO/crossplane/provider-kubernetes:main" already present on machine
                                                                                                                       

And indeed in description of the pod i don't see anything.
I see that in the ControllerConfig we can give the argument args to give the entrypoint but i don't know what to use.

Containers:                                                                                    
  provider-kubernetes:                                                                         
    Container ID:                                                                              
    Image:          PRIVATE-REPO/crossplane/provider-kubernetes:main 
    Image ID:                                                                                  
    Port:           8080/TCP                                                                   
    Host Port:      0/TCP                                                                      
    State:          Waiting                                                                    
      Reason:       CreateContainerError                                                       
    Ready:          False                                                                      
    Restart Count:  0                                                                          
    Environment:                                                                               
      POD_NAMESPACE:  crossplane-system (v1:metadata.namespace)                                
    Mounts:                                                                                    
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-b7jq2 (ro)            
Conditions:                                                                                    

What did i miss and where?
Thank you

What problem are you facing?

In our contest we have to use a private docker registry.

How could Crossplane help solve your problem?

We deployed the provider as such but the provider-kubernetes pod was still trying to reach the docker hub repo.

apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
  name: provider-kubernetes
spec:
  package: "private_repo/crossplane/provider-kubernetes:main"

Would it be possible to have a "dynamic" repo?

What happened?

Kubernetes Deployment created from provider-kubernetes object is not monitored for drift.

How can we reproduce it?

Create the following object:

apiVersion: kubernetes.crossplane.io/v1alpha1
kind: Object
metadata:
  name: whoami
  namespace: default
spec:
  forProvider:
    manifest:
      apiVersion: apps/v1
      kind: Deployment
      metadata:
        namespace: default
      spec:
        metadata:
          name: whoami
          namespace: default
        replicas: 2
        selector:
          matchLabels:
            app: whoami
        template:
          metadata:
            labels:
              app: whoami
          spec:
            containers:
            - name: whoami
              image: docker.io/containous/whoami:v1.5.0
  providerConfigRef:
    name: kubernetes-provider

Edit the created deployment with kubectl edit deployments.apps whoami updating spec.replicas: 3

Expected: deployment drift detected and updated back to spec.replicas: 2.
Actual: Deployment continues to have 3 pods.

Editing the object forProvider.manifest.spec.replicas: 4:
Expected: object drift detected and updated back to forProvider.manifest.spec.replicas: 2 and deployment and pods updated.
Actual: object replicas, deployment replicas and pod count remains at 4

Note that if the object is created using composition, it behaves as above on deployment edit, BUT then if the object is updated forProvider.manifest.spec.replicas: 4 then the drift of object is somehow detected, the replicas reset to 2 and the deployment and pod count similarly fixed up back to 2.

apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
  name: xminapps.example.com
spec:
  group: example.com
  names:
    kind: XMinApp
    plural: xminapps
  claimNames:
    kind: MinAppClaim
    plural: minappclaims
  versions:
  - name: v1alpha1
    served: true
    referenceable: true
    schema:
      openAPIV3Schema:
        type: object
        properties:
          spec:
            type: object
            properties:
              id:
                type: string
              parameters:
                type: object
                properties:
                  imageName:
                    type: string
                  replicas:
                    type: integer
                    default: 1
                required:
                - imageName
            required:
            - id
            - parameters
---
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: minapp
  labels:
    crossplane.io/xrd: xminapp.example.com
    provider: kubernetes
spec:
  compositeTypeRef:
    apiVersion: example.com/v1alpha1
    kind: XMinApp
  resources:
  - name: deployment
    base:
      apiVersion: kubernetes.crossplane.io/v1alpha1
      kind: Object
      spec:
        forProvider:
          manifest:
            apiVersion: apps/v1
            kind: Deployment
            metadata:
              namespace: default
            spec:
              replicas: 1
              template:
                spec:
                  containers:
                  - name: frontend
        providerConfigRef:
          name: kubernetes-provider
    patches:
    - fromFieldPath: spec.id
      toFieldPath: metadata.name
    - fromFieldPath: spec.id
      toFieldPath: spec.forProvider.manifest.metadata.name
    - fromFieldPath: spec.id
      toFieldPath: spec.forProvider.manifest.metadata.labels.app
    - fromFieldPath: spec.id
      toFieldPath: spec.forProvider.manifest.spec.selector.matchLabels.app
    - fromFieldPath: spec.id
      toFieldPath: spec.forProvider.manifest.spec.template.metadata.labels.app
    - fromFieldPath: spec.id
      toFieldPath: spec.forProvider.manifest.spec.template.spec.containers[0].name
    - fromFieldPath: spec.parameters.imageName
      toFieldPath: spec.forProvider.manifest.spec.template.spec.containers[0].image
    - fromFieldPath: spec.parameters.replicas
      toFieldPath: spec.forProvider.manifest.spec.replicas
---
apiVersion: example.com/v1alpha1
kind: MinAppClaim
metadata:
  name: my-whoami
spec:
  id: whoami
  parameters:
    imageName: docker.io/containous/whoami:v1.5.0
    replicas: 2

What environment did it happen in?

Crossplane version: v1.7.0 (helm install)
kubernetes version v1.22.5 (windows docker desktop kubernetes environment)
Win 10 21H2 (19044.1645)
accessing through through WSL2 Ubuntu 20.04.4 LTS (Focal Fossa)
using client kubectl v1.21.5

What problem are you facing?

While building platform configurations with Crossplane, it is quite common that we need to copy some objects from the local cluster (a.k.a control plane) to remote clusters. The most common examples of this are distributing secrets like dockerhub & helm repo credentials or copying connection creds of a managed resource (e.g. db) to a remote cluster (i.e. application cluster).

This was asked a couple of times in the Crossplane slack and the best solution (workaround?) we have so far is using provider-helm's set.valueFrom .secretKeyRef: https://crossplane.slack.com/archives/CEG3T90A1/p1629975361008800?thread_ts=1629969149.007900&cid=CEG3T90A1

There is also an open issue in Crossplane which could be used as a solution here as a composition patch but may not help if a composition is not used or a whole objected needs to be copied.

How could Crossplane help solve your problem?

Currently provider Kubernetes seems to be the best fit to implement/support this in the form of a managed resource.

I came up of something like the following after a quick thought:

apiVersion: kubernetes.crossplane.io/v1alpha2
kind: Object
metadata:
  name: sample-namespace
spec:
  forProvider:
    manifest:
      apiVersion: v1
      kind: Secret
      metadata:
        name: regcred
        namespace: application
    from:
      apiVersion: v1
      kind: Secret
      name: regcred
      namespace: crossplane-system
      fieldPaths: 
        - type
        - data

We have a set of ArgoCD Application resources which are created dynamic and declaratively via the pattern (App of Apps).
This Application.yml does have a field "cluster" which defines a cluster URL.

We want this clusterURL to map directly from an existent secret using this provider.

Is it possible based on this example for provider-kubernets to create a patch from a secret into a set of different kind of objects (finding by label or other attribute) and update a certain definition?

Is there a recommended way to extend RBAC permissions for the provider to other CRDs in the local cluster?

I have some custom resources that are not accessible by the default provider service account due to RBAC, and since the default service account and cluster role are generated it's not possible to automatically provision access to those CRDs for the generated SA.

I can create a Service Account for the provider to use and give it the right permissions, but I'm hoping there's a more elegant solution.

Thanks

What happened?

cannot parse package contents: Object 'Kind' is missing in ' seen in combination with crossplane :v1.2.3 with 1.3.0 everything is working as expected - is 1.3.0 minimum required ?

➜  secret-composition kubectl get pkg
NAME                                                 INSTALLED   HEALTHY   PACKAGE                               AGE
provider.pkg.crossplane.io/provider-kubernetes       True        False     crossplane/provider-kubernetes:main   4m23s

➜  secret-composition kubectl describe providerrevision provider-kubernetes-8f6b7f732d1e
Name:         provider-kubernetes-8f6b7f732d1e
Namespace:    
Labels:       pkg.crossplane.io/package=provider-kubernetes
Annotations:  <none>
API Version:  pkg.crossplane.io/v1
Kind:         ProviderRevision
Metadata:
  Creation Timestamp:  2021-07-25T20:23:39Z
  Finalizers:
    revision.pkg.crossplane.io
  Generation:  2
  Owner References:
    API Version:           pkg.crossplane.io/v1
    Block Owner Deletion:  true
    Controller:            true
    Kind:                  Provider
    Name:                  provider-kubernetes
    UID:                   e540d8a1-dd82-47c9-a202-27f6f2b5f5f9
  Resource Version:        1754741
  Self Link:               /apis/pkg.crossplane.io/v1/providerrevisions/provider-kubernetes-8f6b7f732d1e
  UID:                     e5c8f5a1-f623-42f3-a62a-cf7bf10422c3
Spec:
  Desired State:                  Active
  Ignore Crossplane Constraints:  false
  Image:                          crossplane/provider-kubernetes:main
  Package Pull Policy:            IfNotPresent
  Revision:                       1
  Skip Dependency Resolution:     false
Status:
  Conditions:
    Last Transition Time:  2021-07-25T20:23:40Z
    Reason:                UnhealthyPackageRevision
    Status:                False
    Type:                  Healthy
  Controller Ref:
    Name:  
Events:
  Type     Reason             Age                  From                                         Message
  ----     ------             ----                 ----                                         -------
  Normal   BindClusterRole    88s (x5 over 5m)     rbac/providerrevision.pkg.crossplane.io      Bound system ClusterRole to provider ServiceAccount(s)
  Normal   ApplyClusterRoles  88s (x4 over 4m59s)  rbac/providerrevision.pkg.crossplane.io      Applied RBAC ClusterRoles
  Warning  ParsePackage       0s (x11 over 4m59s)  packages/providerrevision.pkg.crossplane.io  cannot parse package contents: Object 'Kind' is missing in '

What environment did it happen in?

Crossplane version: 1.2.3

What happened?

If a cluster is removed, object resources that were applied to that cluster tend to persist and cannot be removed.

How can we reproduce it?

1. Create a composite resource that create a k8s cluster and applies object resources from the Kubernetes provider.
2. Delete the composite resource
3. Observe that some object resources persist and cannot be deleted from the management cluster.

Here's an example of an XRD that combines AWS and Kubernetes resources.

What environment did it happen in?

Reproducible with any version.

What happened?

The changes in #47 broke the main branch because the CRDs were generated with missing apiVersion attributes

This was caused by the Makefile clean.crds target being called after the generate target because it was configured as generate.done: clean,crds

Since it works in the provider-terraform project with just generate: clean.crds, which means that the CRDs are corrupted and then overwritten, I think the clean.crds target can be removed from the Makefile since whatever it was there for doesn't seem to be a problem anymore.

How can we reproduce it?

Run make generate on the branch from #47 and observe that the generated CRDs are missing the first two lines.

When deploying a K8S cluster, ProviderConfig for the kubernetes provider, and Objects in the same Composition, the Objects can get stuck in the deletion process if the Secret that the ProviderConfig refers to gets deleted before the Objects.

Because the delete process is unsequenced, the Cluster Secret that the ProviderConfig uses can get deleted before the Objects, leading to this:

conditions:
  - lastTransitionTime: "2022-05-03T19:05:33Z"
    reason: Deleting
    status: "False"
    type: Ready
  - lastTransitionTime: "2022-05-03T19:13:46Z"
    message: 'connect failed: cannot get credentials: cannot get credentials secret:
      Secret "footest-bv9r4-4zzkr" not found'
    reason: ReconcileError
    status: "False"
    type: Synced

How can we reproduce it?

Use a single Composition to deploy a K8S Cluster, ProviderConfig for the kubernetes provider, and one or more Objects.

Observe that upon deletion the Objects can get stuck in the above state.

What environment did it happen in?

Crossplane version: 1.7

Given that the failure is in the Connect function, I'm not sure what the solution should be, as Connect() cannot return a valid Kubernetes client object or external{} interface without the credentials, and the Observe will fail anyway since the Cluster is gone.

If someone can suggest a solution I'm happy to push a PR.

I see lots of rate limited requests in the logs of this controller like the ones attached.

I know this has to do with the k8s/client-go discovery cache. I know @negz is working on this and have seen tickets in kubernetes/kubernetes and kubernetes/kubectl.

Does this mean we have to not only solve the discovery cache problem for kubectl, but also for controllers like this one?

I1115 16:57:11.131312       1 request.go:655] Throttling request took 1.040355092s, request: GET:https://<CLUSTER_IP>>:443/apis/cloud.google.com/v1?timeout=32s
I1206 21:42:01.320703       1 request.go:655] Throttling request took 1.197162562s, request: GET:https://<CLUSTER_IP>>:443/apis/microsoft.resources.azure.com/v1alpha1api20200601?timeout=32s
I1206 21:42:11.514747       1 request.go:655] Throttling request took 11.391711778s, request: GET:https://<CLUSTER_IP>>:443/apis/networking.internal.knative.dev/v1alpha1?timeout=32s
I1206 21:42:21.624922       1 request.go:655] Throttling request took 5.194396672s, request: GET:https://<CLUSTER_IP>>:443/apis/database.azure.crossplane.io/v1alpha3?timeout=32s
I1206 21:42:31.682872       1 request.go:655] Throttling request took 1.795218693s, request: GET:https://<CLUSTER_IP>>:443/apis/container.cnrm.cloud.google.com/v1beta1?timeout=32s
I1206 21:42:41.881957       1 request.go:655] Throttling request took 11.993478748s, request: GET:https://<CLUSTER_IP>>:443/apis/crd.projectcalico.org/v1?timeout=32s
I1206 21:42:51.899638       1 request.go:655] Throttling request took 8.589133901s, request: GET:https://<CLUSTER_IP>>:443/apis/security.istio.io/v1beta1?timeout=32s
I1206 21:43:44.499915       1 request.go:655] Throttling request took 1.194056856s, request: GET:https://<CLUSTER_IP>>:443/apis/dataflow.cnrm.cloud.google.com/v1beta1?timeout=32s
I1206 21:43:54.696859       1 request.go:655] Throttling request took 11.391668377s, request: GET:https://<CLUSTER_IP>>:443/apis/microsoft.storage.azure.com/v1alpha1api20210401storage?timeout=32s

What happened?

I have created a cluster in AWS using Crossplane. In the same composition, I also install FluxCD onto the cluster which creates a secret with a couple of values in there. I am interested in the public key value (identity.pub). So I am using the kubernetes provider to try and retrieve the public key from the secret, however, it cannot be found. Currently, I'm just trying to retrieve it and put the value into a config map - just to see if secret retrieval works. Once this works, I will be attempting to output the public key value to the XR using ToCompositeFieldPath so I can use it in another resource (a Gitlab resource that uses the public key to create a Deploy Key)

The following error is returned when I do a describe on the resource.

Events:
  Type     Reason                         Age                   From                                     Message
  ----     ------                         ----                  ----                                     -------
  Warning  CannotConnectToProvider        5m21s (x18 over 17m)  managed/object.kubernetes.crossplane.io  cannot get ProviderConfig: ProviderConfig.kubernetes.crossplane.io "default" not found
  Warning  CannotObserveExternalResource  21s (x7 over 5m4s)    managed/object.kubernetes.crossplane.io  cannot resolve resource references: cannot get referenced resource: secrets "test-vpc-sync" not found

How can we reproduce it?

Have an external resource that is a secret on another cluster and then try and retrieve one of it's values.

Kubernetes Composition

    - name: kubernetes
      base:
        apiVersion: kubernetes.crossplane.io/v1alpha1
        kind: ProviderConfig
        spec:
          credentials:
            source: Secret
            secretRef:
              key: kubeconfig
      patches:
      - fromFieldPath: spec.id
        toFieldPath: metadata.name
      - fromFieldPath: spec.writeConnectionSecretToRef.namespace
        toFieldPath: spec.credentials.secretRef.namespace
      - fromFieldPath: spec.id
        toFieldPath: spec.credentials.secretRef.name
        transforms:
          - type: string
            string:
              fmt: "%s-cluster"
      readinessChecks:
        - type: None
    - name: get-public-key
      base:
        apiVersion: kubernetes.crossplane.io/v1alpha1
        kind: Object
        metadata:
          name: foo
        spec:
          references:
            - patchesFrom:
                apiVersion: v1
                kind: Secret
                name: "test-vpc-sync"
                namespace: flux-system
                fieldPath: data[identity.pub]
              toFieldPath: data.publicKey
          forProvider:
            manifest:
              apiVersion: v1
              kind: ConfigMap
              metadata:
                namespace: flux-system
                name: pubsecret
              data:
                publicKey: sample-value
      patches:
      - fromFieldPath: spec.id
        toFieldPath: spec.providerConfigRef.name
      - fromFieldPath: spec.id
        toFieldPath: metadata.name
      - fromFieldPath: spec.writeConnectionSecretToRef.namespace
        toFieldPath: spec.credentials.secretRef.namespace
      - fromFieldPath: spec.id
        toFieldPath: spec.credentials.secretRef.name
        transforms:
          - type: string
            string:
              fmt: "%s-cluster"
      readinessChecks:
        - type: None

What environment did it happen in?

Crossplane version: Chart version: 1.7.1

• AWS EKS
• K8s: 1.22

What happened?

apiVersion: kubernetes.crossplane.io/v1alpha1
kind: Object
metadata:
  annotations:
    crossplane.io/composition-resource-name: ref-remote-redhat-patch-op-svc
    crossplane.io/external-create-pending: "2022-08-19T19:33:48Z"
    crossplane.io/external-create-succeeded: "2022-08-19T19:33:48Z"
    crossplane.io/external-name: c4-ref-remote-redhat-patch-op-svc
  creationTimestamp: "2022-08-19T18:10:26Z"
  deletionGracePeriodSeconds: 0
  deletionTimestamp: "2022-08-19T19:34:02Z"
  finalizers:
  - finalizer.managedresource.crossplane.io
  generateName: c4-state-init-
  generation: 2
  labels:
    crossplane.io/claim-name: ""
    crossplane.io/claim-namespace: ""
    crossplane.io/composite: c4-state-init
  name: c4-ref-remote-redhat-patch-op-svc
  ownerReferences:
  - apiVersion: hello.com/v1
    controller: true
    kind: ClusterInitializing
    name: c4-state-init
    uid: 91736d6a-1a54-44ed-a082-2532927f3129
  resourceVersion: "1219713"
  uid: 2ee75b76-a47e-43d4-9d5c-37441f84ad84
spec:
  deletionPolicy: Delete
  forProvider:
    manifest:
      apiVersion: v1
      kind: Endpoints
      metadata:
        name: patch-operator-webhook-service
        namespace: argocd
  managementPolicy: Observe
  providerConfigRef:
    name: c4-k8sobjects
status:
  atProvider: {}
  conditions:
  - lastTransitionTime: "2022-08-19T19:34:48Z"
    message: 'connect failed: cannot get credentials: cannot get credentials secret:
      Secret "c4-secret" not found'
    reason: ReconcileError
    status: "False"
    type: Synced
  - lastTransitionTime: "2022-08-19T18:18:27Z"
    reason: Creating
    status: "False"
    type: Ready

This object fails to be cleaned up. notice the finalizers:

  finalizers:
  - finalizer.managedresource.crossplane.io

trying to manually delete the object also hangs because it is waiting on something.

it wont delete unless you remove the finalizers.

cannot tell what it's waiting on.

also its owner reference object has already confirmed to be auto deleted.

has no other finalizers.

the reconciliation problem should also not prevent it from being deleted right?

i also noticed that there is another Object that was auto cleaned up that had a dependsOn to this stuck object. could that be the reason?

i tried removing the dependsOn from the other object and this object on the next experiment cleaned up fine.
but why is it that the auto deleted object declaring dependsOn to this object results in this object not being cleaned up?

How can we reproduce it?

see yaml above.

What environment did it happen in?

Crossplane version:
1.9.0
ProviderKubernetes version:
v0.4.0

Include at least the version or commit of Crossplane you were running. Consider
also including your:

1.24.2-gke.1900

What problem are you facing?

kubectl apply supports applying remote manifests, such as manifests available on github.

When an operator chooses to use an alpha or beta version of an operator, it is necessary to apply updated CRDs files which are very large and would clutter a composition.

How could Crossplane help solve your problem?

Add the ability to source a manifest from a URL instead of including it in-line.

It is often necessary to declare dependencies between resources in a composition.
Some default crossplane cloud providers have dependencies between specific resources. The crossplane docs contain an example of a MySQL managed resource that depends on a resource group. The resource group reference fields are implemented in the manifest of the MySQLServer managed resource.

However for the kubernetes provider we would need a more arbitrary resource dependency declaration on the Object, because we don't know what the resource actually is. Maybe a dependsOn array of objects with resource references defined by group, version, kind, name and namespace.
The dependsOn property should be a sibling to the manifest property.
Only once status.condition named Ready is true for all referenced resource, the resource is actually templated and posted to the server.
We could reference resources from the composition or resources that aren't even part of the composition. For referenced resources within the composition we could simplify the reference and don't need to put all defining properties like group, version, kind, name and namespace, but rather only the name of the resource in the composition. All the rest can be deferred from the manifest of the referred resource in the composition.

What happened?

Now ARM64 image is available on the following repositories:

• https://hub.docker.com/r/crossplane/provider-kubernetes/tags
• https://hub.docker.com/r/crossplane/provider-kubernetes-controller
• https://hub.docker.com/r/crossplane/provider-kubernetes-controller

Even in some of these there are 2 ARCHS although both are defined as amd64.

How can we reproduce it?

What environment did it happen in?

Crossplane version:

Images prior to v0.2.0-rc.0.12.g4d722ef-main seem to have arm64 arch in this repository: https://hub.docker.com/r/crossplane/provider-kubernetes-controller/tags

Using provider-aws to create an EKS cluster and in a composition would be nice to use this provider to "patch" the aws-auth configMap with additional mapRoles or mapUsers.

Not sure if this should be implemented here or in provider-aws.

What problem are you facing?

When dealing with a large number of Objects it would be helpful to have additional columns in the kubectl get output to help with differentiating and identifying specific Objects.

For example, it would be helpful to have the Kind, apiVersion, metadata.name and metadata.namespace of the Kubernetes object that is in the manifest, and the providerConfigRef.name that was used, so that when needed the specific Object can be quickly identified.

I'll push a PR with changes for review

What problem are you facing?

Currently DependsOn is setting only finalizers on the dependant object and it is not enough to handle the cases like ordered deletion.

How could Crossplane help solve your problem?

Set ownerReferences on the parent object.

It will enable controlled ordered deletion with kubectl delete --cascade=foreground by deleting the child objects first.

Related links:

• https://kubernetes.io/blog/2021/05/14/using-finalizers-to-control-deletion/#owner-references
• crossplane/crossplane#3116

Scenario:
I have EKS cluster and I need to update aws-node daemonset environment variable.
I try to do it with following Object definition, this is part of composition:

  - name: aws-node-daemonset
    base:
      apiVersion: kubernetes.crossplane.io/v1alpha1
      kind: Object
      spec:
        managementPolicy: ObserveCreateUpdate
        providerConfigRef:
          name: kubernetes-provider
        forProvider:
          manifest:
            apiVersion: apps/v1
            kind: DaemonSet
            metadata:
              name: aws-node
              namespace: kube-system
            spec:
              template:
                spec:
                  containers:
                  - env:
                    - name: ENABLE_PREFIX_DELEGATION
                      value: "true"
                    name: aws-node

Result is that I get error update failed: cannot apply object: cannot patch object: DaemonSet.apps "aws-node" is invalid: spec.template.spec.containers[0].image: Required value

So it looks like I have to add complete daemonset definition to make it work but that's not optimal, I just want to modify existing a little bit.
Is there anyway to do some kind of "merge patch" to existing kubernetes object? I think that way this could work

What problem are you facing?

This issue is aimed to track all discussions around the design proposal documented in https://github.com/morningspace/capabilities-shim/blob/main/docs/enhanced-provider-k8s.md. It adds below features to provider-kubernetes.

• Allow user to define resource management policy to instruct the provider how to manage Kubernetes resources in a fine-grained manner.
• Allow user to define resource references for an Object as its dependencies to retrieve values from these dependent resources at runtime and guarantee the resource rendering in a specified order. (This implements the Generic references design at provider level).

There is a reference implementation that adds the above features at: https://github.com/morningspace/provider-kubernetes/tree/capabilities-shim, which is a repository forked from here.

How could Crossplane help solve your problem?

By implementing the above features, user can:

• Manage arbitrary Kubernetes resources using provider-kubernetes in a much fine-grained manner.
• Patch resource with values by resolving from its dependent resources.
• Guarantee both install and uninstall in specified order.

I try to deploy simple Kubernetes resources to the external cluster through crossplane/provider-kubernetes:v0.3.0 and got error:

Status:
  At Provider:
  Conditions:
    Last Transition Time:  2022-03-11T13:18:02Z
    Message:               observe failed: cannot get object: no matches for kind "Namespace" in version "v1"
    Reason:                ReconcileError
    Status:                False
    Type:                  Synced
Events:
  Type     Reason                         Age                From                                     Message
  ----     ------                         ----               ----                                     -------
Warning  CannotObserveExternalResource  10s (x5 over 23s)  managed/object.kubernetes.crossplane.io  cannot get object: no matches for kind "Namespace" in version "v1"

Could you help me, with how can I deploy some resources to the external cluster using crossplane kubernetes provider?

kind: ProviderConfig
kind: ProviderConfig
metadata:
  name: kubernetes-provider
spec:
  credentials:
    source: Secret
    secretRef:
      namespace: crossplane-system
      name: gke-cluster-secret
      key: kubeconfig
---
apiVersion: container.gcp.crossplane.io/v1beta2
kind: Cluster
metadata:
  name: workload-cluster
spec:
  forProvider:
............
  writeConnectionSecretToRef:
    namespace: crossplane-system
    name: gke-cluster-secret
---
apiVersion: kubernetes.crossplane.io/v1alpha1
kind: Object
metadata:
  name: flux-namespace
spec:
  forProvider:
    manifest:
      apiVersion: v1
      kind: Namespace
      metadata:
        name: flux-system
  providerConfigRef:
    name: dev-kubernetes-provider

What happened?

composition

       - type: ToCompositeFieldPath
          fromFieldPath: status.atProvider.manifest.status
          toFieldPath: status.nginx

  nginx:
    availableReplicas: 1
    conditions:
      - lastTransitionTime: '2022-12-19T02:11:59Z'
        lastUpdateTime: '2022-12-19T02:11:59Z'
        message: Deployment has minimum availability.
        reason: MinimumReplicasAvailable
        status: 'True'
        type: Available
      - lastTransitionTime: '2022-12-19T02:11:35Z'
        lastUpdateTime: '2022-12-19T02:11:59Z'
        message: ReplicaSet "abcd-7fd8bc7fcd" has successfully progressed.
        reason: NewReplicaSetAvailable
        status: 'True'
        type: Progressing
    replicas: 1
    unavailableReplicas: 1
    updatedReplicas: 1

deployment created by object

      status:
        availableReplicas: 1
        conditions:
        - lastTransitionTime: "2022-12-19T02:11:59Z"
          lastUpdateTime: "2022-12-19T02:11:59Z"
          message: Deployment has minimum availability.
          reason: MinimumReplicasAvailable
          status: "True"
          type: Available
        - lastTransitionTime: "2022-12-19T02:11:35Z"
          lastUpdateTime: "2022-12-19T02:11:59Z"
          message: ReplicaSet "abcd-7fd8bc7fcd" has successfully progressed.
          reason: NewReplicaSetAvailable
          status: "True"
          type: Progressing
        observedGeneration: 1
        readyReplicas: 1
        replicas: 1
        updatedReplicas: 1

status.nginx.unavailableReplicas of the XR is still exitst when status.unavailableReplicas key of the deployment create by object removed.

How can we reproduce it?

What environment did it happen in?

Crossplane version:
provider_kubernetes : provider-kubernetes-f935b3d8b7ec

kubernetes version 1.19