I've pulled master for all NEONFORGE repos and am seeing malformed rolebinding files being generated for neon-cluster-operator and neon-cluster-agent. Here's an example:

Currently they only work on controllers

It appears that the OperatorSDK is not generating when building using MSBUILD:

C:\src\neonKUBE\Services\neon-cluster-operator\OperatorStartup.cs(91,18): error CS1061: 'IServiceCollection' does not contain a definition for 'AddKubernetesOperator' and no accessible extension method 'AddKubernetesOperator' accepting a first argument of type 'IServiceCollection' could be found (are you missing a using directive or an assembly reference?) [C:\src\neonKUBE\Services\neon-cluster-operator\neon-cluster-operator.csproj]
C:\src\neonKUBE\Services\neon-cluster-operator\OperatorStartup.cs(105,17): error CS1061: 'IApplicationBuilder' does not contain a definition for 'UseKubernetesOperator' and no accessible extension method 'UseKubernetesOperator' accepting a first argument of type 'IApplicationBuilder' could be found (are you missing a using directive or an assembly reference?) [C:\src\neonKUBE\Services\neon-cluster-operator\neon-cluster-operator.csproj]

I've noticed that when I open the NEONCLOUD solution with :\src\neonKUBE\Services\neon-cluster-operator\OperatorStartup.cs open that it takes sometime for the call to UseKubernetesOperator() there turns blue, indicating that the method has been generated and is actually valid. I also notice that the code is generated in a user temp directory and not in a git folder, so we can't rely on source files generated previously in a VS session that gets committed to GitHub.

I wonder if there's a way to have MSBUILD run all analyzers before building?

Currently we're seeing changes to RBAC rules which is annoying, especially for manifests included in version control.

public static class Constants
{
    public static class Olm
    {
        public const string OperatorName = "garnet-operator";
    }
}

Referencing the above like this doesn't work

[assembly: Name(Name = Constants.Olm.OperatorName)]

Enum validation for Custom Resource Definitions is currently not supported.

We're seeing CRDs being generated for resources with the [Ignore] attribute. It seems to be intermittent.

Currently the ResourceManager doesn't check permissions for dependent resources.

The neon-cluster-operator service specifies the following RBAC rules:

[RbacRule<V1ConfigMap>(Verbs = RbacVerb.All, Scope = EntityScope.Cluster)]
[RbacRule<V1Secret>(Verbs = RbacVerb.All, Scope = EntityScope.Cluster)]
[RbacRule<V1Pod>(Verbs = RbacVerb.List, Scope = EntityScope.Namespaced, Namespace = KubeNamespace.NeonSystem)]
public partial class Service : NeonService

The OperatorSDK generates two role files:

clusterrole-neon-cluster-operator.yaml
role-neon-cluster-operator.yaml

clusterrole-neon-cluster-operator.yaml looks OK:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: neon-cluster-operator
rules:
- apiGroups:
  - ""
  resources:
  - configmaps
  - namespaces
  - nodes
  - pods
  - secrets
  verbs:
  - '*'
- apiGroups:
  - ""
  resources:
  - secrets
  verbs:
  - get
  - update
- apiGroups:
  - admissionregistration.k8s.io
  resources:
  - mutatingwebhookconfigurations
  verbs:
  - '*'
- apiGroups:
  - minio.min.io
  resources:
  - tenants
  verbs:
  - '*'
- apiGroups:
  - minio.neonkube.io
  resources:
  - miniobuckets
  - miniobuckets/status
  verbs:
  - '*'
- apiGroups:
  - neonkube.io
  resources:
  - crioconfigurations
  - crioconfigurations/status
  - neonclusterjobs
  - neonclusterjobs/status
  - neoncontainerregistries
  - neoncontainerregistries/status
  - neondashboards
  - neondashboards/status
  - neonnodetasks
  - neonnodetasks/status
  - neonssocallbackurls
  - neonssocallbackurls/status
  - neonssoclients
  - neonssoclients/status
  verbs:
  - '*'

but role-neon-cluster-operator.yaml should be specifying the target namespace in it's metadata, right?

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: neon-cluster-operator
rules:
- apiGroups:
  - ""
  resourceNames:
  - neon-sso-oauth2-proxy
  resources:
  - configmaps
  verbs:
  - get
  - update
- apiGroups:
  - ""
  resources:
  - pods
  verbs:
  - list
- apiGroups:
  - ""
  resources:
  - secrets
  verbs:
  - get
- apiGroups:
  - ""
  resourceNames:
  - neon-cluster-operator-webhook-tls
  resources:
  - secrets
  verbs:
  - watch
- apiGroups:
  - cert-manager.io
  resources:
  - certificates
  verbs:
  - '*'
- apiGroups:
  - coordination.k8s.io
  resources:
  - leases
  verbs:
  - '*'

So, I think the namespace role is actually being applied only to the default namespace right now.

Note that it's possible for a service to reference multiple namespaces, so you'll need to generate multiple role manifests in this file, one per referenced namespace, separating these with "---" lines.

XML docs containing <see ... or other references are missing from CRDs.

What would you think of having the OperatorSDK generated file have a unique file type like .osdk.yaml, ogen.yaml, or something?

The idea here is to make it easy to .gitignore all of these files. Right now, we're explicitly ignoring these by path, but that's fragile because it's very possible to to add a new RBAC conditions that will generate new files that will probably get tracked by Git until we figure that out and have to manually ignore them,

Depends on nforgeio/neonSDK#78

The CRD analyzer can't find enums if they are not in the root namespace.

It would be nice to generate the bundle needed for publishing to OperatorHub.io

Contributing

The main idea is to configure this by using assembly attributes.

[assembly: global::Neon.Operator.Attributes.OperatorName("example-operator")]
[assembly: global::Neon.Operator.Attributes.OperatorDisplayName("ExampleOperator")]
[assembly: global::Neon.Operator.Attributes.OperatorDescription("This is an example operator.")]

• Define V1ClusterServiceVersion resource in Neon.Kubernetes.Resources. See spec and example.
• Define Assembly Attributes for properties that should be configurable in the CSV.
  • Name
  • DisplayName
  • Annotation
  • Description
  • Keyword
  • Maintainer
  • Maturity
  • Provider
  • Version
  • MinKubeVersion
  • InstallMode
  • OwnedResource
  • OwnedResource<T>
  • RequiredResource
  • RequiredResource<T>

Example:

[AttributeUsage(AttributeTargets.Assembly, Inherited = false, AllowMultiple = false)]
public sealed class OperatorNameAttribute : Attribute
{
    public string Name { get; set; }
    public OperatorNameAttribute(string name)
    {
        this.Name = name;
    }
}

• Generate the operator.clusterserviceversion.yaml. We will use the assembly attributes from above to define these settings
• Generate annotations.yaml
• Generate the Dockerfile

Dev tunnels are now included in Visual Studio GA and the docs are out of date.

The Xunit lib should support fieldSelector and labelSelector in the API server.

Test fixture should support dependency injection

Add an attribute to support this: https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/#validation-rules

The Xunit lib should support API watches

ResourceApiController in Neon.Operator.Xunit is missing GET, PUT, PATCH and DELETE methods.