cnoe-io / proposals Goto Github PK

We want to be able to have a way to have a core set of components that CNOE supports and that tools like IDPBuilder have an easy way and schema to add supported packages while being able to make "non-supported" changes. As a platform engineer we should have a way to quickly add components to IDPBuilder or our GitOps repo that follow a "CNOE" standard on application/applicationset definition. Ideally IDPBuilder becomes a plug and play model where testing components from your actual GitOps in a local environment.

Notes to consider surrounding this proposal:

1. We need to define a specification for what CNOE expects "plug and play" packages to look like. We may want to consider moving towards applicationsets rather then applications in ArgoCD.
2. We can look at the patterns in https://github.com/gitops-bridge-dev to determine how we want to define that specification.
3. We do not want to become a package manager or worry about dealing with dependencies at scale. We should consider versioning addons with some sort of release process and only guarantee that the versions specified in that release work with each other. We do want to allow developers to override or configure additional resources but CNOE will not support anything broken with the difference in versions from the "standard".
4. We will want to have a way for developers to easily add "CNOE Supported" packages. We will want a static repository for those packages. We may want to consider a static public Github repository for that like https://github.com/gitops-bridge-dev/gitops-bridge-argocd-control-plane-template where we can grab the packages from Github directly, deal with tags/releases to track standard package versions, etc;.
5. Standard Packages should be omitted from any binaries or not become "required" to run IDPBuilder or CNOE. We want to keep the necessary components limited to tools like ArgoCD, Ingress NGINX, and Gitea (for local IDPBuilder)

Right now, idpBuilder only supports deploying to Kind clusters.

While this works great for development and test purposes, often times users want to either test the IDP on a remote cluster, or they want to bootstrap a staging environment to carry their development work over. This issue proposes that we should take action on the following:

1. Allow for idpBuilder to deploy to a remote cluster
2. Support bootstrapping of a managed Kubernetes cluster (starting with EKS)

Allow for idpBuilder to deploy to a remote cluster

During the process of developing an IDP, the idpBuidler deploys a git repository, the Argo workflows, and nginx. Building a more complex IDP involves extending on top of these tools and deploying other Argo applications in the form of a pre-configured "stack". However, the process of deploying the stack to a staging environment requires the users to create a Kubernetes cluster, deploy Argo CD and nginx, and configure their Git repository to work with the tooling. They should then push the developed "stack" to this remote repo and continue on with the work of configuring their stack. This creates a disjoint experience. If the idpBuidler can deploy its core packages as well as the developed stack directly to the core cluster, saving energy and effort for the platform engineers.

• requirement is for users to setup a different git repository other than Gitea as the repository to utilize (starting with GitHub).

Support bootstrapping a managed Kubernetes cluster

The previous proposal requires users to bring in their own managed Kubernetes cluster for the idpBuilder to deploy to. This can be a cumbersome task given that the cluster needs to be created out of band and permissions need to be managed for the cluster to have sufficient permissions when running its IaC tools.

For the EKS, with the introduction of PodIdentity into EKS, it appears that cluster level permissions can be better handled by creating and connecting IAM roles to services accounts. The support is natively available in eksctl as well. Given the new developments, this proposal suggests that we should wrap the eksctl functionality to create an EKS cluster and include PodIdentity add-on as well as capabilities to it in such a way that idpBuilder is capable of bootstrapping and creating EKS clusters.

We can later on extend on this plan and implement the Provider API proposed here cnoe-io/idpbuilder#183 and offer a more generic approach that would support managed Kubernetes clusters other than EKS as well.