The GitOps concept originated from Weaveworks back in 2017 and the goal was to automate the operations of a Kubernetes (K8s) system using a model external to the system as the source of truth (History of GitOps).
This repository provides our opinionated point of view on how GitOps
can be used to manage the infrastructure, services and application layers of K8s based systems. It takes into account the various personas interacting with the system and accounts for separation of duties. The instructions and examples are focused around the Red Hat OpenShift platform and IBM Cloud Paks.
The reference architecture for this GitOps workflow can be found here.
- An OpenShift v4.7+ cluster is required.
-
Install the git CLI.
-
Install the OpenShift CLI oc (version 4.7+) . The binary can be downloaded from the Help menu from the OpenShift Console.
-
Log in from a terminal window.
oc login --token=<token> --server=<server>
-
The
IBM Entitlement Key
is required to pull IBM Cloud Pak specific container images from the IBM Entitled Registry. To get an entitlement key,- Log in to MyIBM Container Software Library with an IBMid and password associated with the entitled software.
- Select the View library option to verify your entitlement(s).
- Select the Get entitlement key to retrieve the key.
-
A Secret containing the entitlement key is created in the
tools
namespace.oc new-project tools || true oc create secret docker-registry ibm-entitlement-key -n tools \ --docker-username=cp \ --docker-password="<entitlement_key>" \ --docker-server=cp.icr.io
- The following set of Git repositories will be used for our GitOps workflow.
- Main GitOps repository (https://github.com/cloud-native-toolkit/multi-tenancy-gitops): This repository contains all the ArgoCD Applications for the
infrastructure
,services
andapplication
layers. Each ArgoCD Application will reference a specific K8s resource (yaml resides in a separate git repository), contain the configuration of the K8s resource, and determine where it will be deployed into the cluster. - Infrastructure GitOps repository (https://github.com/cloud-native-toolkit/multi-tenancy-gitops-infra): Contains the YAMLs for cluster-wide and/or infrastructure related K8s resources managed by a cluster administrator. This would include
namespaces
,clusterroles
,clusterrolebindings
,machinesets
to name a few. - Services GitOps repository (https://github.com/cloud-native-toolkit/multi-tenancy-gitops-services): Contains the YAMLs for K8s resources which will be used by the
application
layer. This could includesubscriptions
for Operators, YAMLs of custom resources provided, or Helm Charts for tools provided by a third party. These resource would usually be managed by the Administrator(s) and/or a DevOps team supporting application developers.
- Main GitOps repository (https://github.com/cloud-native-toolkit/multi-tenancy-gitops): This repository contains all the ArgoCD Applications for the
- Create a new GitHub Organization using instructions from this GitHub documentation.
- From each template repository, click the
Use this template
button and create a copy of the repository in your new GitHub Organization. - Clone the repositories locally.
mkdir -p gitops-repos cd gitops-repos # Clone using SSH git clone [email protected]:<GIT_ORG>/multi-tenancy-gitops.git git clone [email protected]:<GIT_ORG>/multi-tenancy-gitops-infra.git git clone [email protected]:<GIT_ORG>/multi-tenancy-gitops-services.git
- Update the default Git URl and branch references in your
multi-tenancy-gitops
repository by running the provided script./scripts/set-git-source.sh
script.cd multi-tenancy-gitops GIT_ORG=<GIT_ORG> GIT_BRANCH=master ./scripts/set-git-source.sh git commit -m "Update Git URl and branch references" git push origin master
- Red Hat OpenShift GitOps uses Argo CD, an open-source declarative tool, to maintain and reconcile cluster resources.
- Install the OpenShift GitOps Operator, create a
ClusterRole
and deploy a default instance of ArgoCD.oc apply -f setup/ocp47/ while ! oc wait crd applications.argoproj.io --timeout=-1s --for=condition=Established 2>/dev/null; do sleep 30; done while ! oc wait pod --timeout=-1s --for=condition=Ready -l '!job-name' -n openshift-gitops > /dev/null; do sleep 30; done
- Delete the default ArgoCD instance
oc delete gitopsservice cluster -n openshift-gitops || true oc delete argocd openshift-gitops -n openshift-gitops || true
- Create a custom ArgoCD instance with custom checks
oc apply -f gitops-0-bootstrap/setup/ocp47/argocd-instance/ -n openshift-gitops while ! oc wait pod --timeout=-1s --for=condition=ContainersReady -l app.kubernetes.io/name=openshift-gitops-cntk-server -n openshift-gitops > /dev/null; do sleep 30; done
- The bootstrap YAML follows the app of apps pattern.
- Select a profile and delete the others from the
0-bootstrap
directory. If this is your first usage of the gitops workflow, use thesingle-cluster
profile and deploy the ArgoCD Bootstrap Application.GITOPS_PROFILE="0-bootstrap/single-cluster" oc apply -f ${GITOPS_PROFILE}/bootstrap.yaml
- Retrieve the ArgoCD/GitOps URL and admin password:
oc get route -n openshift-gitops openshift-gitops-cntk-server -o template --template='https://{{.spec.host}}' oc extract secrets/openshift-gitops-cntk-cluster --keys=admin.password -n openshift-gitops --to=-
- Clone the
multi-tenancy-gitops
repository in your Git Organization if you have not already done so and select the K8s resources to deploy in the infrastructure and services layers. - Existing recipes are available and additional ones will be made available in the doc directory.
- Select a profile and delete the others from the
0-bootstrap
directory. If this is your first usage of the gitops workflow, Use thesingle-cluster
profile.GITOPS_PROFILE="0-bootstrap/single-cluster"
- Review the
Infrastructure
layer kustomization.yaml and un-comment the resources to deploy. - Review the
Services
layer kustomization.yaml and un-comment the resources to deploy. - Commit and push changes to your git repository
git add . git commit -m "initial bootstrap setup" git push origin
- Validate the recipe was deployed correctly following the
Validation
section in the recipe.