paulsilcock / gke-terraform Goto Github PK

This will replace the need for a service account key

• VSCode recommended extensions
• VSCode auto-format/validate settings
• README quickstart
• CI linting/validation/plan
• Terraform apply on merge

• #13
  • Static manifests don't appear to be updated
  • Should be easier to configure static IP
• #14

Currently code is linted and validated to check for basic errors, and we output the terraform plan on pull requests.

It would be nice to verify policies - e.g. check that only secret-manager google service account can access secrets.

Creating ephemeral environments for pull requests would give further assurances that the generated terraform plan will apply and behave correctly (terratest looks useful here). One approach is to create GCP projects on-the-fly, although this requires a Google Workspace account if we want to do this programmatically from terraform (which costs money!) We could use raw gcloud commands to do this instead. Alternatively we could create/destroy clusters inside the current project, but there is a potentially troublesome lack of isolation here.

Testing ingress/static IP addressing could also prove tricky, although it looks like Google Domains can be imported into GCP. In tandem with Cloud DNS, we could create DNS records for pull request environments, e.g. *.pr5.pauljs.io.

Monitor deployed infrastructure and compare to expected Terraform plan, alert if there are changes!

Doing this would save money, as the e2-micro node used for cluster ingress would qualify for the free tier (https://cloud.google.com/free/docs/free-cloud-features#free-tier-usage-limits)

Note that a GKE cluster cannot be moved to a new region, so it would have to be destroyed and recreated!

Load balancer daily costs are somewhat prohibitive for a personal project!

KubeIP is one option - this monitors nodes in a GKE cluster and will assign them static IPs. We could create a single node in an ingress node pool, and use KubeIP to assign our static IP to it. The ingress-nginx controller would also need reconfiguring, to ensure it lands on the correct node, and that is uses the node's host network.

Currently the argo-workflow service account can submit workflows to the cluster, as well as publish Docker images (so they can be used in workflows). Other repositories will soon need to publish Docker images - rather than re-use the argo-workflow service account we should create a separate account with registry permissions only. This guards against being over-permissive. We should also remove registry permissions from the argo-workflow account.

• #24
• Remove roles/artifactregistry.writer from argo-workflow

• #11
  • Add client secret to Secret Manager
  • Add ExternalSecret object
• Add Dex config to argocd:argocd-cm ConfigMap
• Use secure backend for argocd-server
  • Remove --insecure flag from argocd:argocd-server Deployment container args:
  • Redirect to port 443 in argocd:argocd-server-ingress
• Create RBAC policy
  • Give single user admin
  • No default policy

Convert ExternalSecret from argocd namespace to a ClusterExternalSecret, and make available to argo and argocd namespaces. This will eventually allow Argo Workflows (which is installed to the argo namespace) to use SSO.