• This repository contains source code for my VTI DevOps course.

• I created a simple http server by golang.
• Build and run command

docker build -t dinhtranvan/simple-http-server .
docker run -p 8080:8080 dinhtranvan/simple-http-server

• I created 3 files.
  • deployment.yaml: for deploying pod.
  • service.yaml: for route.
  • deploy-k8s.sh: for deploying 2 above files to k8s.

• Creating a secrect store in k8s.

  • Created a new file secret.yaml to store data. The data is encoded by base64 format.

  apiVersion: v1
  kind: Secret
  metadata:
    name: secret
  # `Opaque` means that the secret is not decoded by Kubernetes.
  type: Opaque
  data:
    username: ZGluaC10cmFu
    password: ZGluaDE5OTA=

• Access value in the container.

  • Mount secret store as a volume in deployment.yaml. K8s automatically decodes base64 value.

  containers:
  - name: simple-http-server
    image: dinhtranvan/simple-http-server:v2
    ports:
    - containerPort: 8080
    volumeMounts:
    - name: secret-volume
      mountPath: /etc/secret-volume
      readOnly: true
  
  volumes:
  - name: secret-volume
    secret:
      secretName: secret

• Access the data by go to link http://localhost:8080/credential

• Using Github action runner in file
• Diagrams

---
title: Github Wokflow
---
flowchart TB
    checkout-source[Checkout source] -->
    build-docker-image[Build Docker Image] -->
    push-docker-image[Push Docker Image] -->
    install-aws-cli[Install AWS CLI] -->
    install-kubectl[Install kubectl] -->
    connect-kubectl-with-EKS[Connect kubectl with EKS] -->
    update-image-version[Update Image Version in Deployment] -->
    deploy-k8s[Deploy K8s]

• Provision a K8s cluster on AWS EKS Service by using Terraform. Link to source code.

• Provision a K8s cluster
• Setup a github action pipeline.
  • Setup environment variables and secret keys.
  • Build docker image.
  • Push the build image to the Docker registry.
  • Connect kubectl to the k8s cluster.
  • Deploy to EKS.
  • (Optional) Auto run unit tests.
  • (Optional) Auto run E2E tests.

1. Adding a new github workflow action in folder .github/workflows/deploy-to-k8s.yaml.
2. In github repository page, go to Settings -> Secrets and variables. Add new environment variables and secret keys.

3. Go to folder terraform/k8s then deployment an EKS cluster on AWS. After deployment, get the cluster name by below command.

terraform output -raw cluster_name

Note The process takes approximately 20 minutes, be patient.

4. Replace the secret EKS_CLUSTER_NAME by cluster name value.
5. Push code to trigger the pipeline.
6. Destroy the EKS cluster to avoid unnecessary charge.

Important When deploying k8s services, AWS will automatically create a Load Balancer which is not provisioned by terraform. When teardown resources, it'll cause error when detroying the VPC. To solve this, go to AWS console, VPC, Load Balancer, then delete it and run the command terraform destroy again.

• Setup a CI/CD pipeline by using Jenkins.
• (Optional) Host the Jenkins with Docker server on AWS EC2 using terraform.

Important Jenkins server needs an public address for configure webhook. If you don't want to provision the Jenkins server on AWS EC2, consider using ngrok for obtain a temporary public address.

• Steps diagram:

---
title: Jenkins
---
flowchart TB
    provision-the-jenkins-server[Provision the Jenkins server] -->
    obtain-github-credential[Obtain Github credential] -->
    define-build-steps[Define Build Steps] -->
    setup-pipeline[Setup pipeline] -->
    adding-build-trigger[Adding Build Trigger] -->
    setup-docker["(Optional) Setup Docker"] -->
    add-build-image-step[Add Build Image Step] -->
    add-push-image-step[Add Push Image Step]

1. Provision an Jenkin server in AWS by source.
2. SSH to the newly created Jenkins server and get admin password.
3. Go to server page, port 8080. Install recommend plugins and create a new user.

• To access to Github data, Jenkins needs a username/password key.
• Go to Personal Settings (not project setting), Developer settings, Personal access token, Fine-grained tokens.

• Generate a new token, choose the correspondence project repository. In repository permission list, grant read-only for:
  • Access to code.
  • Commit status and metadata.

• Click on generate button then copy the token.
• Go back to Jenkins, go to Dashboard, Manage Jenkins, Security, Credential, System, Global credentials (unrestricted).

• Add a new credential:

• Add a new Jenkinsfile to the source code which contains very basic steps.

pipeline {
  agent any
  stages {
        stage('Build') {
            steps {
                echo 'Hello World'
            }
        }
        stage('Test') {
            steps {
                echo 'Test'
            }
        }
        stage('Deploy') {
            steps {
                echo 'Deploy'
            }
        }
    }
}

• Go back to Jenkins Dashboard. On the left panel, click on + New Item, fill Enter an item name then click on the Pipeline option.

• In the Configure page, fill the description.
• In General section, check GitHub project and fill the project repository url.

Important The github repository url must contain .git at the end. For example: https://github.com/dinh-van-tran/VTI.git.

• In Build Triggers section, select checkbox GitHub hook trigger for GITScm polling for later step Adding Build Trigger.
• Scroll to the end of page, in the Pipeline section, from Definition dropdown, choose Pipeline script from SCM.
  • Select SCM value Git.
  • Fill Repository URL the same value as General section.
  • Click on Credentials dropdown, select the newly created Github credential.
  • In Branches to build, fill the desired branch. In case build happens when there are commits to main branch, fill */main.
  • In Script Path, fill Jenkinsfile.
• Click Save.

• In the left panel, click on Build Now to trigger pipeline. If everything is correct, you'll see the result like this.

• If you want to trigger the Jenkins server build when new code is pushed to the repository, you need to setup a webhook in Github for notifying the Jenkins server.
• Make sure you've already selected the option GitHub hook trigger for GITScm polling when creating the pipeline.
• Go back to the Github repository, go to Settings, Webhooks then register a new webhook for the Jenkins server.
  • Payload URL: http://jenkins_public_ip:8080/github-webhook/

Important Payload url must have slash character / at the end.

• Content type: application/x-www-form-urlencoded
• Secret: empty
• Which events would you like to trigger this webhook: Just the push event

• In case you don't provision the Jenkins server by terraform script, you have to setup Docker manually.

1. Loggin to the Jenkins server by SSH.
2. Install Docker on the Jenkins server following this link
3. Authorize Jenkins running docker by running below command.

sudo usermod -aG docker jenkins

4. Restart the Jenkins server by go to the link https://jenkins_pubic_ip:8080/restart.

• Modify the build step on Jenkinsfile as follow.

stage('Build Docker Image') {
    steps {
        script {
            def gitSha = sh(returnStdout: true, script: 'git rev-parse HEAD').trim()
            sh "docker build -t dinhvantran/simple-http-server:${gitSha} ."
        }
    }
}

1. In the Jenkins server, go to Dashboad, Manage Jenkins, Credentials, add a new credential for docker hub.

2. In Jenkinsfile, add a new step push image step

stage('Push Docker Image') {
    steps {
        withCredentials([usernamePassword(credentialsId: 'docker-hub-credentials', usernameVariable: 'DOCKER_USERNAME', passwordVariable: 'DOCKER_PASSWORD')]) {
            script {
                def gitSha = sh(returnStdout: true, script: 'git rev-parse HEAD').trim()
                sh "docker login -u ${DOCKER_USERNAME} -p ${DOCKER_PASSWORD}"
                sh "docker push dinhtranvan/simple-http-server:${gitSha}"
            }
        }
    }
}

• ArgoCD is a CD tool which was built specifically for K8s.

• Configure an application on Argocd that listens changes on k8s folder.
• Modify the Github action:
  • Ignore the k8s folder changes.
  • Replace image version in the file deployment.
  • Automatically commit/push new changes in k8s folder to the git repository.
• Argocd will detect changes in the k8s folder and apply the app accordingly.

---
title: CI/CD Workflow
---
flowchart TB
    subgraph github-action[Github Action]
      commit-code[Commit Code] -->
      build-push-docker-image[Build/Push Docker Image] -->
      make-changes-in-k8s-folder[Make changes in k8s folder] -->
      push-new-k8s-change-to-git[Push new k8s changes to git]
    end

    subgraph git-repo[Git Repository]
    end

    subgraph argocd[Argocd]
      detect-new-changes-in-k8s-folder[Detect new changes in k8s folder] -->
      apply-k8s-changes[Apply k8s changes]
    end

    push-new-k8s-change-to-git --> git-repo
    detect-new-changes-in-k8s-folder --> git-repo

• By default, github action doesn't have permission to push new commits.
• Go to the github repository, Settings, Actions, Worflow permissions then select Read and write permissons.

• Follow steps in terraform/k8s to provision an EKS cluster on AWS.

kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

brew install argocd

kubectl port-forward svc/argocd-server -n argocd 8080:443

• Export variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY in the terminal then run following code.

argocd admin initial-password -n argocd

• By CLI

argocd app create simple-http-server \
  --repo https://github.com/dinh-van-tran/VTI.git \
  --path k8s \
  --revision main \
  --dest-server https://kubernetes.default.svc 
  --dest-namespace default

• By UI, go to argocd to create a new application.