- This repository contains source code for my VTI DevOps course.
- I created a simple http server by golang.
- Build and run command
docker build -t dinhtranvan/simple-http-server .
docker run -p 8080:8080 dinhtranvan/simple-http-server
- I created 3 files.
deployment.yaml
: for deploying pod.service.yaml
: for route.deploy-k8s.sh
: for deploying 2 above files to k8s.
-
Creating a secrect store in k8s.
- Created a new file secret.yaml to store data. The data is encoded by base64 format.
apiVersion: v1 kind: Secret metadata: name: secret # `Opaque` means that the secret is not decoded by Kubernetes. type: Opaque data: username: ZGluaC10cmFu password: ZGluaDE5OTA=
-
Access value in the container.
- Mount secret store as a volume in deployment.yaml. K8s automatically decodes base64 value.
containers: - name: simple-http-server image: dinhtranvan/simple-http-server:v2 ports: - containerPort: 8080 volumeMounts: - name: secret-volume mountPath: /etc/secret-volume readOnly: true volumes: - name: secret-volume secret: secretName: secret
-
Access the data by go to link http://localhost:8080/credential
- Using Github action runner in file
- Diagrams
---
title: Github Wokflow
---
flowchart TB
checkout-source[Checkout source] -->
build-docker-image[Build Docker Image] -->
push-docker-image[Push Docker Image] -->
install-aws-cli[Install AWS CLI] -->
install-kubectl[Install kubectl] -->
connect-kubectl-with-EKS[Connect kubectl with EKS] -->
update-image-version[Update Image Version in Deployment] -->
deploy-k8s[Deploy K8s]
- Provision a K8s cluster on AWS EKS Service by using Terraform. Link to source code.
- Provision a K8s cluster
- Setup a github action pipeline.
- Setup environment variables and secret keys.
- Build docker image.
- Push the build image to the Docker registry.
- Connect kubectl to the k8s cluster.
- Deploy to EKS.
- (Optional) Auto run unit tests.
- (Optional) Auto run E2E tests.
- Adding a new github workflow action in folder
.github/workflows/deploy-to-k8s.yaml
. - In github repository page, go to
Settings
->Secrets and variables
. Add new environment variables and secret keys.
Name | Type | Usage |
---|---|---|
DOCKER_USERNAME |
Secret | For pushing docker image. |
DOCKER_PASSWORD |
Secret | For pushing docker image. |
AWS_ACCESS_KEY_ID |
Secret | Authorize AWS for connect to EKS. |
AWS_SECRET_ACCESS_KEY |
Secret | Authorize AWS for connect to EKS. |
EKS_CLUSTER_NAME |
Secret | Connect to EKS cluster. Leave it empty. |
- Go to folder
terraform/k8s
then deployment an EKS cluster on AWS. After deployment, get the cluster name by below command.
terraform output -raw cluster_name
Note The process takes approximately 20 minutes, be patient.
- Replace the secret
EKS_CLUSTER_NAME
by cluster name value. - Push code to trigger the pipeline.
- Destroy the EKS cluster to avoid unnecessary charge.
Important When deploying k8s services, AWS will automatically create a Load Balancer which is not provisioned by terraform. When teardown resources, it'll cause error when detroying the VPC. To solve this, go to AWS console, VPC, Load Balancer, then delete it and run the command
terraform destroy
again.
- Setup a CI/CD pipeline by using Jenkins.
- (Optional) Host the Jenkins with Docker server on AWS EC2 using terraform.
Important Jenkins server needs an public address for configure webhook. If you don't want to provision the Jenkins server on AWS EC2, consider using ngrok for obtain a temporary public address.
- Steps diagram:
---
title: Jenkins
---
flowchart TB
provision-the-jenkins-server[Provision the Jenkins server] -->
obtain-github-credential[Obtain Github credential] -->
define-build-steps[Define Build Steps] -->
setup-pipeline[Setup pipeline] -->
adding-build-trigger[Adding Build Trigger] -->
setup-docker["(Optional) Setup Docker"] -->
add-build-image-step[Add Build Image Step] -->
add-push-image-step[Add Push Image Step]
- Provision an Jenkin server in AWS by source.
- SSH to the newly created Jenkins server and get admin password.
- Go to server page, port 8080. Install recommend plugins and create a new user.
- To access to Github data, Jenkins needs a username/password key.
- Go to Personal Settings (not project setting),
Developer settings
,Personal access token
,Fine-grained tokens
.
- Generate a new token, choose the correspondence project repository. In repository permission list, grant read-only for:
- Access to code.
- Commit status and metadata.
- Click on generate button then copy the token.
- Go back to Jenkins, go to Dashboard,
Manage Jenkins
,Security
,Credential
,System
,Global credentials (unrestricted)
.
- Add a new credential:
Field | Value |
---|---|
Kind |
Username and password |
Scope |
Global. |
Username |
Github username. |
Treat username as secret |
uncheck. |
Pasword |
Fine-grained token value. |
ID |
Github. |
- Add a new
Jenkinsfile
to the source code which contains very basic steps.
pipeline {
agent any
stages {
stage('Build') {
steps {
echo 'Hello World'
}
}
stage('Test') {
steps {
echo 'Test'
}
}
stage('Deploy') {
steps {
echo 'Deploy'
}
}
}
}
- Go back to Jenkins Dashboard. On the left panel, click on
+ New Item
, fillEnter an item name
then click on thePipeline
option.
- In the
Configure
page, fill the description. - In
General
section, checkGitHub project
and fill the project repository url.
Important The github repository url must contain
.git
at the end. For example:https://github.com/dinh-van-tran/VTI.git
.
- In
Build Triggers
section, select checkboxGitHub hook trigger for GITScm polling
for later stepAdding Build Trigger
. - Scroll to the end of page, in the
Pipeline
section, fromDefinition
dropdown, choosePipeline script from SCM
.- Select
SCM
valueGit
. - Fill
Repository URL
the same value asGeneral
section. - Click on
Credentials
dropdown, select the newly created Github credential. - In
Branches to build
, fill the desired branch. In case build happens when there are commits tomain
branch, fill*/main
. - In
Script Path
, fillJenkinsfile
.
- Select
- Click
Save
.
- In the left panel, click on
Build Now
to trigger pipeline. If everything is correct, you'll see the result like this.
- If you want to trigger the Jenkins server build when new code is pushed to the repository, you need to setup a webhook in Github for notifying the Jenkins server.
- Make sure you've already selected the option
GitHub hook trigger for GITScm polling
when creating the pipeline. - Go back to the Github repository, go to
Settings
,Webhooks
then register a new webhook for the Jenkins server.Payload URL
:http://jenkins_public_ip:8080/github-webhook/
Important Payload url must have slash character
/
at the end.
Content type
:application/x-www-form-urlencoded
Secret
: emptyWhich events would you like to trigger this webhook
:Just the push event
- In case you don't provision the Jenkins server by terraform script, you have to setup Docker manually.
- Loggin to the Jenkins server by SSH.
- Install Docker on the Jenkins server following this link
- Authorize Jenkins running docker by running below command.
sudo usermod -aG docker jenkins
- Restart the Jenkins server by go to the link
https://jenkins_pubic_ip:8080/restart
.
- Modify the build step on Jenkinsfile as follow.
stage('Build Docker Image') {
steps {
script {
def gitSha = sh(returnStdout: true, script: 'git rev-parse HEAD').trim()
sh "docker build -t dinhvantran/simple-http-server:${gitSha} ."
}
}
}
- In the Jenkins server, go to
Dashboad
,Manage Jenkins
,Credentials
, add a new credential for docker hub.
Field | Value |
---|---|
Kind |
Username and password |
Scope |
Global. |
Username |
Your docker hub username. |
Treat username as secret |
check. |
Pasword |
Your docker hub password. |
ID |
docker-hub-credentials . |
- In Jenkinsfile, add a new step push image step
stage('Push Docker Image') {
steps {
withCredentials([usernamePassword(credentialsId: 'docker-hub-credentials', usernameVariable: 'DOCKER_USERNAME', passwordVariable: 'DOCKER_PASSWORD')]) {
script {
def gitSha = sh(returnStdout: true, script: 'git rev-parse HEAD').trim()
sh "docker login -u ${DOCKER_USERNAME} -p ${DOCKER_PASSWORD}"
sh "docker push dinhtranvan/simple-http-server:${gitSha}"
}
}
}
}
- ArgoCD is a CD tool which was built specifically for K8s.
- Configure an application on Argocd that listens changes on k8s folder.
- Modify the Github action:
- Ignore the k8s folder changes.
- Replace image version in the file deployment.
- Automatically commit/push new changes in k8s folder to the git repository.
- Argocd will detect changes in the k8s folder and apply the app accordingly.
---
title: CI/CD Workflow
---
flowchart TB
subgraph github-action[Github Action]
commit-code[Commit Code] -->
build-push-docker-image[Build/Push Docker Image] -->
make-changes-in-k8s-folder[Make changes in k8s folder] -->
push-new-k8s-change-to-git[Push new k8s changes to git]
end
subgraph git-repo[Git Repository]
end
subgraph argocd[Argocd]
detect-new-changes-in-k8s-folder[Detect new changes in k8s folder] -->
apply-k8s-changes[Apply k8s changes]
end
push-new-k8s-change-to-git --> git-repo
detect-new-changes-in-k8s-folder --> git-repo
- By default, github action doesn't have permission to push new commits.
- Go to the github repository,
Settings
,Actions
,Worflow permissions
then selectRead and write permissons
.
- Follow steps in terraform/k8s to provision an EKS cluster on AWS.
kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml
brew install argocd
kubectl port-forward svc/argocd-server -n argocd 8080:443
- Export variables
AWS_ACCESS_KEY_ID
andAWS_SECRET_ACCESS_KEY
in the terminal then run following code.
argocd admin initial-password -n argocd
- By CLI
argocd app create simple-http-server \
--repo https://github.com/dinh-van-tran/VTI.git \
--path k8s \
--revision main \
--dest-server https://kubernetes.default.svc
--dest-namespace default
- By UI, go to argocd to create a new application.