The project builds a Kubernetes cluster on a local environment using Vagrant to run a simple 'Hello World' web application written in Python.
At least 4 GBs of free memory available is recommended.
Also, following tools need to be installed on your machine:
- Oracle VirtualBox
- Ansible
- Vagrant
- Open a terminal on your machine.
- Go to root directory of the project.
- Run :
bash up.sh
Wait until the script ends (appr. 15-20 minutes).
Then open a browser and go to http://192.168.50.11:31320 to access the application.
git -C ~/ clone https://github.com/baturayozcan/python-helloworld.git
vagrant plugin install vagrant-reload
vagrant up
- clones the public repository of Python application
- installs vagrant-reload plugin
- runs Vagrantfile
Vagrant: to create local environment for Kubernetes cluster
VirtualBox: to be used as provider in Vagrant
Ansible: to build Kubernetes cluster inside VMs and make configurations for CI/CD operations
Skaffold: to build and deploy the application on the Kubernetes cluster
A simple 'Hello World' web application written in Python. It uses Flask as web server and MySQL as database. Database connection variables are read through environment variables and the application is run on port 3000.
You can access the public repository of the application through the link: https://github.com/baturayozcan/python-helloworld
import os
import flask
import pymysql
pymysql.install_as_MySQLdb()
import MySQLdb
application = flask.Flask(__name__)
application.debug = True
@application.route('/')
def hello_world():
storage = Storage()
storage.populate()
score = storage.score()
return "Hello Devops 123, %d!" % score
class Storage():
def __init__(self):
self.db = MySQLdb.connect(
user = os.getenv('MYSQL_USERNAME'),
passwd = os.getenv('MYSQL_PASSWORD'),
db = os.getenv('MYSQL_INSTANCE_NAME'),
host = os.getenv('MYSQL_PORT_3306_TCP_ADDR'),
port = int(os.getenv('MYSQL_PORT_3306_TCP_PORT'))
)
cur = self.db.cursor()
cur.execute("CREATE TABLE IF NOT EXISTS scores(score INT)")
def populate(self):
cur = self.db.cursor()
cur.execute("INSERT INTO scores(score) VALUES(1234)")
def score(self):
cur = self.db.cursor()
cur.execute("SELECT * FROM scores")
row = cur.fetchone()
return row[0]
if __name__ == "__main__":
application.run(host='0.0.0.0', port=3000)
In order MySQLdb module to work with Python3, a workaround solution is implemented: pymysql module is installed as MySQLdb.
import pymysql
pymysql.install_as_MySQLdb()
Flask
pymysql
These modules are required for the application and installed via pip in Dockerfile.
FROM alpine:3.12.1
RUN apk add --no-cache python3 && \
apk add --no-cache py-pip && \
apk add build-base && \
apk add python3-dev && \
apk add libffi-dev && \
apk add openssl-dev
RUN pip install cryptography
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .
CMD ["python3", "hello.py"]
pymysql module needs cryptography package to work proper and that package also requires build-base, python3-dev, libffi-dev and openssl-dev packages. Therefore all these packages are installed into the container image.
apiVersion: skaffold/v2beta10
kind: Config
metadata:
name: python-helloworld
build:
artifacts:
- image: hello-python
docker:
dockerfile: Dockerfile
local:
push: false
deploy:
kubectl:
manifests:
- /vagrant/manifests/app-deployment.yaml
When skaffold run command is run, skaffold reads and runs this file. It creates a container image using Dockerfile and apply the deployment yaml file (detailed in Manifests section.) onto the Kubernetes cluster. Container images are kept in VMs itself so they are not pushed to any registry.
local:
push: false
Ansible is used to create Kubernetes cluster and to make necessary configurations for build and deploy operations of the application into the cluster. Playbooks are run during the provision state of Vagrantfile.
---
- hosts: all
become: true
roles:
- docker
- swap
- k8s-binaries
- k8s-initialize
- kubeconfig
- flannel
- join-command
handlers:
- name: docker status
service: name=docker state=started
The script is run on Master Node of the Kubernetes cluster to do following operations respectively:
- install Docker and its dependencies
- add Vagrant user to docker group
- disable swap on the system, otherwise Kubelet will not start
- install Kubernetes binaries (kubelet, kubeadm, kubectl)
- initialize the cluster (10.244.0.0/16 subnet is set as pod network cidr so that flannel works proper)
- setup kubeconfig for vagrant user and fetch it to the local (will be used later on)
- install flannel as network plugin
- create a token for nodes to join the cluster and fetch it to the local as a file(will be used later on)
In order vagrant user to run docker operations on the VM, after adding vagrant user to docker group, logoff/login is needed. vagrant-reload plugin is installed to make it happen (will be detailed in Vagrantfile section). Tasks are separated as before and after the reload operation.
---
- hosts: all
become: true
roles:
- docker
- swap
- pwgen
- skaffold
- k8s-binaries
- kubejoin
handlers:
- name: docker status
service: name=docker state=started
The script is run on Worker Node of the Kubernetes cluster to do following operations respectively:
- install Docker and its dependencies
- add Vagrant user to docker group
- disable swap on the system, otherwise Kubelet will not start
- install pwgen package to generate random passwords (needed for MySQL, will be explained later in this section)
- install skaffold for build and deploy operations of the applicaton
- install Kubernetes binaries (kubelet, kubeadm, kubectl)
- copy kubejoin file (fetched before) to the node and run it to join the node into the cluster
---
- hosts: all
become: true
roles:
- launch
handlers:
- name: docker status
service: name=docker state=started
The script is run on Worker Node of the Kubernetes cluster to do following operations respectively:
- create .kube directory for vagrant user
- upload kubeconfig to the node (to run kubectl operations)
- install flannel as network plugin
- run launch.sh
#!/bin/bash
export DB_PASSWORD=`pwgen -Bs1 12`
kubectl apply -f /vagrant/manifests/mysql-deployment.yaml
kubectl apply -f /vagrant/manifests/mysql-service.yaml
kubectl apply -f /vagrant/manifests/app-service.yaml
cd /home/vagrant/hello-python
skaffold run
- generate a random password to be used as root password of the database while applying MySQL deployment
- apply MySQL deployment to the cluster
- apply a ClusterIP service for MySQL to the cluster
- apply a NodePort service for Python application to the cluster
- go to root directory of the Python application (synced with Vagrantfile, will be detailed in Vagrantfile section) and run skaffold run to start build and deploy operations of the Python application.
4 Kubernetes manifest files are created to apply:
- MySQL deployment
- MySQL ClusterIP service
- Python app deployment
- Python app NodePort service
apiVersion: apps/v1
kind: Deployment
metadata:
name: mysql
namespace: default
spec:
replicas: 1
selector:
matchLabels:
app: mysql
template:
metadata:
labels:
app: mysql
spec:
containers:
- name: mysql
image: mysql
ports:
- containerPort: 3306
env:
- name: MYSQL_ROOT_PASSWORD
value: "${DB_PASSWORD}"
- name: MYSQL_USER
value: "python"
- name: MYSQL_PASSWORD
value: "mysecretpassword"
- name: MYSQL_DATABASE
value: main
pulls mysql image from Dockerhub and runs it on port 3306 taking following environment variables: MYSQL_ROOT_PASSWORD: randomly generated
MYSQL_USER_PASSWORD: 'python' as username to be created for Python app
MYSQL_PASSWORD: 'mysecretpassword' as password to be created for Python app (since it is a local environment, security concerns ignored)
MYSQL_DATABASE: 'main' as database name to be created for Python app
apiVersion: v1
kind: Service
metadata:
name: service-mysql
spec:
selector:
app: mysql
ports:
- port: 3306
creates a ClusterIP type service to expose MySQL on port 3306
apiVersion: apps/v1
kind: Deployment
metadata:
name: hello-app
namespace: default
spec:
replicas: 1
selector:
matchLabels:
app: hello-app
template:
metadata:
labels:
app: hello-app
spec:
containers:
- name: hello-app
image: hello-python
ports:
- containerPort: 3000
env:
- name: MYSQL_USERNAME
value: "python"
- name: MYSQL_PASSWORD
value: "mysecretpassword"
- name: MYSQL_INSTANCE_NAME
value: "main"
- name: MYSQL_PORT_3306_TCP_ADDR
value: $(SERVICE_MYSQL_SERVICE_HOST)
- name: MYSQL_PORT_3306_TCP_PORT
value: "3306"
runs the Python application from the image 'hello-python' (created with skaffold) taking following environment variables:
MYSQL_USERNAME: set as 'python' while creating MySQL deployment
MYSQL_PASSWORD: set as 'mysecretpassword' while creating MySQL deployment
MYSQL_INSTANCE_NAME: set as 'main' while creating MySQL deployment
MYSQL_PORT_3306_TCP_ADDR: ClusterIP of the MySQL service as environment variable $(SERVICE_MYSQL_SERVICE_HOST)
MYSQL_PORT_3306_TCP_PORT: Port of the MySQL service as 3306
apiVersion: v1
kind: Service
metadata:
name: service-app
spec:
selector:
app: hello-app
ports:
- port: 3000
nodePort: 31320
protocol: TCP
targetPort: 3000
type: NodePort
creates a NodePort type service to expose Python app on port 31320 of the node
IMAGE_NAME = "ubuntu/focal64"
N = 1
Vagrant.configure("2") do |config|
config.ssh.insert_key = false
config.vm.provider "virtualbox" do |v|
v.memory = 4096
v.cpus = 4
end
config.vm.define "k8s-master" do |master|
master.vm.box = IMAGE_NAME
master.vm.network "private_network", ip: "192.168.50.10"
master.vm.hostname = "k8s-master"
master.vm.provision "ansible" do |ansible|
ansible.playbook = "ansible/master-playbook.yaml"
ansible.extra_vars = {
node_ip: "192.168.50.10",
}
end
end
(1..N).each do |i|
config.vm.define "node-#{i}" do |node|
node.vm.box = IMAGE_NAME
node.vm.network "private_network", ip: "192.168.50.#{i + 10}"
node.vm.hostname = "node-#{i}"
node.vm.provision "ansible" do |ansible|
ansible.playbook = "ansible/node-playbook-before.yaml"
ansible.extra_vars = {
node_ip: "192.168.50.#{i + 10}",
}
end
node.vm.provision :reload
node.vm.provision "ansible" do |ansible|
ansible.playbook = "ansible/node-playbook-after.yaml"
ansible.extra_vars = {
node_ip: "192.168.50.#{i + 10}",
}
end
node.vm.synced_folder "~/python-helloworld/", "/home/vagrant/hello-python"
end
end
end
Creates a local environment with 2 virtual machines (1 for Kubernetes Master Node, 1 for Kubernetes Worker Node) using ubuntu/focal64 as base image and virtualbox as provider. Virtual memory and virtual CPU values are set as 4096 and 4 respectively for each VM to avoid resource issues.
On the master node,
- IP address is set as '192.168.50.10'
- Hostname as 'k8s-master'
- 'master-playbook.yaml' is run in the provision state
On the worker node,
- IP address is set as automatically '192.168.50.#{i + 10}'
- Hostname as 'node-#{i}'
- 'node-playbook-before.yaml' is run in the first provision state
- then VM is reloaded in order vagrant user to run docker operations
- 'node-playbook-after.yaml' is run in the last provision state
- '~/python-helloworld/' folder (Python app repository is cloned here) is synced to the '/home/vagrant/hello-python' path in the VM
Continous Delivery cannot be implemented. Many solutions has been tried but the main problem is to notify the VM when there is a file change on the local machine. Although the application directory is synced with Vagrant, VM cannot detect the changes (File sync works by the way). Solutions like vagrant-notify-forwarder or vagrant-fsnotify did not work. The main objective was to detect file changes in VM and run skaffold run.
Before thinking to run skaffold in the VM, running it on the local has also been tried. However, issue with that approach was pushing container images into the VM. Creating a private Docker registry in the VM might be the solution but it also came with other issues and canceled.
Therefore, after the code changes, vagrant ssh node-1 -- -c 'cd /home/vagrant/hello-python && skaffold run; /bin/bash' should be run manually on local machine or skaffold run in the VM.