In this project, you will apply the skills you have acquired in this course to operationalize a Machine Learning Microservice API.
You are given a pre-trained, sklearn
model that has been trained to predict housing prices in Boston according to several features, such as average rooms in a home and data about highway access, teacher-to-pupil ratios, and so on. You can read more about the data, which was initially taken from Kaggle, on the data source site. This project tests your ability to operationalize a Python flask app—in a provided file, app.py
—that serves out predictions (inference) about housing prices through API calls. This project could be extended to any pre-trained machine learning model, such as those for image recognition and data labeling.
Your project goal is to operationalize this working, machine learning microservice using kubernetes, which is an open-source system for automating the management of containerized applications. In this project you will:
- Test your project code using linting
- Complete a Dockerfile to containerize this application
- Deploy your containerized application using Docker and make a prediction
- Improve the log statements in the source code for this application
- Configure Kubernetes and create a Kubernetes cluster
- Deploy a container using Kubernetes and make a prediction
- Upload a complete Github repo with CircleCI to indicate that your code has been tested
You can find a detailed project rubric, here.
The final implementation of the project will showcase your abilities to operationalize production microservices.
-
Install python3.7 and virtualenv using Homebrew
brew update brew install [email protected] virtualenv
-
Create a virtualenv with python3.7 (location of interpreter depends on homebrew install configuration, here the default location is used) and activate it
make setup-local source ~/.devops/bin/activate
-
Install python dependencies using requirements file
make install
-
Install hadolint to lint Dockerfile
brew install hadolint
-
Install pylint to lint python files
pip install pylint==2.7.4
-
Run lint on Dockerfile and python files
make lint
-
Run flask app locally
python app.py
-
Install docker-desktop to run containers
-
Execute the below bash script with defaults to build image and run as a container (note: host port 8000 is exposed)
./run_docker.sh
-
Install kubernetes cli
brew install kubernetes-cli
-
Install minikube
-
Start minikube cluster
minikube start
-
Execute the below bash script with defaults to upload an image to dockerhub
./upload_docker.sh
-
Execute the below bash script with defaults to run app in kubernetes (note: host port 8000 is exposed and re-run again after the first run, when the pod status is running -
kubectl get pods
)./run_kubernetes.sh
Run the flask app and execute the below bash script to make predictions
./make_prediction.sh
-
Install circleci
brew install circleci
-
Validate circleci config file
make validate-circleci
-
Run circleci locally circleci config file (note: make sure docker is running)
make run-circleci-local
Note: You can deactivate an active virtualenv by running the deactivate
command