The following is an end-to-end example of how to build a machine learning model, deploy it as an API, and test it for an specific use case: flight delay prediction. The model was trained with public and real data, and it was deployed in an AWS Lambda function. The API was built with FastAPI and it was deployed in a Docker container. The CI/CD pipeline was built with GitHub Actions. The entire workflow was automated with a Makefile and can be use as a template for future projects.
A jupyter notebook (exploration.ipynb) has been provided with the work of a Data Scientist (from now on, the DS). The DS, trained a model to predict the probability of delay for a flight taking off or landing at SCL airport. The model was trained with public and real data, below we provide you with the description of the dataset:
| Column | Description |
|---|---|
Fecha-I |
Scheduled date and time of the flight. |
Vlo-I |
Scheduled flight number. |
Ori-I |
Programmed origin city code. |
Des-I |
Programmed destination city code. |
Emp-I |
Scheduled flight airline code. |
Fecha-O |
Date and time of flight operation. |
Vlo-O |
Flight operation number of the flight. |
Ori-O |
Operation origin city code. |
Des-O |
Operation destination city code. |
Emp-O |
Airline code of the operated flight. |
DIA |
Day of the month of flight operation. |
MES |
Number of the month of operation of the flight. |
AĆO |
Year of flight operation. |
DIANOM |
Day of the week of flight operation. |
TIPOVUELO |
Type of flight, I =International, N =National. |
OPERA |
Name of the airline that operates. |
SIGLAORI |
Name city of origin. |
SIGLADES |
Destination city name. |
In addition, the DS considered relevant the creation of the following columns:
| Column | Description |
|---|---|
high_season |
1 if Date-I is between Dec-15 and Mar-3, or Jul-15 and Jul-31, or Sep-11 and Sep-30, 0 otherwise. |
min_diff |
difference in minutes between Date-O and Date-I |
period_day |
morning (between 5:00 and 11:59), afternoon (between 12:00 and 18:59) and night (between 19:00 and 4:59), based on Date-I. |
delay |
1 if min_diff > 15, 0 if not. |
-
Use the main branch for any official release that we should review. It is highly recommended to use GitFlow development practices. NOTE: do not delete your development branches.
-
All the documentation and explanations that you have to give, must go in the
challenge.mdfile insidedocsfolder.
We need to operationalize the data science work for the airport team. For this, we have decided to enable an API in which they can consult the delay prediction of a flight.
In order to operationalize the model, transcribe the .ipynb file into the model.py file:
- The DS proposed a few models in the end. Choose the best model at your discretion, argue why.
- Apply all the good programming practices that you consider necessary in this item.
- The model should pass the tests by running
make model-test.
Deploy the model in an API with FastAPI using the api.py file.
- The
APIshould pass the tests by runningmake api-test.
Deploy the API in your favorite cloud provider.
- Put the
API's url in theMakefile(line 26). - The
APIshould pass the tests by runningmake stress-test.
Implement a proper CI/CD implementation for this development. For this, you must use Github Actions.
- Complete both
ci.ymlandcd.yml(consider what you did in the previous parts).
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent