High-performance Async REST API, in Python. FastAPI + GINO + Uvicorn (powered by PostgreSQL).
- Clone this Repository.
git clone https://github.com/wri/gfw-data-api.git
- Run
./scripts/setup
from the root directory. (Runpip install pipenv
first, if necessary.) - Run locally using docker-compose.
./scripts/develop
- Generate a DB Migration:
./scripts/migrate
(noteapp/settings/prestart.sh
will run migrations automatically when running/scripts/develop
) - Run tests:
./scripts/test
- Run specific tests:
./scripts/test tasks/test_vector_source_assets.py::test_vector_source_asset
- Debug memory usage of Batch jobs with memory_profiler:
- Install memory_profiler in the job's Dockerfile
- Modify the job's script to run with memory_profiler. Ex:
pixetl "${ARG_ARRAY[@]}"
->mprof run -M -C -T 1 --python /usr/local/app/gfw_pixetl/pixetl.py "${ARG_ARRAY[@]}"
- scp memory_profiler's .dat files off of the Batch instance (found in /tmp by default) while the instance is still up
- FastAPI: touts performance on-par with NodeJS & Go + automatic Swagger + ReDoc generation.
- GINO: built on SQLAlchemy core. Lightweight, simple, asynchronous ORM for PostgreSQL.
- Uvicorn: Lightning-fast, asynchronous ASGI server.
- Optimized Dockerfile: Optimized Dockerfile for ASGI applications, from https://github.com/tiangolo/uvicorn-gunicorn-docker.
- Pydantic: Core to FastAPI. Define how data should be in pure, canonical python; validate it with pydantic.
- Alembic: Handles database migrations. Compatible with GINO.
- SQLAlchemy_Utils: Provides essential handles & datatypes. Compatible with GINO.
- PostgreSQL: Robust, fully-featured, scalable, open-source.