Data parser from GitHub
-
Install Docker and Docker Compose if they are not already installed on your system.
-
Clone the project repository:
git clone https://github.com/Djama1GIT/github-parser.git
cd github-parser
-
Configure the environment variables in the .env-non-dev file
-
Start the project:
docker-compose up --build
After starting the project, you can access the Swagger user interface at: http://localhost:7354/docs.
In Swagger, you can view the available endpoints and their parameters, and also make requests to the API.
Note: Install dependencies using pip install -r requirements.txt
.
pytest .
It is assumed that you already have a postgresql server running. If this is not the case, you can uncomment the lines in docker-compose.yml and launch the container on Yandex Cloud.
The function runs every half hour. I deemed it necessary to do so because it's not a very short or very long period, as there won't be too many changes during this time, and the number of API requests will be moderate.
/bin/bash ./src/github_parser/create_cloud_function.sh
Example: