saagie / api-saagie Goto Github PK
View Code? Open in Web Editor NEWPython library to interact with Saagie API
License: Apache License 2.0
Python library to interact with Saagie API
License: Apache License 2.0
Describe the bug
There is no consistency on the instance information we can get as we go by the job or by the instance.
When I called job.get_instance, it didn't return the start date and end date of the instance.
When I called job.get_info, I had the dates but not the version information.
Expected behavior
I want the same level of information on the instance as I use one or another method.
Timezone can now be configured in Job & Pipeline scheduling
Update saagie api accordingly
Describe the bug
A clear and concise description of what the bug is.
Expected behavior
A clear and concise description of what you expected to happen.
Additional context
Add any other context about the problem here.
This is a test
To ensure maintainability of the current codebase, the following refactoring should be done
saagieapi
saagie_api
pipelines
pipelines.py
gql.py
jobs
apps
projects
env_vars
docker_credentials
This would mean creating new classes for each main functionnality (pipelines, jobs...) that inherits the SaagieApi main class.
Let's take an example with the create_job function that would be now in the jobs.py file, and named create so that calling this function would mean calling saagieapi.jobs.create. The original create_job in saagieapi would be tagged as deprecated and will call the new function.
As in the doc of gql : https://gql.readthedocs.io/en/stable/usage/variables.html
We can use variables with $ instead of doing string.format()
See gql_create_graph_pipeline in gql_template for example
Currently editing a project is not possible
Currently, when creating a job, only the technology name is specified. If this technology name is used in different catalogs, the saagie-api will get the first one available.
It would be nice to be able to specify the catalog (if no catalog specified, take the official) to deal with that.
Change the graphql schema for the test
When creating a project, only one group can be assigned to it. It should be possible to specify a list with multiple groups and their corresponding roles on the project.
When using get_project_web_apps to get the list of applications, if the technology no longer exists, the function give an error instead of the list.
Is your feature request related to a problem? Please describe.
In project's jobs list, get the current version
When I tried to list the app of a project, I got the following error message :
TransportQueryError: {'message': 'io.saagie.projectsandjobs.domain.exception.NonExistingTechnologyException: Technology <technology_id> does not exist', 'locations': [], 'extensions': {'classification': 'DataFetchingException'}}
In Saagie UI, I saw that one of the app has a deleted technology. In the developper console, I saw there is 2 results for the graphql request: an error and some data.
It will be great if the package can do the same.
The correction can be performed in utils>gqlclient.py>execute() link this:
except TransportQueryError as transport_error: #if data not null #return data and print error or warning to show the error message print(transport_error.data)
tester et documenter le endpoint "/api/open/refresh" pour prolonger la vie d'un token
get the project id by the project name
For jobs, pipelines and apps, add a create_or_upgrade method that will check if a given job, app or pipeline already exists inside a project (based on its name) and decide if it needs to be created or updated
Adding deprecation warnings for deprecated warnings such as linear pipelines
Upgrade to gql 3.0 that now supports file uploads (instead of using requests library)
apps.get_info does not return infoirmation about current instance id
This method should return a list of instances for a given app (with a max instance parameter, as it's done for the jobs and pipelines)
Describe the bug
There is no option to configure the print in the different methods
Expected behavior
A parameter in all methods which prints results to configure it, and a global parameter to turn everything on or off
If a try to add a job with a technology not allowed inside my project/technology, the create_job fiunction does not return an error, and the API returns
{'errors': [{'message': 'Job not valid', 'extensions': {'technology': 'not exists', 'classification': 'ValidationError'}}], 'data': None}
Update jobs doesn't work from version 0.8.4 and up
Project name used for integration tests is always the same and if an integration test is launched while another one is already running, it will fail.
Jobs can now be configured with CPU & memory requests/limits
Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is.
Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.
Currently, it's only possible to edit the following parameters of a job :
job_name
description
is_scheduled
cron_scheduling
schedule_timezone
resources
It should also be possible to upgrade a job (creating a now job version) :
runtime_version,
command_line
release_note
file
The upgrade should be able by passing either :
-the name of the job and its project (name is unique inside a project)
-the id of the job and its project
Add retry number as parameter of RequestsHTTPTransport
Improve logging:
Add get, create, update, delete operations for dockerCredentials
Add the features for creating an app :
Add a new parameter for the function create_project in order to have technology of other catalogs
Handle alerting for jobs, pipelines and apps
When the project creation is taking too much tiome on Saagie, the integration tests are failing without explicit messages.
To do :
Add requests_toolbelt librairy as dependencies
requests_toolbelt is needed when we import the package saagieapi, it will be better if we include this package in the dependencies
Is your feature request related to a problem? Please describe.
Our platform got a breaking change in its API. I wanted to know which version of saagieapi I should use, but the release description does not specify which version of Saagie is compatible with the releases of saagieapi.
Describe the solution you'd like
In each release note, it should be specified which version(s) of Saagie is supported.
Additionnaly, saagieapi package should check directly with the Saagie platform if it is compatible.
For example, When creating an object from SaagieApi class, the init should query /version (or something similar) to check if the version of the API is handled by the package. If not, raise Exception with specific message.
Function to return pipeline ID with pipeline name argument.
get_pipeline_id(self, project_id, pipeline_name) :
"""
"""
return id
Is your feature request related to a problem? Please describe.
No
Describe the solution you'd like
Add field instances in the query GQL_GET_PIPELINE
To ensure consistency on code convention, code linter (isort and black) should be run before every commit made on this repository.
Apps available within a project can now be configured
Is your feature request related to a problem? Please describe.
Possibility to import/export a project to a new platform
Is your feature request related to a problem? Please describe.
Currently, there is no feature to change an environment variable
Describe the solution you'd like
Add two methods to up either global or porject environment variable
Import projects, jobs, pipelines, envs and apps from json files
It should be possible to have a folder containing json with jobs and pipelines definitions and a function that gets this definitions and create the corresponding jobs and pipelines (or update them if they exist)
Severals functions use instances_limit as parameter in order to limit the number of instance in results. This parameter could be optional : if specified, you request the n instances; if not, you request all the instances
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.