cytomine / cytomine-software-router Goto Github PK
View Code? Open in Web Editor NEWCytomine module for communication between Cytomine-Core and remote algorithms
License: Apache License 2.0
Cytomine module for communication between Cytomine-Core and remote algorithms
License: Apache License 2.0
Problem : Jobs was not ran without output in the job logs.
Logs of software_router component : "Batch job submission failed: Unable to contact slurm controller (connect failure)".
How to check : run squeue into the slurm component sudo docker exec -it slurm squeue
you should get the same error message.
When : A long singularity image building on a Kubuntu VM (8 proc, 8Gb RAM)
Fixed by a sh restart.sh
of the Cytomine bootstrap
I met
"log4j:WARN No appenders could be found for logger (be.cytomine.software.consumer.Main).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info."
when I was checking the logs of software_router, can you help fix this? Thanks.
When we are behind a corporate proxy, we cannot natively reach github API.
Solution : java -jar -Dhttp.proxyHost=... -Dhttp.proxyPort= ... (idem with https) Cytomine-software-router.jar
As you are often running in Docker containers, allow an auto set of these parameters if environment variables used by Docker are set.
When we add a software user repository and fetch all the targeted repositories, sometimes I get "Error: unable to build: conveyor failed to get: no descriptor found for reference".
It seems that, as our external softwares are often based on the same image base (cytomine/python-software for example), we got a multithread issue with concurrent fetch.
There is example in the related issue
https://github.com/sylabs/singularity/issues/3634 and https://github.com/sylabs/singularity/issues/5020
Short solution : implement a pullingQueue to assure that there is no concurrency pulling.
Later solution : Should be resolved with singularity 3.6
If Core is not found, it is useless to continue.
+
with docker restart container policy, the exit 1 of the software_router will kill the container then restart it until core is found.
Currently, it is possible to add software by Github and DockerHub repositories and uploading a zip containing a Dockerfile.
Adding a "manual upload" with a Singularity recipe file instead of Dockerfile in a zip could be supported.
https://github.com/cytomine/Cytomine-software-router/blob/master/src/main/groovy/be/cytomine/software/management/SingleSoftwareManager.groovy#L60
Improve backward communication from SR to the Core for error handling (not existing repositories or other potential errors).
Currently, we have a dependency between the name and the repository to fetch.
So, I cannot run a software with name "Statistic_Computation" on a repo S_Stat_Comput
The problem occurs during the launching step because softwareTable is filled with the software name that will be used for later fetching.
We should not use the name to determine a repository url
OR
We should have two different fields : name (user friendly name) and repositoryName
See https://stackoverflow.com/questions/53134212/invalid-privatekey-when-using-jsch
It seems to be mandatory to have -----BEGIN RSA PRIVATE KEY----- header
See how to support OpenSSH keys
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.