pmelchior / hydra Goto Github PK

currently the system pushes jobs onto the nodes, which limits the number of jobs in the queue to those that can immediately be pushed.

Replace with system in which a script is started at each of the nodes, which then pulls from a single central node how ever many jobs the host wants. This script can then also execute the cpu utilization query and write the results back in the DB.

currently, both index.html and cputils.py need to know properties of the host. Replace with a central location (e.g. static json) that lists resource limits, OS, etc, and that also tells how many CPUs are available for new jobs.

When each node checks its own description, they can ask for the number of CPUs they can and should utilize whenever a slot becomes available. They can and should also check if they have the resources (e.g. memory) needed for the running these jobs.