A repository that contains helper function for using queue based cluster compute systems
- First ssh into the desired cluster (see #3 and #4).
- Clone the latest version of hpc-utils
- Create a new venv and run
pip install -e .
from the hpc-utils directory - You should now be able to run the api and worker using
hpcutils-api
andhpcutils-worker
respectively
- From the app, create a project
- From the terminal, run
source hpc-utils/hpcutils/helpers/{cluster}.sh
- Create the virtual environment associated with your project using
prepare_virtual_env {venv_path} {github_user} {github_repo}
(alternatively if your project does not use asetup.py
you can create your environment manually, but first make sure to runget_gpu_modules
) - Move venv to project location,
cp -r {venv_path} {cluster_dir}/job_metadata/{project_name}/venv
- Now you should be able to run jobs fine and dandy
nohup (to be updated)