quantum-accelerators / covalent-hpc-plugin Goto Github PK

The issue is upstream in PSI/J.

Reference:

• ExaWorks/psij-python#423

Currently, when an error is raised, only the error message itself is reported rather than any details regarding the traceback. This makes it very difficult to debug, unlike when an error arises locally where a full traceback is present.

A clearer traceback report should be provided in the UI.

This is a mirror of AgnostiqHQ/covalent#1782

Tracking AgnostiqHQ/covalent-slurm-plugin#64. There was an initial attempted implementation in AgnostiqHQ/covalent-slurm-plugin#71.

i.e. run the server on the HPC machine and submit there without SSH. Linking this to AgnostiqHQ/covalent-slurm-plugin#55.

What new feature would you like to see?

From Sankalp at Covalent:

For the first point, the DaskExecutor supports task packing currently so if you wanna give your custom plugin a shot that would be a good reference 🙂 .
Steps you can use to check whether task packing is working as expected:
Temporarily add app_log.warning(f"Submitting {len(task_specs)} tasks to dask") to this line locally.
Run the workflow example you showed
Check covalent logs and you'll see that 1 task gets submitted to dask 2 times
Now, enable task packing with ct.set_config("sdk.task_packing", "true") in your workflow, and restart your notebook and the server
Then when you'll check the covalent logs, you'll see that 2 tasks were submitted to dask in 1 go -> i.e. the dictionary that was supposed to be created as a separate task, now gets packed with the task that uses it directly
That is essentially how it's going to work in practice and we're going to enable task packing by default once we have enough executors supporting it. It is still something that we haven't thoroughly tested but should help as a reference for the executors that do want to support it. (The send, poll and receive methods of the dask executor should help in understanding, and the class attribute of SUPPORTS_MANAGED_EXECUTION)

Have the plugin write them out to a pre_launch/post_launch Bash file and then clean it up. A good example is loading of modules.