Repeated Failure and Restarting of Worker Deployment about ray-llm HOT 5 CLOSED

Would like to add, that I encounter the same behaviour - adding pg_timeout_s to the models does not seem to help, as it seems like the instances themself are timing out (before the models). And being stuck in pending phase

If there is a param i can add to aviary-cluster.yml to increase the timeout, that would be great.

from ray-llm.

Hey @njgerner, thanks for the report! Let me try to reproduce and get back to you. In the mean time, can you see if adding pg_timeout_s: [replace with a very large integer] to scaling_config in model yaml files helps?

from ray-llm.

Also, slimming down the aviary image is something that's on our to-do list.

from ray-llm.

Got it, thanks. Will spend some time on this today.

from ray-llm.

This should be now fixed in 54d0ebb . The docker image was also reduced in size (though it is still rather large). Please reopen the issue if it persists!

from ray-llm.