[Feature Request] Add `--state [STATE]` to `pueue wait` about pueue HOT 10 CLOSED

Confirmed that a wait does stop this going wrong (it just means I don't get to see the output whenever the race hits).

from pueue.

Thanks for reaching out!

I thought pueue follow would wait until a task started and follow its output until it completed. Instead, I'd guess that what's happening is that we're racing on the daemon: the task hasn't quite started yet and pueue follow fails, and then the pueue status jq invocation fails because the task status isn't Done (I could make it more robust but jq's syntax is so horrific and ill-documented that I can't face it). Stranger yet, sometimes it works fine, but pueue status then complains that it can't find the logfile which pueue follow was just happily following for days.

I think you're right about follow failing because the task hasn't started yet.

The daemon is running asynchronously, as it needs to manage tasks and check if there're any free slots. Tasks are enqueued every ~200ms, which is why this sometimes works and sometimes fails.

However, there's an easy fix for this. Pueue has never been designed to be scrictable, it just received a few convenience commands and helpers which allow it to be (somewhat) scriptable.
One of those helpers is the pueue wait command, which waits for either all tasks, a group, or a set of tasks to finish.

In your case, you could replace the pueue follow with pueue wait and you should be good to go.

from pueue.

Yeah, that'll fix it I hope (testing initiated, may be a day or so before I know one way or the other). The ideal would be a bit different though.

I'm also trying to display the output of the command as I go (since it's handbrake: it takes hours and does display useful progress output). Something like pueue wait --follow or (more generally) pueue wait --until STATE (which waits until the task switches state to STATE) or just pueue wait --until-changed (which returns whenever the task changes state at all) would be ideal: I could follow that with a pueue follow in the certainty that it would actually work. pueue wait --until-follow-will-work-dammit is what I'm after, but that sounds a bit... specific.

(I'm surprised how useful it is in scripts actually -- mostly for squashing serial things running in high-parallelism queues down into parallel things in low-parallelism queues :) )

from pueue.

I think pueue wait --state $STATE seems like the best solution for this.
That's basically the same thing the integration test suite does.

from pueue.

Oh if the test suite is already doing something just like this, so much the better! Proof it's useful :)

from pueue.

Hey @nickalcock

I implemented the feature in the linked pull request.
Could you check out, if this works for you :)?

git clone [email protected]:Nukesor/pueue
cd pueue
git switch wait-for-status
cargo install --path ./pueue

from pueue.

Finally got around to testing this, a bit late since it hit the master branch long ago. Seems to work, and lets me replace a lot of not-really-working kludgery with something much simpler ("pueue wait --status running").

It is a little odd that the status in the pueue output reads 'Running' but that you have to wait for 'running': waiting for 'Running' fails...

from pueue.

I'm confused. What do you mean with 'waiting for Running fails' in your last sentence?

When I'm running this I get this output:

[nuke:~] 130! pueue wait --status running  1
18:21:44 - Task 1 changed from Stashed to Queued
18:21:52 - Task 1 changed from Queued to Running

from pueue.

Oh yes, that works fine -- what's odd is that you can't say pueue wait --status Running, even though the status is given with a capitalized first letter in the status output. (Probably both should be accepted?)

from pueue.

Ahh.

Fixed :). Pueue will allow capitalized stati with the next release.

from pueue.