Execution cannot be interrupted with Ctrl-C about pbmcapply HOT 8 CLOSED

Thanks for reporting this issue, Martin! Because pbmcapply hands over the actual parallel computation to a new process (via future), by killing the progress monitor (your main process), the newly created process is not affected.

I'll look into handling SIGINT signals in the main process and relay it to the parallel process. I'll post updates once I make some progress.

from pbmcapply.

Hi @kvnkuang ,

I'd like to first thank you for pbmcapply package, it's really helpful! 👍
This issue is actually the major problem I'm having when using pbmcapply and I'm really keen to see it fixed, so I've given it a try. I noticed future creates a child process and this child process creates its own descendants when calling mclapply as well. To kill all these processes when interrupted, I found this blog post provides a solution to this. Basically, we can assign a group process id to the processMonitor and kill it and its descendants based on the group process id on interruption. Here is what I did:

1. I noticed the futureCall given plan(multiprocess) calls mcparallel under the hook so I replaced it with mcparallel.
2. Another reason I did this is that in order to assign the group process id I need to copy the mcparallel source code and insert the assignment code in between.
3. I added the on.exit call to clean up processes based on the group process id.

This works on both on my macbook and ubuntu desktop so far from my tests. If you are interested in this solution, I would be more than happy to work a bit more to tidy up my code in terms of error handling and create a pull request. Keen to contribute to this 😄

Cheers,
Quyu

from pbmcapply.

Hi Quyu,

Thank you so much for your research and fix on this issue. It looks pretty promising and I've been thinking about replacing future with native code as we're only using a very small portion of its functions.

It would be much appreciated if you could initiate a pull request. Once tested, I will accept it and push an update to CRAN as soon as possible.

Regards,
Kevin

from pbmcapply.

Hi Quyu,

I've merged your pull request (and a series of subsequent pushes I made) into the dev branch. Could you please help me test it around to make sure everything works?

Also, I updated the contribution sections to include your name (also in the dev branch currently). Thanks again for your contribution!

Regards,
Kevin

from pbmcapply.

Hi Kevin,

Thanks! I just tested your dev branch on both Mac and Ubuntu regarding pbmclapply and pbmcmapply running completely and running with interruption, all worked as expected. I also tested compilation on Windows (didn't realize my code didn't compile on windows but you fixed that 😄 ).

While testing, I noticed another trivial bug. As i'm used to pbmclapply, I sometimes forgot the difference in argument order between pbmclapply and pbmcmapply where FUN is the first argument. This might cause the program hanging forever at ReadBin. So I guess we should handle that at the beginning of the progressMonitor function block with FUN <- match.fun(FUN). But since this is a separate problem, we can handle it in another issue.

Regards,
Quyu

from pbmcapply.

Thanks again for your feedback! I've included the code you suggested to improve function handling.

Regards,
Kevin

from pbmcapply.

Hi!

I just encountered this same error (Ctrl-C not stopping pbcmlapply) under Linux using rscript.

Perhaps the fix by @qykong does not work anymore since the addition of ignore.interactive? Since I am using it from the terminal I need to switch ignore.interactive to T in order to see the progress bar, but then Ctrl-C does not work (at least in version 1.5.0)

Thank you!

from pbmcapply.

I have the same problem. I cannot interrupt the pbcmlapply under linux. I must close R to stop it

from pbmcapply.