Terminating app on Windows behaves differently than Linux about capsule HOT 37 CLOSED

As to the lesser issue, Windows shell issues, what Capsule version are you using?

As to the main issue, in both cases Capsule calls Process.destroy. On Windows, that is implemented by TerminateProcess(handle, 1). If you could investigate further and perhaps suggest a possible cause, I'll be grateful.

from capsule.

0.10.0 via the Maven plugin.

Is there Capsule logging that would be helpful? I can try building against your master. I guess I would have to build the plugin too?

from capsule.

-Dcapsule.log=debug should output the most detailed log. 0.10.0 is good enough for this. That code hasn't changed in 1.0.

from capsule.

Hmm. The logging did not generate any output.

Looks like you are not calling process.waitFor() after you call process.destroy(). I wonder if that would change things.

from capsule.

I've added waitFor. It will be released in 1.0-rc1, and we'll see if it helps. The problem is that on Unix/Linux, Capsule can at least try some signal tricks. On Windows, all we have is whatever the JRE does on Windows...

I know someone has built a Caplet that even uses JNR to call execve for the child process, so that the capsule process becomes the app process rather than spawn it as a child, but again, Windows... (Windows does actually have execve but I don't think it behaves the same way).

from capsule.

Can you test it now with version 1.0-rc1?

from capsule.

Have you tested with 1.0-rc1?
About to close for inactivity (you can alway reopen later).

from capsule.

So sorry. I'll get this tested today and report back my findings.

from capsule.

Just tested with 1.0-rc1. Sorry for the delay. Sadly, no different. On a couple terminations I started to see my app's termination console logging. Most times I did not. And through Cygwin, the second process remains orphaned and is not properly terminated.

from capsule.

Thank you. I don't have enough Windows knowledge to pinpoint the issue (I doubt Cygwin has its own process management -- it probably just uses the Windows API). You can help by finding similar issues, and hopefully a workaround. This could well be a JDK Windows bug.

from capsule.

This is one of those dark, murky corners of the JVM where cross-platform falls on it's face. There appear to be some choices using JNI, but that will bloat Capsule and introduce architecture-specific native libs.

Just to be sure this wasn't just an issue of swallowing stdout/stderr on child shutdown I added a file touch call at the end of shutdown process of my app. Linux touches the temp file correctly. Windows does nothing at all which leads me to conclude that the shutdown process is being initiated but is never completing because the JVM is aggressively killed. This would explain why I sometimes see the first couple logging lines of my shutdown process; it's a timing race-condition.

Some further searching into the Windows sources for the JVM show that win32 TerminateProcess is being called through the destroy() method. That's essentially the POSIX equivalent of SIGKILL which means that you can't cleanly shutdown a child JVM on Windows. :(

I did try playing the the trampoline variant and generating a .cmd file from the definition, but I think it was too long for the Windows environment. I ended up simplifying the generated definition by using wildcard inclusion of the cache artifacts instead of naming them individually and switching from javaw.exe to java.exe. That did work, as expected, and responded properly to ctrl-c.

I would suggest adding a caveat about the lack of clean shutdowns on Windows. It may also be worthwhile to improve the trampoline design for those needing to deploy on Windows.

from capsule.

There appears to be an open JDK issue addressing this.

There is actually a mechanism in Capsule to handle long command lines on Windows, which creates a temporary JAR with the classpath listed in its manifest, but this mechanism is disabled for trampolines, because there's no one to clean up and delete the temporary JAR file.

Suggestions on how to handle this would be welcome, but I'd just like to mention that wildcards are probably not a good idea, as the list of classpath dependencies is quickly made explicit in Capsule so that it could be examined and manipulated by caplets.

from capsule.

I just had an idea how to solve this. It's a bit of work, but not too much (under 100LOC).

The idea is this: on Windows, Capsule will inject a Java agent into the app that starts a thread (running in the app) that will communicate with the Capsule process via a socket. When the Capsule process terminates, it sends a message on that socket, and the injected thread calls System.exit().

It will work (in fact, the native solutions suggested work similarly), and I think it can even be done without adding classes (remember, Capsule must compile to a single class file). For example, Capsule itself will implement the agent, and the thread will use the same trick used now to handle another Windows JDK bug with IO piping threads.

I could get around to it in a few weeks, but if you want to get started, you can start working on a PR.

from capsule.

OK, I've added the agent solution. Can you try it now? I have tested the agent being injected and connecting to the capsule process, but not the actual shutdown itself.

You must add Premain-Class: Capsule to your manifest. Also, please turn on verbose logging on so we can see what's going on.

from capsule.

In any event, I'm pretty confident now that this solution will work once we've worked out the kinks.

from capsule.

Do I have to build Capsule from source?

from capsule.

Preferably, yes, or use 1.0-rc2-SNAPSHOT from the Sonatype snapshot Maven repository (https://oss.sonatype.org/content/repositories/snapshots).

from capsule.

Not sure how to proceed. I use the Maven plugin which AFAICS doesn't allow me to control the Capsule version.

from capsule.

@chrischristo Can you help @gitblit by telling him how to use a snapshot version of Capsule with the Maven plugin?

from capsule.

@pron @gitblit just add capsule.version as maven property, see here.

from capsule.

@gitblit Have you tried it?

from capsule.

If tried this one Windows with Oracle Java 1.7.0_75 (within a cygwin shell, but that should not matter) and can confirm (e.g. in Process explorer or jvisualvm) that Ctrl-C will terminate the capsule-VM, but NOT the started process.
Doesn't Runtime.addShutdownHook() work as the Javadoc implies?
Using latest 1.0rc-1 with latest maven plugin.

from capsule.

Doesn't Runtime.addShutdownHook() work as the Javadoc implies?

Process.destroy() on a JVM process on Windows apparently shuts the JVM forcefully without running shutdown hooks, and that is the mechanism the capsule process terminates the app process (which is why Windows requires a different mechanism).

from capsule.

Why... close? - Only the capsule VM is terminated. The started VM stays around forever... ?!
I just confirmed that the bug is still valid :-( please re-open.

from capsule.

Oh, sorry. I misunderstood. :/

from capsule.

I did play around a bit with some app simply writing to a file on Windows 7 in cmd and cygwin and aborting it while in action. And in both cases Ctrl-C terminated both capsule and the application processes. I tried that with Oracle Java 1.7.0_45 and 1.8.0_25. Even placed commons-io in it to have at least one jar and created a non-daemon Thread in the sample app assuming that this might make a difference. But no... terminates everything successfully.
Well good in a way, because this means it's not generally an issue on Windows. Same capsule 1.0.-rc1, same capsule-maven-plugin 0.10.6.

But I currently have no clue why it doesn't work with the other app (Spring-based, lot's of jars) with Oracle 1.7.0_75 on another Windows 7 in cygwin (not yet tried cmd there...).
But the started app shouldn't have an impact for the actual shutdown code, so I'm confused that I cannot reproduce it.
Is there any way to enable some more diagnosis code in capsule (e.g. to see if the shutdown-hook is triggered at all or something). I'd probably could dig a bit further if time. At least I'll try the demo app on the other environment I have.

from capsule.

@tngmech Does -Dcapsule.log=verbose tell something interesting on the system where it doesn't work? You should see a log when the socket server starts, accepts connections and receives a connection. The client too will log when connected and also "exit" message exchanges should be logged as well as exceptional conditions.

from capsule.

java -jar target/capsuleTest-0.0.1-SNAPSHOT-capsule-fat.jar -Dcapsule.log=verbose does not log anything when run.
I cannot test the other system before next week.

from capsule.

JVM options like -D must precede -jar.

from capsule.

:-) my mistake... Yes, that logs a bit more, but primarily JVM version, path, classpath, etc. - After that the App logging starts, but nothing when I hit Ctrl-C, it just stops the demo app as intended. Also I don't see something like "connections".

from capsule.

How about the succesful attempts on other systems, do you see logging about the new closing mechanism? Also, even on the problematic system cmd and cygwin behave in the same way?

from capsule.

If successful, I do not see any CAPSULE: logs when I hit Ctrl-C. Neither in cmd nor cygwin. Haven't tried that dummy app on other systems.
Will have to enable logging for the problematic system/app next week to check if cmd/cygwin makes a difference there. Will also check that on a Linux box to see if it somehow relates to the app or the OS.

from capsule.

Ok thanks, although this specific socket-based mechanism is enabled only for Windows (on Unixes the child process destruction method is graceful, see https://bugs.openjdk.java.net/browse/JDK-8056139).

from capsule.

Also, this fix requires master's tip capsule, which has version 1.0-rc2-SNAPSHOT (you were mentioning rc1). The logging about the communication being setup for the clean closing should happen even before hitting CTRL+C.

from capsule.

Have you tried with the newest version?

from capsule.

@tngmech Have you been able to use the latest version and get more insight about the Windows box where this doesn't seem to work?

from capsule.

Our tests show that this issue has been resolved. Please reopen if you continue to encounter it.

from capsule.