Comments (16)
dmlloyd
commented on June 12, 2024
1
Ah, great; can you use jstack to capture the stack of the "hung" process in JVM mode?
from quarkus.
Sanne
commented on June 12, 2024
Probably trivial for @dmlloyd
from quarkus.
dmlloyd
commented on June 12, 2024
It may be a bit trickier to do as we would have to install the signal handler in static init. I don't think we should be making connections in static init blocks though: having clean startup/shutdown depends on things starting and stopping in order in the "main" method.
from quarkus.
Sanne
commented on June 12, 2024
I don't think we should be making connections in static init blocks though
What kind of connections are you referring to?
from quarkus.
dmlloyd
commented on June 12, 2024
Presumably your failure is occurring because there are exceptions being thrown before the application starts. The signal handler is installed at the top of the main method, so if it is the case that the problem arises due to an exception occurring before the signal handler is installed, then it must be connecting from a static init block.
However, I don't think this is the case. More likely there's some framework that doesn't shut down correctly when a database connection occurs. Each extension should be registering a closeable that can terminate that extension in the event of a problem; as long as this is done correctly, then everything should shut down cleanly in any case.
from quarkus.
Sanne
commented on June 12, 2024
Sounds reasonable to improve on this, in this case I suspect it's the connection pool.
Still, don't you think the users will expect a SIGQUIT to terminate the application no matter what went wrong?
from quarkus.
dmlloyd
commented on June 12, 2024
We don't handle SIGQUIT no matter what. We could add that though - maybe we could introduce some diagnostic capabilities similarly to how HotSpot prints stack traces.
from quarkus.
dmlloyd
commented on June 12, 2024
But for SIGINT/SIGTERM, yes but we also expect a clean shutdown. There is no workable semantic where we heuristically shut things down and later give up and kill the process, as there is no way to differentiate between a slow "clean" shutdown and a "stuck" shutdown.
Still, in your case I'd be interested to see the stack trace of the failure, just in case it's a GraalVM bug or something. Also, how does JVM mode compare to native in this case?
from quarkus.
Sanne
commented on June 12, 2024
Agreed on the semantics puzzle. Not asking to have both simultaneously, perhaps this specific case should just "crash fast".
Both JVM mode and native mode hang in the same way. There's no stack trace, it just hangs and ignores requests to terminate, so I need to open a new terminal and figure out the pid...
Nothing too horrible, it's just a different experience than what people is used to.
from quarkus.
dmlloyd
commented on June 12, 2024
(it might be a bug after all!)
from quarkus.
dmlloyd
commented on June 12, 2024
TBH I don't think the startup-failed path is even tested.
from quarkus.
Sanne
commented on June 12, 2024
2018-12-05 17:36:01
Full thread dump GraalVM 1.0.0-rc9 (25.192-b12-jvmci-0.49 mixed mode):
"Attach Listener" #27 daemon prio=9 os_prio=0 tid=0x00007fb164003000 nid=0x5f66 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"SIGINT handler" #26 daemon prio=9 os_prio=0 tid=0x00007fb164002000 nid=0x5d80 waiting for monitor entry [0x00007fb1faffc000]
java.lang.Thread.State: BLOCKED (on object monitor)
at java.lang.Shutdown.exit(Shutdown.java:212)
- waiting to lock <0x00000003d6cf9e08> (a java.lang.Class for java.lang.Shutdown)
at java.lang.Terminator$1.handle(Terminator.java:52)
at sun.misc.Signal$1.run(Signal.java:212)
at java.lang.Thread.run(Thread.java:748)
"SIGINT handler" #25 daemon prio=9 os_prio=0 tid=0x00007fb164001000 nid=0x5d7f waiting for monitor entry [0x00007fb1fb1fe000]
java.lang.Thread.State: BLOCKED (on object monitor)
at java.lang.Shutdown.exit(Shutdown.java:212)
- waiting to lock <0x00000003d6cf9e08> (a java.lang.Class for java.lang.Shutdown)
at java.lang.Terminator$1.handle(Terminator.java:52)
at sun.misc.Signal$1.run(Signal.java:212)
at java.lang.Thread.run(Thread.java:748)
"Shutdown thread" #22 prio=5 os_prio=0 tid=0x00007fb238d2b800 nid=0x5d7d waiting on condition [0x00007fb1fb0fd000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000003d685be08> (a java.lang.Thread)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at org.jboss.shamrock.runtime.graal.ShutdownHookThread.run(ShutdownHookThread.java:39)
"DestroyJavaVM" #24 prio=5 os_prio=0 tid=0x00007fb23800b000 nid=0x5d4a in Object.wait() [0x00007fb23f0db000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000003d7053aa0> (a org.jboss.shamrock.runtime.graal.ShutdownHookThread)
at java.lang.Thread.join(Thread.java:1252)
- locked <0x00000003d7053aa0> (a org.jboss.shamrock.runtime.graal.ShutdownHookThread)
at java.lang.Thread.join(Thread.java:1326)
at java.lang.ApplicationShutdownHooks.runHooks(ApplicationShutdownHooks.java:107)
at java.lang.ApplicationShutdownHooks$1.run(ApplicationShutdownHooks.java:46)
at java.lang.Shutdown.runHooks(Shutdown.java:123)
at java.lang.Shutdown.sequence(Shutdown.java:167)
at java.lang.Shutdown.shutdown(Shutdown.java:234)
- locked <0x00000003d6cf9e08> (a java.lang.Class for java.lang.Shutdown)
"HotSpotGraalManagement Bean Registration" #18 daemon prio=1 os_prio=0 tid=0x00007fb15c152000 nid=0x5d78 waiting on condition [0x00007fb201098000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at org.graalvm.compiler.hotspot.management.HotSpotGraalManagement$RegistrationThread.run(HotSpotGraalManagement.java:140)
"Service Thread" #17 daemon prio=9 os_prio=0 tid=0x00007fb2383c6000 nid=0x5d76 runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C1 CompilerThread11" #16 daemon prio=9 os_prio=0 tid=0x00007fb2383c3000 nid=0x5d75 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C1 CompilerThread10" #15 daemon prio=9 os_prio=0 tid=0x00007fb2383c0800 nid=0x5d74 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C1 CompilerThread9" #14 daemon prio=9 os_prio=0 tid=0x00007fb2383bf000 nid=0x5d73 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"C1 CompilerThread8" #13 daemon prio=9 os_prio=0 tid=0x00007fb2383bd000 nid=0x5d72 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"JVMCI CompilerThread7" #12 daemon prio=9 os_prio=0 tid=0x00007fb2383bb000 nid=0x5d70 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"JVMCI CompilerThread6" #11 daemon prio=9 os_prio=0 tid=0x00007fb2383b9000 nid=0x5d6d waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"JVMCI CompilerThread5" #10 daemon prio=9 os_prio=0 tid=0x00007fb2383b7000 nid=0x5d6c waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"JVMCI CompilerThread4" #9 daemon prio=9 os_prio=0 tid=0x00007fb2383ac800 nid=0x5d69 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"JVMCI CompilerThread3" #8 daemon prio=9 os_prio=0 tid=0x00007fb2383aa800 nid=0x5d67 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"JVMCI CompilerThread2" #7 daemon prio=9 os_prio=0 tid=0x00007fb2383a8800 nid=0x5d66 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"JVMCI CompilerThread1" #6 daemon prio=9 os_prio=0 tid=0x00007fb2383a6800 nid=0x5d65 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"JVMCI CompilerThread0" #5 daemon prio=9 os_prio=0 tid=0x00007fb2383a3800 nid=0x5d64 waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"Signal Dispatcher" #4 daemon prio=9 os_prio=0 tid=0x00007fb2383a2000 nid=0x5d63 runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"Finalizer" #3 daemon prio=8 os_prio=0 tid=0x00007fb23836f000 nid=0x5d61 in Object.wait() [0x00007fb2025e4000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000003d684b688> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144)
- locked <0x00000003d684b688> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:216)
"Reference Handler" #2 daemon prio=10 os_prio=0 tid=0x00007fb23836a800 nid=0x5d60 in Object.wait() [0x00007fb2026e5000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000003d68534c8> (a java.lang.ref.Reference$Lock)
at java.lang.Object.wait(Object.java:502)
at java.lang.ref.Reference.tryHandlePending(Reference.java:191)
- locked <0x00000003d68534c8> (a java.lang.ref.Reference$Lock)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153)
"VM Thread" os_prio=0 tid=0x00007fb238363000 nid=0x5d5f runnable
"GC task thread#0 (ParallelGC)" os_prio=0 tid=0x00007fb238021000 nid=0x5d4b runnable
"GC task thread#1 (ParallelGC)" os_prio=0 tid=0x00007fb238023000 nid=0x5d4c runnable
"GC task thread#2 (ParallelGC)" os_prio=0 tid=0x00007fb238024800 nid=0x5d4d runnable
"GC task thread#3 (ParallelGC)" os_prio=0 tid=0x00007fb238026800 nid=0x5d4e runnable
"GC task thread#4 (ParallelGC)" os_prio=0 tid=0x00007fb238028000 nid=0x5d4f runnable
"GC task thread#5 (ParallelGC)" os_prio=0 tid=0x00007fb23802a000 nid=0x5d50 runnable
"GC task thread#6 (ParallelGC)" os_prio=0 tid=0x00007fb23802b800 nid=0x5d51 runnable
"GC task thread#7 (ParallelGC)" os_prio=0 tid=0x00007fb23802d800 nid=0x5d52 runnable
"GC task thread#8 (ParallelGC)" os_prio=0 tid=0x00007fb23802f000 nid=0x5d53 runnable
"GC task thread#9 (ParallelGC)" os_prio=0 tid=0x00007fb238031000 nid=0x5d54 runnable
"GC task thread#10 (ParallelGC)" os_prio=0 tid=0x00007fb238032800 nid=0x5d55 runnable
"GC task thread#11 (ParallelGC)" os_prio=0 tid=0x00007fb238034000 nid=0x5d56 runnable
"GC task thread#12 (ParallelGC)" os_prio=0 tid=0x00007fb238036000 nid=0x5d57 runnable
"GC task thread#13 (ParallelGC)" os_prio=0 tid=0x00007fb238037800 nid=0x5d58 runnable
"GC task thread#14 (ParallelGC)" os_prio=0 tid=0x00007fb238039800 nid=0x5d59 runnable
"GC task thread#15 (ParallelGC)" os_prio=0 tid=0x00007fb23803b000 nid=0x5d5a runnable
"GC task thread#16 (ParallelGC)" os_prio=0 tid=0x00007fb23803d000 nid=0x5d5b runnable
"GC task thread#17 (ParallelGC)" os_prio=0 tid=0x00007fb23803e800 nid=0x5d5c runnable
"GC task thread#18 (ParallelGC)" os_prio=0 tid=0x00007fb238040800 nid=0x5d5d runnable
"GC task thread#19 (ParallelGC)" os_prio=0 tid=0x00007fb238042000 nid=0x5d5e runnable
"VM Periodic Task Thread" os_prio=0 tid=0x00007fb2383cb000 nid=0x5d77 waiting on condition
JNI global references: 693
from quarkus.
Sanne
commented on June 12, 2024
@dmlloyd BTW running that demo is trivial. To reproduce you don't even have to start pgsql ;)
just build it from mvn.
Then:
java -jar target/rest-http-crud-shamrock-1.0.0.Alpha1-SNAPSHOT-runner.jar
from quarkus.
Sanne
commented on June 12, 2024
To reproduce you don't even have to start pgsql
To be clear, you really have to NOT start pgsql - otherwise everything works fine ;)
from quarkus.
dmlloyd
commented on June 12, 2024
Yeah it looks like a bug, but not in the signal handler. I'll have a fix shortly.
from quarkus.
dmlloyd
commented on June 12, 2024
Well, I couldn't get your demo to start, but I do have a fix (I think - well, at least it doesn't break anything that isn't already broken). Here comes the PR.
from quarkus.
Related Issues (20)
- Kind of wish LookupIfProperty and LookupUnlessProperty were a little more powerful HOT 6
- quarkus-hibernate-orm - lazy loading of parameterized attributes from MappedSuperclass with getter does not work. HOT 5
- 3.10.1: ConfigMapping w/ Map: does not map to any root HOT 8
- Reentrant locks in CDI contexts can leak Thread instances into image heap HOT 6
- Build Error after bump jandex-maven-plugin version from 3.1.8 to 3.2.0 HOT 2
- [PROPOSAL] Add Teradata database support HOT 2
- Is Quarkus supposed to be supported by AWS CDK ? HOT 7
- Micrometer-Microprofile adapter checks annotations in abstract classes HOT 1
- Promoting a Quarkus app with resource limits in the deployment-config leads to request queuing and thread blocks, impairing functionality. Without limits, the app performs well. HOT 1
- Owerride configuration properties from smallrye.config.locations HOT 1
- Podman guide for Linux is incorrect HOT 1
- JUnit TestTemplate tests are not re-run on code change in dev mode
- Increased augmentation times in Quarkus 3.8.4 HOT 19
- Enable the JDBC instrumentation without additional property
- During IT: Redis host not configured HOT 4
- Behavior unexpected sending bytes compressed with gzip setting property 'resteasy.gzip.max.input' HOT 5
- Support for `@InjectMock EntityManager` HOT 6
- Regression in 3.11.0: jpamodelgen WITH ECLIPSE COMPILER does not run and logs a warning: "Both Quarkus Hibernate ORM and Hibernate Reactive with Panache detected: this is not supported, so will proceed as if none were there" HOT 23
- Since 3.10.1 RequestContext handling is broken on request cancle HOT 12
- JDBC Driver - Microsoft SQL Server extension doesn't work in FIPS-enabled environment with OpenJDK 17 and RHEL8 but works with OpenJDK 21 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from quarkus.