proper graceful shutdown settings about stackstorm-k8s HOT 6 OPEN

Never comment anything ever. EVER.

from stackstorm-k8s.

After making this change for a shorter action_runner.graceful_shutdown we still see the actions getting stuck in a running state.

from stackstorm-k8s.

Let this be a lesson to everyone. Do NOT put inline comments in your config file:

...
 return opt.type(value)","host":"stackstorm","severity":"info","facility":"user",
{"timestamp":"2023-12-04T18:23:26+00:00","message":" 2023-12-04T18:23:19.7932543Z stdout F   File \"/opt/stackstorm/st2/lib/python3.8/site-packages/oslo_config/types.py\", line 145, in __call__","host":"st
{"timestamp":"2023-12-04T18:23:26+00:00","message":" 2023-12-04T18:23:19.79325789Z stdout F     value = int(value)","host":"stackstorm","severity":"info","facility":"user","sysl
{"timestamp":"2023-12-04T18:23:26+00:00","message":" 2023-12-04T18:23:19.793259422Z stdout F ValueError: invalid literal for int() with base 10: '610 # 10 mins'","host":"stackstorm
...

from stackstorm-k8s.

Whelp it turns out I STILL am not seeing graceful shutdowns. All executions immediately get abandoned.
I did some more digging.
Here we check the st2actionrunner service if it has any members.
But in the config for st2 coordination | service_registry defaults to FALSE.
So set service_registry = True in the coorination settings in the config.
We should probably make this the default?

from stackstorm-k8s.

I can confirm turning on the service registry fixed my graceful shutdown.

from stackstorm-k8s.

But in the config for st2 coordination | service_registry defaults to FALSE. So set service_registry = True in the coorination settings in the config. We should probably make this the default?

This looks like the right place to add anything in the chart:

We could do this:

     {{- if index .Values "redis" "enabled" }}
     [coordination]
+    service_registry = True
     url = redis://{{ template "stackstorm-ha.redis-password" $ }}{{ template "stackstorm-ha.redis-nodes" $ }}
     {{- end }}

I do not use the redis subchart, so that would not change the default for me. I pass this in via st2.config in values. So, we could also do something like:

+    [coordination]
+    service_registry = True
     {{- if index .Values "redis" "enabled" }}
-    [coordination]
     url = redis://{{ template "stackstorm-ha.redis-password" $ }}{{ template "stackstorm-ha.redis-nodes" $ }}
     {{- end }}

Which option would you prefer? Or something else entirely?

from stackstorm-k8s.