While the workflow instance is in progress (not in a stopped state), it would be nice to show some visual indicator for this (like a spinner), both in the search results and in the workflow view.

We should consider adding these indexes.

https://www.sqlshack.com/index-foreign-key-columns-sql-server/

Hi, we were doing testing with NFLOW and we wanted to purge all existing NFLOW data from Postgres. But after deleting all tables, triggers and functions, leaving just an empty database, NFLOW failed to start because it would seem that it didn't realize that the tables are gone and it needs to re-create them. How can we get Nflow working again on deletion of back end workflow data?

We are seeing the following entries in our logs:

17:42:25.518 INFO [main] io.nflow.engine.internal.storage.db.DatabaseInitializer [DatabaseInitializer.java : 40] - Creating database populator using script 'scripts/db/postgresql.create.ddl.sql'
17:42:25.596 WARN [main] io.nflow.engine.internal.storage.db.DatabaseInitializer [DatabaseInitializer.java : 31] - Failed to create the database, possibly already created: Failed to execute SQL script statement #1 of class path resource [scripts/db/postgresql.create.ddl.sql]: create type workflow_status as enum ('created', 'executing', 'inProgress', 'finished', 'manual'); nested exception is org.postgresql.util.PSQLException: ERROR: type "workflow_status" already exists

I have an NflowWorkflow that looks similar to this:

@Component
public class CustomerCreationWorkflow extends WorkflowDefinition<CustomerCreationWorkflow.State> {

	protected final fooRemoteService fooRemoteService;
	
	public CustomerCreationWorkflow(FooRemoteService fooRemoteService) {
			super(TYPE, State.createCustomer, State.done);
			this.fooRemoteService = fooRemoteService;
	}
	
	// STATE ETC
	
	public NextAction createCustomer(StateExecution execution,
	                                 @StateVar(value = REQUEST_DATA, readOnly = true) CustomerEnrollmentState request) {
		fooRemoteService.get(request.something)
	}
}

Which is leading to org.hibernate.lazyinitializationexception could not initialize proxy no session
Is there a way of using spring jpa inside an nflow workflow?
Or is that even possible?

What is the preferred way of doing something like this?

Sorry for bombarding your issue logs.
Is there a better place to put these?

I have included nflow-tests in my application's pom.xml like so:

<dependencies>
   ...
   <dependency>
      <groupId>io.nflow</groupId>
      <artifactId>nflow-tests</artifactId>
      <version>${nflow.version}</version>
      <scope>test</scope>
  </dependency>
</dependencies>

Upon starting my test application, it always gives this warning:

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/C:/Users/jacks/.m2/repository/ch/qos/logback/logback-classic/1.2.3/logback-classic-1.2.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/Users/jacks/.m2/repository/io/nflow/nflow-tests/7.2.4/nflow-tests-7.2.4.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
09:51:45,715 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback-test.xml]
09:51:45,717 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback.groovy]

I am not interested in the logback.xml that is added by nflow-tests, so I would like to exclude it. The normal way of doing that, is by excluding that dependency. But that doesn't help. I tried to exclude a lot of possible combinations like so:

<dependency>
  <groupId>io.nflow</groupId>
  <artifactId>nflow-tests</artifactId>
  <version>${nflow.version}</version>
  <scope>test</scope>
  <exclusions>
    <exclusion>
	    <groupId>org.slf4j</groupId>
	    <artifactId>jcl-over-slf4j</artifactId>
    </exclusion>
    <exclusion>
	    <groupId>org.slf4j</groupId>
	    <artifactId>log4j-over-slf4j</artifactId>
    </exclusion>
    <exclusion>
	    <groupId>ch.qos.logback</groupId>
	    <artifactId>logback-classic</artifactId>
    </exclusion>
    <exclusion>
	    <groupId>ch.qos.logback</groupId>
	    <artifactId>logback-core</artifactId>
    </exclusion>
    <exclusion>
	    <groupId>ch.qos.logback</groupId>
	    <artifactId>logback-access</artifactId>
    </exclusion>
    <exclusion>
	    <groupId>ch.qos.logback</groupId>
	    <artifactId>logback-parent</artifactId>
    </exclusion>
    <exclusion>
	    <groupId>org.slf4j</groupId>
	    <artifactId>slf4j-log4j12</artifactId>
    </exclusion>
    <exclusion>
	    <groupId>log4j</groupId>
	    <artifactId>log4j</artifactId>
    </exclusion>
    <exclusion>
	    <groupId>slf4j</groupId>
	    <artifactId>slf4j</artifactId>
    </exclusion>
    <exclusion>
	    <groupId>org.slf4j</groupId>
	    <artifactId>slf4j-api</artifactId>
    </exclusion>
    <exclusion>
	    <groupId>org.slf4j</groupId>
	    <artifactId>slf4j-parent</artifactId>
    </exclusion>
    <exclusion>
	    <groupId>org.slf4j</groupId>
	    <artifactId>slf4j-simple</artifactId>
    </exclusion>
    <exclusion>
	    <groupId>org.slf4j</groupId>
	    <artifactId>slf4j-nop</artifactId>
    </exclusion>
  </exclusions>
</dependency>

But none seems to get rid of the warning.

Taking the Credit application example, when the 'yes' path is followed it is possible to want to execute n parallel, independent Actions. Considering that 'moveTo' supports a single 'NextAction', how can this scenario be handled?

Currently nFlow has this index which is intended to make polling fast.

create index nflow_workflow_activation on nflow_workflow(next_activation, modified);

Now days this doesn't match any more the polling code:
https://github.com/NitorCreations/nflow/blob/master/nflow-engine/src/main/java/io/nflow/engine/internal/dao/WorkflowInstanceDao.java#L516

String whereConditionForInstanceUpdate() {
  return "where executor_id is null and status in (" + sqlVariants.workflowStatus(created) + ", "
      + sqlVariants.workflowStatus(inProgress) + ") and " + sqlVariants.dateLtEqDiff("next_activation", "current_timestamp")
      + " and " + executorInfo.getExecutorGroupCondition() + " order by next_activation asc";
}

There should be an index that is something like this to replace the current index

create index nflow_workflow_activation2 on nflow_workflow (next_activation, executor_id, status, executor_group);

Workflow instance page, Action history shows max 100 newest actions and numbers them from 1 to 100 even when there are more than 100 actions.

It should support fetching all the actions or paging to fetch them all. Also numbering actions is misleading currently.

http://bank.nflow.io/nflow/ui/explorer/#!/workflow/26

Currently embedded nFlow puts default CORS headers (e.g. "access-control-allow-origin": "*") to all requests in given Spring (Boot) application. These settings cannot be overridden by application. Could you make this feature configurable? Preferably nFlow's header-settings should not effect other endpoints of the application at all.

2org.springframework.dao.EmptyResultDataAccessException: Incorrect result size: expected 1, actual 0
	at org.springframework.dao.support.DataAccessUtils.nullableSingleResult(DataAccessUtils.java:97) ~[spring-tx-5.3.16.jar:5.3.16]
	at org.springframework.jdbc.core.JdbcTemplate.queryForObject(JdbcTemplate.java:880) ~[spring-jdbc-5.3.16.jar:5.3.16]
	at org.springframework.jdbc.core.JdbcTemplate.queryForObject(JdbcTemplate.java:906) ~[spring-jdbc-5.3.16.jar:5.3.16]
	at io.nflow.engine.internal.dao.WorkflowInstanceDao.updateWorkflowInstanceWithCTE(WorkflowInstanceDao.java:430) ~[nflow-engine-7.4.0.jar:na]
	at io.nflow.engine.internal.dao.WorkflowInstanceDao.updateWorkflowInstanceAfterExecution(WorkflowInstanceDao.java:326) ~[nflow-engine-7.4.0.jar:na]
	at io.nflow.engine.internal.dao.WorkflowInstanceDao$$FastClassBySpringCGLIB$$f1fc6e3.invoke(<generated>) ~[nflow-engine-7.4.0.jar:na]
	at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) ~[spring-core-5.3.16.jar:5.3.16]
	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:689) ~[spring-aop-5.3.16.jar:5.3.16]
	at io.nflow.engine.internal.dao.WorkflowInstanceDao$$EnhancerBySpringCGLIB$$f4fce4d3.updateWorkflowInstanceAfterExecution(<generated>) ~[nflow-engine-7.4.0.jar:na]
	at io.nflow.engine.internal.executor.WorkflowStateProcessor.persistWorkflowInstanceState(WorkflowStateProcessor.java:332) ~[nflow-engine-7.4.0.jar:na]
	at io.nflow.engine.internal.executor.WorkflowStateProcessor.saveWorkflowInstanceState(WorkflowStateProcessor.java:300) ~[nflow-engine-7.4.0.jar:na]
	at io.nflow.engine.internal.executor.WorkflowStateProcessor.runImpl(WorkflowStateProcessor.java:201) ~[nflow-engine-7.4.0.jar:na]
	at io.nflow.engine.internal.executor.WorkflowStateProcessor.run(WorkflowStateProcessor.java:129) ~[nflow-engine-7.4.0.jar:na]
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[na:na]
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[na:na]
	at java.base/java.lang.Thread.run(Thread.java:833) ~[na:na]

2022-03-15 13:52:20.826 ERROR 19087 --- [flow-executor-7] i.n.e.i.executor.WorkflowStateProcessor  : Failed to save workflow instance 428 new state, retrying after PT60S seconds.

This error is in loop. And this is the code that creates the loop

    while (true) {
      try {
        return persistWorkflowInstanceState(execution, instance.stateVariables, actionBuilder, instanceBuilder);
      } catch (Exception ex) {
        if (shutdownRequested.get()) {
          logger.error(
              "Failed to save workflow instance {} new state, not retrying due to shutdown request. The state will be rerun on recovery.",
              instance.id, ex);
          // return the original instance since persisting failed
          return instance;
        }
        StateSaveExceptionHandling handling = stateSaveExceptionAnalyzer.analyzeSafely(ex, saveRetryCount++);
        if (handling.logStackTrace) {
          nflowLogger.log(logger, handling.logLevel, "Failed to save workflow instance {} new state, retrying after {} seconds.",
              new Object[] { instance.id, handling.retryDelay, ex });
        } else {
          nflowLogger.log(logger, handling.logLevel,
              "Failed to save workflow instance {} new state, retrying after {} seconds. Error: {}",
              new Object[] { instance.id, handling.retryDelay, ex.getMessage() });
        }
        sleepIgnoreInterrupted(handling.retryDelay.getStandardSeconds());
      }
    }

The record is existing in the database , and if I run new instance it will be processed. But is there any way to break this loop ? or to restart the dispatcher/executioner ?

mvn versions:display-plugin-updates
...
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  3.483 s (Wall Clock)
[INFO] Finished at: 2020-01-22T08:03:25+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:versions-maven-plugin:2.7:display-plugin-updates (default-cli) on project nflow-engine: Failed to interpolate field: private java.util.List org.apache.maven.model.ModelBase.repositories on class: org.apache.maven.model.ModelBase: class org.apache.maven.project.interpolation.StringSearchModelInterpolator$InterpolateObjectAction cannot access a member of class org.apache.maven.model.ModelBase with modifiers "private" -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <args> -rf :nflow-engine

According to git bisect this started with commit dae3f13 Enable parallel maven module builds and junit tests

I have two data sources: a database and a message queue.
I have two steps in my workflow: Step A makes changes in the database, then Step B sends a message to the message queue.

I use the next scenario:

1. I run the workflow
2. Step A of the workflow makes changes in the database
3. I turn off the database (to simulate the fail)
4. nFlow tries to write to the database the fact that Step A is finished and obviously fails (log attached: nflow-sandbox.log.txt)
5. I turn on the database. As a result, the data is in inconsistent state: the changes were made in the database by Step A, but the notification was not sent by Step B.

Expected:
nFlow takes actions to make the data consistent again (e.g., starts the failed workflow from the beginning relying on the idempotency of the data sources).
Actual:
The data is left in the inconsistent state.

The project I use to reproduce the issue is here:
https://github.com/yaskovdev/distributed-transaction-sandbox

I am not sure I am interpreting this correctly but I have a timer workflow which retries every 30 minutes. But it doesn't seem to work when the nflow service is restarted or a new container set is deployed. What exactly is the recovered state?

This should fail at startup or configuration time:

  public static enum State implements WorkflowState {
    start, process, done
  }

  public DemoWorkflow() {
    super("demo", State.start);
    permit(State.start, State.process);
    permit(State.process, State.done);
  }

  public void start(StateExecution execution) {
    execution.setNextState(State.process);
    execution.setNextActivation(DateTime.now());    
  }

// process(StateExecution) method is missing

// done exists but has wrong signature
  public void done() {
    execution.setNextState(State.process);
    execution.setNextActivation(DateTime.now());    
  }

Hi,

I'm running a custom workflow using the plain Jetty runner (from the quickstart guide) like:
new StartNflow().startJetty(7500, "local", "");

When I use the nFLow Explorer to update the state I encounter the error below. It seems to be some classloading issue since it also worked on one occasion with the same config.

Tested on both version 7.2.4 and 7.3.0 on both Java 8 and 14. Same behaviour.

2021-04-08 20:59:17 WARN [qtp825658265-39] lipse.jetty.server.HttpChannel - [] /nflow/api/v1/workflow-instance/id/1
java.lang.NoClassDefFoundError: com/fasterxml/jackson/core/StreamReadCapability
at com.fasterxml.jackson.datatype.joda.deser.DateTimeDeserializer._fromString(DateTimeDeserializer.java:111)
at com.fasterxml.jackson.datatype.joda.deser.DateTimeDeserializer.deserialize(DateTimeDeserializer.java:53)
at com.fasterxml.jackson.datatype.joda.deser.DateTimeDeserializer.deserialize(DateTimeDeserializer.java:24)
at com.fasterxml.jackson.databind.deser.impl.FieldProperty.deserializeAndSet(FieldProperty.java:138)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:293)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:156)
at com.fasterxml.jackson.databind.ObjectReader._bind(ObjectReader.java:2020)
at com.fasterxml.jackson.databind.ObjectReader.readValue(ObjectReader.java:1179)
at com.fasterxml.jackson.jaxrs.base.ProviderBase.readFrom(ProviderBase.java:816)
at org.apache.cxf.jaxrs.utils.JAXRSUtils.readFromMessageBodyReader(JAXRSUtils.java:1430)
at org.apache.cxf.jaxrs.utils.JAXRSUtils.readFromMessageBody(JAXRSUtils.java:1382)
at org.apache.cxf.jaxrs.utils.JAXRSUtils.processRequestBodyParameter(JAXRSUtils.java:896)
at org.apache.cxf.jaxrs.utils.JAXRSUtils.processParameters(JAXRSUtils.java:832)
at org.apache.cxf.jaxrs.interceptor.JAXRSInInterceptor.processRequest(JAXRSInInterceptor.java:214)
at org.apache.cxf.jaxrs.interceptor.JAXRSInInterceptor.handleMessage(JAXRSInInterceptor.java:78)
at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:308)
at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121)
at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:267)
at org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:234)
at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:208)
at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:160)
at org.apache.cxf.transport.servlet.CXFNonSpringServlet.invoke(CXFNonSpringServlet.java:225)
at org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:298)
at org.apache.cxf.transport.servlet.AbstractHTTPServlet.doPut(AbstractHTTPServlet.java:234)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
at org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:273)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:791)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:550)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1435)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1350)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:516)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:388)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:633)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:380)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:773)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:905)
at java.base/java.lang.Thread.run(Thread.java:832)
Caused by: java.lang.ClassNotFoundException: com.fasterxml.jackson.core.StreamReadCapability
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:602)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)

Output of maven dependency tree:

[INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ nflow-test ---
[INFO] nl.scuro:nflow-test:jar:1.0-SNAPSHOT
[INFO] - io.nflow:nflow-jetty:jar:7.2.4:compile
[INFO] +- io.nflow:nflow-rest-api-jax-rs:jar:7.2.4:compile
[INFO] | +- io.nflow:nflow-rest-api-common:jar:7.2.4:compile
[INFO] | | - io.swagger:swagger-annotations:jar:1.6.2:compile
[INFO] | +- javax.ws.rs:javax.ws.rs-api:jar:2.1.1:compile
[INFO] | +- javax.validation:validation-api:jar:2.0.1.Final:compile
[INFO] | +- org.glassfish:javax.el:jar:3.0.0:runtime
[INFO] | - org.hibernate.validator:hibernate-validator:jar:6.1.7.Final:compile
[INFO] | +- jakarta.validation:jakarta.validation-api:jar:2.0.2:compile
[INFO] | +- org.jboss.logging:jboss-logging:jar:3.3.2.Final:compile
[INFO] | - com.fasterxml:classmate:jar:1.3.4:compile
[INFO] +- io.nflow:nflow-server-common:jar:7.2.4:compile
[INFO] | +- io.nflow:nflow-engine:jar:7.2.4:compile
[INFO] | | +- org.springframework:spring-context:jar:5.2.8.RELEASE:compile
[INFO] | | | +- org.springframework:spring-aop:jar:5.2.8.RELEASE:compile
[INFO] | | | - org.springframework:spring-expression:jar:5.2.8.RELEASE:compile
[INFO] | | +- org.springframework:spring-jdbc:jar:5.2.8.RELEASE:compile
[INFO] | | | - org.springframework:spring-tx:jar:5.2.8.RELEASE:compile
[INFO] | | +- com.fasterxml.jackson.core:jackson-annotations:jar:2.12.0:compile
[INFO] | | +- com.fasterxml.jackson.datatype:jackson-datatype-joda:jar:2.12.0:compile
[INFO] | | +- joda-time:joda-time:jar:2.10.8:compile
[INFO] | | +- javax.inject:javax.inject:jar:1:compile
[INFO] | | - com.zaxxer:HikariCP:jar:3.4.5:compile
[INFO] | +- org.slf4j:slf4j-api:jar:1.7.30:compile
[INFO] | +- org.slf4j:log4j-over-slf4j:jar:1.7.30:compile
[INFO] | +- org.slf4j:jcl-over-slf4j:jar:1.7.30:compile
[INFO] | +- ch.qos.logback:logback-classic:jar:1.2.3:compile
[INFO] | | - ch.qos.logback:logback-core:jar:1.2.3:compile
[INFO] | - org.slf4j:jul-to-slf4j:jar:1.7.30:compile
[INFO] +- com.nitorcreations:core-utils:jar:1.4:compile
[INFO] +- org.eclipse.jetty:jetty-server:jar:9.4.35.v20201120:compile
[INFO] | +- javax.servlet:javax.servlet-api:jar:3.1.0:compile
[INFO] | +- org.eclipse.jetty:jetty-http:jar:9.4.35.v20201120:compile
[INFO] | - org.eclipse.jetty:jetty-io:jar:9.4.35.v20201120:compile
[INFO] +- org.eclipse.jetty:jetty-servlet:jar:9.4.35.v20201120:compile
[INFO] | +- org.eclipse.jetty:jetty-security:jar:9.4.35.v20201120:compile
[INFO] | - org.eclipse.jetty:jetty-util-ajax:jar:9.4.35.v20201120:compile
[INFO] +- org.eclipse.jetty:jetty-jmx:jar:9.4.35.v20201120:compile
[INFO] | - org.eclipse.jetty:jetty-util:jar:9.4.35.v20201120:compile
[INFO] +- org.apache.cxf:cxf-rt-frontend-jaxrs:jar:3.4.1:compile
[INFO] | +- org.apache.cxf:cxf-core:jar:3.4.1:compile
[INFO] | | +- org.glassfish.jaxb:jaxb-runtime:jar:2.3.3:compile
[INFO] | | | +- org.glassfish.jaxb:txw2:jar:2.3.3:compile
[INFO] | | | - com.sun.istack:istack-commons-runtime:jar:3.0.11:compile
[INFO] | | +- com.fasterxml.woodstox:woodstox-core:jar:6.2.1:compile
[INFO] | | | - org.codehaus.woodstox:stax2-api:jar:4.2.1:compile
[INFO] | | +- org.apache.ws.xmlschema:xmlschema-core:jar:2.2.5:compile
[INFO] | | - jakarta.xml.bind:jakarta.xml.bind-api:jar:2.3.3:compile
[INFO] | +- jakarta.ws.rs:jakarta.ws.rs-api:jar:2.1.6:compile
[INFO] | +- jakarta.annotation:jakarta.annotation-api:jar:1.3.5:compile
[INFO] | +- org.apache.cxf:cxf-rt-transports-http:jar:3.4.1:compile
[INFO] | +- org.apache.cxf:cxf-rt-security:jar:3.4.1:compile
[INFO] | +- jakarta.xml.ws:jakarta.xml.ws-api:jar:2.3.3:compile
[INFO] | +- jakarta.jws:jakarta.jws-api:jar:2.1.0:compile
[INFO] | +- jakarta.xml.soap:jakarta.xml.soap-api:jar:1.4.2:compile
[INFO] | | - jakarta.activation:jakarta.activation-api:jar:1.2.2:compile
[INFO] | +- com.sun.activation:jakarta.activation:jar:1.2.2:compile
[INFO] | +- com.sun.xml.messaging.saaj:saaj-impl:jar:1.5.2:runtime
[INFO] | | - org.jvnet.staxex:stax-ex:jar:1.8.3:runtime
[INFO] | +- org.apache.geronimo.specs:geronimo-jta_1.1_spec:jar:1.1.1:compile
[INFO] | - org.jboss.spec.javax.rmi:jboss-rmi-api_1.0_spec:jar:1.0.6.Final:compile
[INFO] +- org.apache.cxf:cxf-rt-features-logging:jar:3.4.1:compile
[INFO] +- org.apache.cxf:cxf-rt-management:jar:3.4.1:compile
[INFO] +- org.apache.cxf:cxf-rt-rs-service-description-swagger:jar:3.4.1:compile
[INFO] | +- org.apache.cxf:cxf-rt-rs-service-description-common-openapi:jar:3.4.1:compile
[INFO] | - org.apache.cxf:cxf-rt-rs-service-description-swagger-ui:jar:3.4.1:compile
[INFO] +- io.swagger:swagger-jaxrs:jar:1.6.2:compile
[INFO] | +- com.fasterxml.jackson.core:jackson-databind:jar:2.11.1:compile
[INFO] | | - com.fasterxml.jackson.core:jackson-core:jar:2.11.1:compile
[INFO] | +- io.swagger:swagger-core:jar:1.6.2:compile
[INFO] | | +- com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:jar:2.11.1:compile
[INFO] | | | - org.yaml:snakeyaml:jar:1.26:compile
[INFO] | | - io.swagger:swagger-models:jar:1.6.2:compile
[INFO] | +- org.reflections:reflections:jar:0.9.11:compile
[INFO] | | - org.javassist:javassist:jar:3.21.0-GA:compile
[INFO] | - com.google.guava:guava:jar:27.0.1-android:compile
[INFO] | +- com.google.guava:failureaccess:jar:1.0.1:compile
[INFO] | +- com.google.guava:listenablefuture:jar:9999.0-empty-to-avoid-conflict-with-guava:compile
[INFO] | +- com.google.code.findbugs:jsr305:jar:3.0.2:compile
[INFO] | +- org.checkerframework:checker-compat-qual:jar:2.5.2:compile
[INFO] | +- com.google.errorprone:error_prone_annotations:jar:2.2.0:compile
[INFO] | +- com.google.j2objc:j2objc-annotations:jar:1.1:compile
[INFO] | - org.codehaus.mojo:animal-sniffer-annotations:jar:1.17:compile
[INFO] +- com.fasterxml.jackson.jaxrs:jackson-jaxrs-json-provider:jar:2.12.0:compile
[INFO] | +- com.fasterxml.jackson.jaxrs:jackson-jaxrs-base:jar:2.12.0:compile
[INFO] | - com.fasterxml.jackson.module:jackson-module-jaxb-annotations:jar:2.12.0:compile
[INFO] +- org.springframework:spring-web:jar:5.2.8.RELEASE:compile
[INFO] | +- org.springframework:spring-beans:jar:5.2.8.RELEASE:compile
[INFO] | - org.springframework:spring-core:jar:5.2.8.RELEASE:compile
[INFO] | - org.springframework:spring-jcl:jar:5.2.8.RELEASE:compile
[INFO] +- org.apache.commons:commons-lang3:jar:3.11:compile
[INFO] +- com.h2database:h2:jar:1.4.200:compile
[INFO] +- io.dropwizard.metrics:metrics-servlets:jar:4.1.16:compile
[INFO] | +- io.dropwizard.metrics:metrics-core:jar:4.1.16:compile
[INFO] | +- io.dropwizard.metrics:metrics-healthchecks:jar:4.1.16:compile
[INFO] | +- io.dropwizard.metrics:metrics-json:jar:4.1.16:compile
[INFO] | +- io.dropwizard.metrics:metrics-jvm:jar:4.1.16:compile
[INFO] | - com.helger:profiler:jar:1.1.1:compile
[INFO] +- javax.xml.bind:jaxb-api:jar:2.3.1:compile
[INFO] | - javax.activation:javax.activation-api:jar:1.2.0:compile
[INFO] - javax.xml.ws:jaxws-api:jar:2.3.1:compile
[INFO] +- javax.xml.soap:javax.xml.soap-api:jar:1.4.0:compile
[INFO] - javax.annotation:javax.annotation-api:jar:1.3.2:compile

Problem

The nFlow version and configuration (properties) are currently not visible and need to be deduced from runtime environment specific configuration. It might be unclear which configuration is effective, if the runtime platform hides deployment details.

Even if the current configuration can be reliably deduced, troubleshooting historical issues may be challenging without knowing which nFlow version and configuration which the executor ran.

Related to nFlow version, Explorer has currently no way of knowing what to expect (e.g. which REST API operations it supports) from the nFlow executor that generated the response. Explorer might no longer even support a very old nFlow REST API.

Solution

Add and persist the following DB columns from nflow-engine.

• nflow_executor.nflow_version (varchar): nFlow version of the executor
• nflow_executor.configuration (json): JSON structure generated from system properties

The included properties could be e.g. configured using include- and exclude-properties (regular expressions). By default include could have regular expression ^nflow.* and exclude .*password.*.

Expose the new nflow_executor columns from /v1/workflow-executor REST endpoint. Add the nFlow version as a response header to every endpoint so that Explorer knows which nFlow version generated the response.

Visualize the new nflow_executor columns in Explorer executors-tab. Probably useful to enable browser in-memory free text search feature from mui-datatables-component.

Add deprecation warning to nflow-explorer e.g. About page that it will be deprecated and replaced by nflow-explorer-ng in nFlow 9.0.0. Add link to wiki to for further information.

Legacy nflow-explorer got queryArchive configuration parameter while nflow-explorer-ng was being developed:
https://github.com/NitorCreations/nflow/blob/master/nflow-explorer/src/config.js#L43

The capability should be integrated into nflow-explorer-ng as well. Perhaps a checkbox in workflow instance search form would be a more flexible solution, so users could control when potentially time consuming archive search is performed.

Since I interpreted the Getting Started guide, that you should be able to start nFlow REST API also in a non-Spring (but JEE) environment, I thought it should be possible to embed it into Quarkus (quarkus.io) adding to a bare minimum Quarkus app simply pom.xml:

    <dependency>
        <groupId>io.nflow</groupId>
        <artifactId>nflow-engine</artifactId>
        <version>${nflow.version}</version>
    </dependency>

I added two lines in application.properties to allow CDI to discover the JAX-RS beans

quarkus.index-dependency.nflow.group-id=io.nflow
quarkus.index-dependency.nflow.artifact-id=nflow-rest-api-jax-rs

But unfortunately that seemed not to be sufficient, still some beans can not be wired, on Quarkus (dev) startup the following dependencies are claimed:

ERROR [io.qua.dep.dev.IsolatedDevModeMain] (main) Failed to start quarkus: java.lang.RuntimeException: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
	[error]: Build step io.quarkus.arc.deployment.ArcProcessor#validate threw an exception: javax.enterprise.inject.spi.DeploymentException: Found 15 deployment problems: 
[1] Unsatisfied dependency for type org.springframework.core.env.Environment and qualifiers [@Default]
	- java member: io.nflow.rest.config.jaxrs.CorsHeaderContainerResponseFilter#<init>()
	- declared on CLASS bean [types=[java.lang.Object, io.nflow.rest.config.jaxrs.CorsHeaderContainerResponseFilter], qualifiers=[@Default, @Any], target=io.nflow.rest.config.jaxrs.CorsHeaderContainerResponseFilter]
[2] Unsatisfied dependency for type io.nflow.rest.v1.converter.StatisticsConverter and qualifiers [@Default]
	- java member: io.nflow.rest.v1.jaxrs.StatisticsResource#statisticsConverter
	- declared on CLASS bean [types=[java.lang.Object, io.nflow.rest.v1.jaxrs.StatisticsResource], qualifiers=[@Default, @Any], target=io.nflow.rest.v1.jaxrs.StatisticsResource]
[3] Unsatisfied dependency for type io.nflow.engine.service.StatisticsService and qualifiers [@Default]
	- java member: io.nflow.rest.v1.jaxrs.StatisticsResource#statisticsService
	- declared on CLASS bean [types=[java.lang.Object, io.nflow.rest.v1.jaxrs.StatisticsResource], qualifiers=[@Default, @Any], target=io.nflow.rest.v1.jaxrs.StatisticsResource]
[4] Unsatisfied dependency for type io.nflow.engine.service.WorkflowInstanceService and qualifiers [@Default]
	- java member: io.nflow.rest.v1.jaxrs.WorkflowInstanceResource#<init>()
	- declared on CLASS bean [types=[java.lang.Object, io.nflow.rest.v1.jaxrs.WorkflowInstanceResource], qualifiers=[@Default, @Any], target=io.nflow.rest.v1.jaxrs.WorkflowInstanceResource]
[5] Unsatisfied dependency for type io.nflow.rest.v1.converter.CreateWorkflowConverter and qualifiers [@Default]
	- java member: io.nflow.rest.v1.jaxrs.WorkflowInstanceResource#<init>()
	- declared on CLASS bean [types=[java.lang.Object, io.nflow.rest.v1.jaxrs.WorkflowInstanceResource], qualifiers=[@Default, @Any], target=io.nflow.rest.v1.jaxrs.WorkflowInstanceResource]
[6] Unsatisfied dependency for type io.nflow.rest.v1.converter.ListWorkflowInstanceConverter and qualifiers [@Default]
	- java member: io.nflow.rest.v1.jaxrs.WorkflowInstanceResource#<init>()
	- declared on CLASS bean [types=[java.lang.Object, io.nflow.rest.v1.jaxrs.WorkflowInstanceResource], qualifiers=[@Default, @Any], target=io.nflow.rest.v1.jaxrs.WorkflowInstanceResource]
[7] Unsatisfied dependency for type io.nflow.engine.workflow.instance.WorkflowInstanceFactory and qualifiers [@Default]
	- java member: io.nflow.rest.v1.jaxrs.WorkflowInstanceResource#<init>()
	- declared on CLASS bean [types=[java.lang.Object, io.nflow.rest.v1.jaxrs.WorkflowInstanceResource], qualifiers=[@Default, @Any], target=io.nflow.rest.v1.jaxrs.WorkflowInstanceResource]
[8] Unsatisfied dependency for type io.nflow.engine.internal.dao.WorkflowInstanceDao and qualifiers [@Default]
	- java member: io.nflow.rest.v1.jaxrs.WorkflowInstanceResource#<init>()
	- declared on CLASS bean [types=[java.lang.Object, io.nflow.rest.v1.jaxrs.WorkflowInstanceResource], qualifiers=[@Default, @Any], target=io.nflow.rest.v1.jaxrs.WorkflowInstanceResource]
[9] Unsatisfied dependency for type io.nflow.rest.v1.converter.MaintenanceConverter and qualifiers [@Default]
	- java member: io.nflow.rest.v1.jaxrs.MaintenanceResource#converter
	- declared on CLASS bean [types=[io.nflow.rest.v1.jaxrs.MaintenanceResource, java.lang.Object], qualifiers=[@Default, @Any], target=io.nflow.rest.v1.jaxrs.MaintenanceResource]
[10] Unsatisfied dependency for type io.nflow.engine.service.MaintenanceService and qualifiers [@Default]
	- java member: io.nflow.rest.v1.jaxrs.MaintenanceResource#maintenanceService
	- declared on CLASS bean [types=[io.nflow.rest.v1.jaxrs.MaintenanceResource, java.lang.Object], qualifiers=[@Default, @Any], target=io.nflow.rest.v1.jaxrs.MaintenanceResource]
[11] Unsatisfied dependency for type io.nflow.engine.service.WorkflowExecutorService and qualifiers [@Default]
	- java member: io.nflow.rest.v1.jaxrs.WorkflowExecutorResource#<init>()
	- declared on CLASS bean [types=[io.nflow.rest.v1.jaxrs.WorkflowExecutorResource, java.lang.Object], qualifiers=[@Default, @Any], target=io.nflow.rest.v1.jaxrs.WorkflowExecutorResource]
[12] Unsatisfied dependency for type io.nflow.rest.v1.converter.ListWorkflowExecutorConverter and qualifiers [@Default]
	- java member: io.nflow.rest.v1.jaxrs.WorkflowExecutorResource#<init>()
	- declared on CLASS bean [types=[io.nflow.rest.v1.jaxrs.WorkflowExecutorResource, java.lang.Object], qualifiers=[@Default, @Any], target=io.nflow.rest.v1.jaxrs.WorkflowExecutorResource]
[13] Unsatisfied dependency for type io.nflow.engine.service.WorkflowDefinitionService and qualifiers [@Default]
	- java member: io.nflow.rest.v1.jaxrs.WorkflowDefinitionResource#<init>()
	- declared on CLASS bean [types=[io.nflow.rest.v1.jaxrs.WorkflowDefinitionResource, java.lang.Object], qualifiers=[@Default, @Any], target=io.nflow.rest.v1.jaxrs.WorkflowDefinitionResource]
[14] Unsatisfied dependency for type io.nflow.rest.v1.converter.ListWorkflowDefinitionConverter and qualifiers [@Default]
	- java member: io.nflow.rest.v1.jaxrs.WorkflowDefinitionResource#<init>()
	- declared on CLASS bean [types=[io.nflow.rest.v1.jaxrs.WorkflowDefinitionResource, java.lang.Object], qualifiers=[@Default, @Any], target=io.nflow.rest.v1.jaxrs.WorkflowDefinitionResource]
[15] Unsatisfied dependency for type io.nflow.engine.internal.dao.WorkflowDefinitionDao and qualifiers [@Default]
	- java member: io.nflow.rest.v1.jaxrs.WorkflowDefinitionResource#<init>()
	- declared on CLASS bean [types=[io.nflow.rest.v1.jaxrs.WorkflowDefinitionResource, java.lang.Object], qualifiers=[@Default, @Any], target=io.nflow.rest.v1.jaxrs.WorkflowDefinitionResource]

Hi there
been trying to get nflow working for a project however recently came into this issue when a class has a timestamp
(This is using java11)

INFO   | jvm 1    | 2022/03/01 14:25:21 | String: "Failed to serialize value for fooVariable"
INFO   | jvm 1    | 2022/03/01 14:25:21 | java.lang.RuntimeException: Failed to serialize value for fooVariable
INFO   | jvm 1    | 2022/03/01 14:25:21 |       at io.nflow.engine.internal.workflow.ObjectStringMapper.convertFromObject(ObjectStringMapper.java:106)
INFO   | jvm 1    | 2022/03/01 14:25:21 |       at io.nflow.engine.workflow.instance.WorkflowInstance$Builder.putStateVariable(WorkflowInstance.java:470)
Caused by: com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Java 8 date/time type `java.time.ZonedDateTime` not supported by default: add Module "com.fasterxml.jackson.datatype:jackson-datatype-jsr310" to enable handling (through reference chain: REMOVED)
INFO   | jvm 1    | 2022/03/01 14:25:21 |       at com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:77)
INFO   | jvm 1    | 2022/03/01 14:25:21 |       at com.fasterxml.jackson.databind.SerializerProvider.reportBadDefinition(SerializerProvider.java:1300)
INFO   | jvm 1    | 2022/03/01 14:25:21 |       at com.fasterxml.jackson.databind.ser.impl.UnsupportedTypeSerializer.serialize(UnsupportedTypeSerializer.java:35)
INFO   | jvm 1    | 2022/03/01 14:25:21 |       at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:728)
INFO   | jvm 1    | 2022/03/01 14:25:21 |       at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:774)
INFO   | jvm 1    | 2022/03/01 14:25:21 |       at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178)
INFO   | jvm 1    | 2022/03/01 14:25:21 |       at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:728)
INFO   | jvm 1    | 2022/03/01 14:25:21 |       at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:774)
INFO   | jvm 1    | 2022/03/01 14:25:21 |       at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178)
INFO   | jvm 1    | 2022/03/01 14:25:21 |       at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119)
INFO   | jvm 1    | 2022/03/01 14:25:21 |       at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79)
INFO   | jvm 1    | 2022/03/01 14:25:21 |       at com.fasterxml.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18)
INFO   | jvm 1    | 2022/03/01 14:25:21 |       at com.fasterxml.jackson.databind.ser.BeanPropertyWriter.serializeAsField(BeanPropertyWriter.java:728)
INFO   | jvm 1    | 2022/03/01 14:25:21 |       at com.fasterxml.jackson.databind.ser.std.BeanSerializerBase.serializeFields(BeanSerializerBase.java:774)
INFO   | jvm 1    | 2022/03/01 14:25:21 |       at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:178)
INFO   | jvm 1    | 2022/03/01 14:25:21 |       at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)
INFO   | jvm 1    | 2022/03/01 14:25:21 |       at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
INFO   | jvm 1    | 2022/03/01 14:25:21 |       at com.fasterxml.jackson.databind.ObjectMapper._writeValueAndClose(ObjectMapper.java:4568)
INFO   | jvm 1    | 2022/03/01 14:25:21 |       at com.fasterxml.jackson.databind.ObjectMapper.writeValueAsString(ObjectMapper.java:3821)
INFO   | jvm 1    | 2022/03/01 14:25:21 |       at io.nflow.engine.internal.workflow.ObjectStringMapper.convertFromObject(ObjectStringMapper.java:104)
INFO   | jvm 1    | 2022/03/01 14:25:21 |       ... 108 common frames omitted

The class would have something similar to this:
private LocalDate expiryDate;
Sorry if this is very vague.

Let me know if you need anything else.

I have developed a workflow which periodically executes a single action by ending the action with

return NextAction.retryAfter(dateTime, "Processing timer event");
Where dateTime is something on the order of 5 to 15 minutes in the future with respect to the time the statement is executed.

At any rate, I launch the workflow in the @PostConstruct method of one of our rest controllers as so:

workflowInstances.insertWorkflowInstance( workflowInstanceFactory.newWorkflowInstanceBuilder().setType(TimerWorkflow.TYPE) .setExternalId("TIMER_MODULE") .build());

At any rate the first time I ran this in an environment it worked just fine. But when I re-deployed the container and it started up again, I got this error:

00:11:41.316 WARN [main] io.nflow.engine.internal.dao.WorkflowInstanceDao [WorkflowInstanceDao.java : 200] - Failed to insert workflow instance
org.springframework.dao.DuplicateKeyException: PreparedStatementCallback; SQL [with wf as (insert into nflow_workflow(type, priority, parent_workflow_id, parent_action_id, business_key, external_id, executor_group, status, state, state_text, next_activation, workflow_signal) values (?, ?, ?, ?, ?, ?, ?, ?::workflow_status, ?, ?, ?, ?) returning id) select wf.id from wf]; ERROR: duplicate key value violates unique constraint "nflow_workflow_uniq"
Detail: Key (external_id, type, executor_group)=(TIMER_MODULE, timerWorkflow, nflow) already exists.; nested exception is org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "nflow_workflow_uniq"
Detail: Key (external_id, type, executor_group)=(TIMER_MODULE, timerWorkflow, nflow) already exists.

I presume this is because the external Id is hard coded. So the question is, what is the right way to launch a repeating workflow that continues to execute indefinitely in an environment where the executors may be containers which are by their very nature ephemeral. What should the workflow startup login actually look like?

Also is there any issue with letting a single workflow instance run forever waking periodically to execute sundry housekeeping tasks?

We've seen some issues where a deploy of our service resurrects some old batch workflows due to them being recovered on a restart.

NFlow 7.1.0 running on 4 nflow nodes with 2 executors each on top of jdk11 + postgres11.

In this case we have workflows that are scheduled or poll for data, partition the work and create child workflows for the partitions. Using the basic batch work pattern the children complete work and wake up the parent that then schedules more child workflows until all done.

After upgrade to 7.1.0 we randomly see this deadlock error where something goes wrong on persisting workflow changes.

Currently this causes that any executor this happens to, gets caught in a persistent loop of retrying the same operation every 60 seconds.

This happens initially:

org.springframework.dao.DeadlockLoserDataAccessException: PreparedStatementCallback; SQL [update nflow_workflow set next_activation = (case when executor_id is null then case when next_activation <= current_timestamp then next_activation else current_timestamp end else next_activation end), external_next_activation = current_timestamp where executor_group = 'nflow' and id = ? and next_activation is not null]; ERROR: deadlock detected
  Detail: Process 4017 waits for ShareLock on transaction 18204573; blocked by process 12872.
Process 12872 waits for ShareLock on transaction 18204572; blocked by process 18526.
Process 18526 waits for ExclusiveLock on tuple (1467,1) of relation 16715 of database 16396; blocked by process 4017.
  Hint: See server log for query details.
  Where: while locking tuple (1467,1) in relation "nflow_workflow"; nested exception is org.postgresql.util.PSQLException: ERROR: deadlock detected
  Detail: Process 4017 waits for ShareLock on transaction 18204573; blocked by process 12872.
Process 12872 waits for ShareLock on transaction 18204572; blocked by process 18526.
Process 18526 waits for ExclusiveLock on tuple (1467,1) of relation 16715 of database 16396; blocked by process 4017.
  Hint: See server log for query details.
  Where: while locking tuple (1467,1) in relation "nflow_workflow"
	at org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.doTranslate(SQLErrorCodeSQLExceptionTranslator.java:267)
	at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72)
	at org.springframework.jdbc.core.JdbcTemplate.translateException(JdbcTemplate.java:1443)
	at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:633)
	at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:862)
	at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:917)
	at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:927)
	at io.nflow.engine.internal.dao.WorkflowInstanceDao.addExpectedStatesToQueryAndUpdate(WorkflowInstanceDao.java:502)
	at io.nflow.engine.internal.dao.WorkflowInstanceDao.wakeUpWorkflowExternally(WorkflowInstanceDao.java:480)
	at io.nflow.engine.internal.dao.WorkflowInstanceDao$$FastClassBySpringCGLIB$$f1fc6e3.invoke(<generated>)
	at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:685)
	at io.nflow.engine.internal.dao.WorkflowInstanceDao$$EnhancerBySpringCGLIB$$4f0e63fa.wakeUpWorkflowExternally(<generated>)
	at io.nflow.engine.internal.executor.WorkflowStateProcessor.lambda$processSuccess$2(WorkflowStateProcessor.java:303)
	at java.base/java.util.Optional.ifPresent(Optional.java:183)
	at io.nflow.engine.internal.executor.WorkflowStateProcessor.processSuccess(WorkflowStateProcessor.java:301)
	at io.nflow.engine.internal.executor.WorkflowStateProcessor.persistWorkflowInstanceState(WorkflowStateProcessor.java:292)
	at io.nflow.engine.internal.executor.WorkflowStateProcessor.saveWorkflowInstanceState(WorkflowStateProcessor.java:264)
	at io.nflow.engine.internal.executor.WorkflowStateProcessor.runImpl(WorkflowStateProcessor.java:180)
	at io.nflow.engine.internal.executor.WorkflowStateProcessor.run(WorkflowStateProcessor.java:117)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: org.postgresql.util.PSQLException: ERROR: deadlock detected
  Detail: Process 4017 waits for ShareLock on transaction 18204573; blocked by process 12872.
Process 12872 waits for ShareLock on transaction 18204572; blocked by process 18526.
Process 18526 waits for ExclusiveLock on tuple (1467,1) of relation 16715 of database 16396; blocked by process 4017.
  Hint: See server log for query details.
  Where: while locking tuple (1467,1) in relation "nflow_workflow"
	at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2497)
	at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2233)
	at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:310)
	at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:446)
	at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:370)
	at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:149)
	at org.postgresql.jdbc.PgPreparedStatement.executeUpdate(PgPreparedStatement.java:124)
	at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeUpdate(ProxyPreparedStatement.java:61)
	at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeUpdate(HikariProxyPreparedStatement.java)
	at org.springframework.jdbc.core.JdbcTemplate.lambda$update$0(JdbcTemplate.java:867)
	at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:617)
	... 19 common frames omitted
	

And this every 60 seconds afterwards

org.springframework.dao.EmptyResultDataAccessException: Incorrect result size: expected 1, actual 0
	at org.springframework.dao.support.DataAccessUtils.nullableSingleResult(DataAccessUtils.java:97)
	at org.springframework.jdbc.core.JdbcTemplate.queryForObject(JdbcTemplate.java:784)
	at org.springframework.jdbc.core.JdbcTemplate.queryForObject(JdbcTemplate.java:809)
	at io.nflow.engine.internal.dao.WorkflowInstanceDao.updateWorkflowInstanceWithCTE(WorkflowInstanceDao.java:428)
	at io.nflow.engine.internal.dao.WorkflowInstanceDao.updateWorkflowInstanceAfterExecution(WorkflowInstanceDao.java:325)
	at io.nflow.engine.internal.dao.WorkflowInstanceDao$$FastClassBySpringCGLIB$$f1fc6e3.invoke(<generated>)
	at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:685)
	at io.nflow.engine.internal.dao.WorkflowInstanceDao$$EnhancerBySpringCGLIB$$4f0e63fa.updateWorkflowInstanceAfterExecution(<generated>)
	at io.nflow.engine.internal.executor.WorkflowStateProcessor.persistWorkflowInstanceState(WorkflowStateProcessor.java:290)
	at io.nflow.engine.internal.executor.WorkflowStateProcessor.saveWorkflowInstanceState(WorkflowStateProcessor.java:264)
	at io.nflow.engine.internal.executor.WorkflowStateProcessor.runImpl(WorkflowStateProcessor.java:180)
	at io.nflow.engine.internal.executor.WorkflowStateProcessor.run(WorkflowStateProcessor.java:117)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)

I can see via a thread dump that the executors are stuck in WorkflowStateProcessor#sleepIgnoreInterrupted in the catch branch of WorkflowStateProcessor#saveWorkflowInstanceState on line 268 in version 7.1.0.

Looking at the 7.2.0 changes, they don't quite touch these code paths, but already upgrading to 7.2.0. But as this is completely random, it's quite hard to know if it's fixed.

Currently state processing can modify state variables, nextState and nextStateExecution fields.

When nFlow explorer is exposed via HTTPS, the browser issues console errors as follows:

[Warning] [blocked] The page at https://localhost:8888/explorer/index.html was not allowed to display insecure content from http://fonts.gstatic.com/s/opensans/v10/cJZKeOuBrn4kERxqtaUH3ZBw1xU1rKptJj_0jans920.woff2.

[Warning] [blocked] The page at https://localhost:8888/explorer/index.html#!/workflow-definition/sendMessageWorkflow was not allowed to display insecure content from http://fonts.gstatic.com/s/opensans/v10/u-WUoqrET9fUeobQW7jkRZBw1xU1rKptJj_0jans920.woff2.
vendor.863efce5.js:1:1243323

This is not blocking but apparently you have schemes hard coded to http on some of your resources. Might be better to have these resources default to HTTPS.

Expected something to start up and be able to access NFLOW server on localhost but instead get:

C02WT117HTDG:nflow-polot sowens$ java -jar target/nflow-pilot-jar-with-dependencies.jar --env localstack 2021-10-30 03:06:15 INFO [main] pring.NflowStandardEnvironment - [] Failed to initialize environment-specific properties from resource local.properties 2021-10-30 03:06:15 INFO [main] pring.NflowStandardEnvironment - [] Failed to initialize environment-specific properties from resource common.properties 2021-10-30 03:06:16 INFO [main] io.nflow.jetty.StartNflow - [] nFlow UI static resources served from jar:file:/Users/sowens/SecureIT_Code/nflow-polot/target/nflow-pilot-jar-with-dependencies.jar!/nflow-ui-assets 2021-10-30 03:06:17 INFO [main] onfig.db.DatabaseConfiguration - [] Database connection to h2 using jdbc:h2:mem:test;TRACE_LEVEL_FILE=4 2021-10-30 03:06:17 INFO [main] storage.db.DatabaseInitializer - [] Creating database populator using script 'scripts/db/h2.create.ddl.sql' 2021-10-30 03:06:17 INFO [main] storage.db.DatabaseInitializer - [] Database created. 2021-10-30 03:06:19 INFO [main] ngine.internal.dao.ExecutorDao - [] Joining executor group nflow 2021-10-30 03:06:19 INFO [main] ngine.internal.dao.ExecutorDao - [] Joined executor group nflow as executor 1 running on host spoon.go.com with process id 76235. 2021-10-30 03:06:19 INFO [main] vice.WorkflowDefinitionService - [] Added workflow type: bulk (io.nflow.engine.workflow.curated.BulkWorkflow) 2021-10-30 03:06:19 INFO [main] vice.WorkflowDefinitionService - [] Added workflow type: nFlowMaintenance (io.nflow.engine.workflow.curated.MaintenanceWorkflow) 2021-10-30 03:06:19 WARN [main] ionConfigWebApplicationContext - [] Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'jaxRsServer' defined in io.nflow.jetty.config.NflowJettyConfiguration: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.apache.cxf.endpoint.Server]: Factory method 'jaxRsServer' threw exception; nested exception is org.apache.cxf.service.factory.ServiceConstructionException 2021-10-30 03:06:19 ERROR [main] work.web.context.ContextLoader - [] Context initialization failed org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'jaxRsServer' defined in io.nflow.jetty.config.NflowJettyConfiguration: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.apache.cxf.endpoint.Server]: Factory method 'jaxRsServer' threw exception; nested exception is org.apache.cxf.service.factory.ServiceConstructionException at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:655) at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:635) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1336) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1176) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:556) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:516) at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:324) at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:226) at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:322) at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202) at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:897) at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:879) at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:551) at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:401) at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:292) at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:103) at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1068) at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:572) at org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:997) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:746) at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:379) at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:911) at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.server.Server.start(Server.java:423) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.server.Server.doStart(Server.java:387) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at io.nflow.jetty.StartNflow.startJetty(StartNflow.java:106) at io.nflow.jetty.StartNflow.startJetty(StartNflow.java:86) at io.nflow.jetty.StartNflow.startJetty(StartNflow.java:79) at com.disney.cst.iam.nflow.pilot.Main.main(Main.java:81) Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.apache.cxf.endpoint.Server]: Factory method 'jaxRsServer' threw exception; nested exception is org.apache.cxf.service.factory.ServiceConstructionException at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:185) at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:650) ... 37 common frames omitted Caused by: org.apache.cxf.service.factory.ServiceConstructionException: null at org.apache.cxf.jaxrs.JAXRSServerFactoryBean.create(JAXRSServerFactoryBean.java:216) at io.nflow.jetty.config.NflowJettyConfiguration.jaxRsServer(NflowJettyConfiguration.java:86) at io.nflow.jetty.config.NflowJettyConfiguration$$EnhancerBySpringCGLIB$$ff1f9d45.CGLIB$jaxRsServer$0(<generated>) at io.nflow.jetty.config.NflowJettyConfiguration$$EnhancerBySpringCGLIB$$ff1f9d45$$FastClassBySpringCGLIB$$25801388.invoke(<generated>) at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:244) at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:331) at io.nflow.jetty.config.NflowJettyConfiguration$$EnhancerBySpringCGLIB$$ff1f9d45.jaxRsServer(<generated>) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154) ... 38 common frames omitted Caused by: org.apache.cxf.BusException: No DestinationFactory was found for the namespace http://cxf.apache.org/transports/http. at org.apache.cxf.bus.managers.DestinationFactoryManagerImpl.getDestinationFactory(DestinationFactoryManagerImpl.java:124) at org.apache.cxf.endpoint.ServerImpl.initDestination(ServerImpl.java:81) at org.apache.cxf.endpoint.ServerImpl.<init>(ServerImpl.java:64) at org.apache.cxf.jaxrs.JAXRSServerFactoryBean.create(JAXRSServerFactoryBean.java:170) ... 49 common frames omitted 2021-10-30 03:06:19 ERROR [main] isney.cst.iam.nflow.pilot.Main - [] Unexpected failure org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'jaxRsServer' defined in io.nflow.jetty.config.NflowJettyConfiguration: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.apache.cxf.endpoint.Server]: Factory method 'jaxRsServer' threw exception; nested exception is org.apache.cxf.service.factory.ServiceConstructionException at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:655) at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:635) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1336) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1176) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:556) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:516) at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:324) at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:226) at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:322) at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202) at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:897) at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:879) at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:551) at org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:401) at org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:292) at org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:103) at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1068) at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:572) at org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:997) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:746) at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:379) at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:911) at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) at org.eclipse.jetty.server.Server.start(Server.java:423) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) at org.eclipse.jetty.server.Server.doStart(Server.java:387) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) at io.nflow.jetty.StartNflow.startJetty(StartNflow.java:106) at io.nflow.jetty.StartNflow.startJetty(StartNflow.java:86) at io.nflow.jetty.StartNflow.startJetty(StartNflow.java:79) at com.disney.cst.iam.nflow.pilot.Main.main(Main.java:81) Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.apache.cxf.endpoint.Server]: Factory method 'jaxRsServer' threw exception; nested exception is org.apache.cxf.service.factory.ServiceConstructionException at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:185) at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:650) ... 37 common frames omitted Caused by: org.apache.cxf.service.factory.ServiceConstructionException: null at org.apache.cxf.jaxrs.JAXRSServerFactoryBean.create(JAXRSServerFactoryBean.java:216) at io.nflow.jetty.config.NflowJettyConfiguration.jaxRsServer(NflowJettyConfiguration.java:86) at io.nflow.jetty.config.NflowJettyConfiguration$$EnhancerBySpringCGLIB$$ff1f9d45.CGLIB$jaxRsServer$0(<generated>) at io.nflow.jetty.config.NflowJettyConfiguration$$EnhancerBySpringCGLIB$$ff1f9d45$$FastClassBySpringCGLIB$$25801388.invoke(<generated>) at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:244) at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:331) at io.nflow.jetty.config.NflowJettyConfiguration$$EnhancerBySpringCGLIB$$ff1f9d45.jaxRsServer(<generated>) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154) ... 38 common frames omitted Caused by: org.apache.cxf.BusException: No DestinationFactory was found for the namespace http://cxf.apache.org/transports/http. at org.apache.cxf.bus.managers.DestinationFactoryManagerImpl.getDestinationFactory(DestinationFactoryManagerImpl.java:124) at org.apache.cxf.endpoint.ServerImpl.initDestination(ServerImpl.java:81) at org.apache.cxf.endpoint.ServerImpl.<init>(ServerImpl.java:64) at org.apache.cxf.jaxrs.JAXRSServerFactoryBean.create(JAXRSServerFactoryBean.java:170) ... 49 common frames omitted

org.eclipse.jetty.server.NCSARequestLog gets deprecated in Jetty 9.4.15.v20190215 and our build will fail if using deprecated classes.

This class is used by StartNflow.java. Need to replace it with CustomRequestLog and RequestLogWriter when upgrading jetty.

We are currently "abusing" nflow for some data import batch processing by splitting the work into several hundred child workflows that the cluster can then execute in parallel. The same cluster is also responsible for some higher priority work like sending emails.

Problem here is that if the batch processing takes e.g. 30 minutes total, nothing else will be executed in that time due to how next_activation is prioritized.

I've hacked around this with:

// schedule incrementally in the future, so execution won't lag badly
.setNextActivation(now.plusMillis(1000 * increment.getAndIncrement()))

, but some native way of prioritizing workflows relative to each other would be nice. In this case it would be enough to say "only execute this workflow when there's nothing else to execute", so that an email workflow with a later next_activation still gets executed before these batch workflows.

Willing to work on a PR if you outline something, and at least Esa Puttonen is familiar with the usecase.

Below this paragraph is an example (you won't be able to see it):

The image is above this paragraph.

Currently nflow-examples uses 5.7.0

When trying to execute the sample under nflow-examples/spring-boot/full-stack-kotlin with gradle bootRun the following error occurs on startup:

No workflow definition found for type [repeatingWorkflow]

Full stack trace:

         ___   _                    _  __        _     _   _
  _ _   | __| | |  ___  __ __ __   | |/ /  ___  | |_  | | (_)  _ _
 | ' \  | _|  | | / _ \ \ V  V /   | ' <  / _ \ |  _| | | | | | ' \
 |_||_| |_|   |_| \___/  \_/\_/    |_|\_\ \___/  \__| |_| |_| |_||_|
  :: Spring Boot ::                                (v0.0.1.SNAPSHOT)

2021-01-20 22:35:12.077  WARN 17333 --- [kground-preinit] o.s.h.c.j.Jackson2ObjectMapperBuilder    : For Jackson Kotlin classes support please add "com.fasterxml.jackson.module:jackson-module-kotlin" to the classpath
2021-01-20 22:35:13.568  WARN 17333 --- [           main] ConfigServletWebServerApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'workflowApplication': Invocation of init method failed; nested exception is java.lang.RuntimeException: No workflow definition found for type [repeatingWorkflow]
2021-01-20 22:35:13.590 ERROR 17333 --- [           main] o.s.boot.SpringApplication               : Application run failed

org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'workflowApplication': Invocation of init method failed; nested exception is java.lang.RuntimeException: No workflow definition found for type [repeatingWorkflow]
	at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:160) ~[spring-beans-5.2.1.RELEASE.jar:5.2.1.RELEASE]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:416) ~[spring-beans-5.2.1.RELEASE.jar:5.2.1.RELEASE]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1795) ~[spring-beans-5.2.1.RELEASE.jar:5.2.1.RELEASE]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:595) ~[spring-beans-5.2.1.RELEASE.jar:5.2.1.RELEASE]
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:517) ~[spring-beans-5.2.1.RELEASE.jar:5.2.1.RELEASE]
	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:323) ~[spring-beans-5.2.1.RELEASE.jar:5.2.1.RELEASE]
	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222) ~[spring-beans-5.2.1.RELEASE.jar:5.2.1.RELEASE]
	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:321) ~[spring-beans-5.2.1.RELEASE.jar:5.2.1.RELEASE]
	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202) ~[spring-beans-5.2.1.RELEASE.jar:5.2.1.RELEASE]
	at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:879) ~[spring-beans-5.2.1.RELEASE.jar:5.2.1.RELEASE]
	at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:878) ~[spring-context-5.2.1.RELEASE.jar:5.2.1.RELEASE]
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:550) ~[spring-context-5.2.1.RELEASE.jar:5.2.1.RELEASE]
	at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:141) ~[spring-boot-2.2.1.RELEASE.jar:2.2.1.RELEASE]
	at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:747) [spring-boot-2.2.1.RELEASE.jar:2.2.1.RELEASE]
	at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:397) [spring-boot-2.2.1.RELEASE.jar:2.2.1.RELEASE]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:315) [spring-boot-2.2.1.RELEASE.jar:2.2.1.RELEASE]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1226) [spring-boot-2.2.1.RELEASE.jar:2.2.1.RELEASE]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1215) [spring-boot-2.2.1.RELEASE.jar:2.2.1.RELEASE]
	at nflow.kotlin.NflowApplicationKt.main(NflowApplication.kt:42) [main/:na]
Caused by: java.lang.RuntimeException: No workflow definition found for type [repeatingWorkflow]
	at io.nflow.engine.internal.workflow.WorkflowInstancePreProcessor.process(WorkflowInstancePreProcessor.java:34) ~[nflow-engine-6.1.0.jar:na]
	at io.nflow.engine.service.WorkflowInstanceService.insertWorkflowInstance(WorkflowInstanceService.java:68) ~[nflow-engine-6.1.0.jar:na]
	at io.nflow.engine.service.WorkflowInstanceService$$FastClassBySpringCGLIB$$9f2681f6.invoke(<generated>) ~[nflow-engine-6.1.0.jar:na]
	at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) ~[spring-core-5.2.1.RELEASE.jar:5.2.1.RELEASE]
	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:685) ~[spring-aop-5.2.1.RELEASE.jar:5.2.1.RELEASE]
	at io.nflow.engine.service.WorkflowInstanceService$$EnhancerBySpringCGLIB$$8577c805.insertWorkflowInstance(<generated>) ~[nflow-engine-6.1.0.jar:na]
	at nflow.kotlin.WorkflowApplication.createExampleWorkflowInstance(NflowApplication.kt:30) ~[main/:na]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_231]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_231]
Caused by: java.lang.RuntimeException: No workflow definition found for type [repeatingWorkflow]

	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_231]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_231]
	at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:389) ~[spring-beans-5.2.1.RELEASE.jar:5.2.1.RELEASE]
	at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:333) ~[spring-beans-5.2.1.RELEASE.jar:5.2.1.RELEASE]
	at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:157) ~[spring-beans-5.2.1.RELEASE.jar:5.2.1.RELEASE]
	... 18 common frames omitted


> Task :bootRun FAILED

Execution failed for task ':bootRun'.
> Process 'command '/Library/Java/JavaVirtualMachines/jdk1.8.0_231.jdk/Contents/Home/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

I am trying to test my nflow based application by junit which is hosted as a rest api and i am getting below error logs.
We are using nflow 4.2.0 as of now.

org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'jaxRsServer' defined in class path resource [fi/eunet/conflow/NflowJettyConfigurationForConflow.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.apache.cxf.endpoint.Server]: Factory method 'jaxRsServer' threw exception; nested exception is org.apache.cxf.service.factory.ServiceConstructionException
at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:599)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1173)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1067)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:513)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)

List<Integer> children = instance.childWorkflows.computeIfAbsent(parentActionId, ArrayList::new);

This actually ends up calling new ArrayList(parentActionId), which causes ArrayLists with massive empty backing arrays relative in size to the age of the database.

Hi,

Could following line in NflowJettyConfiguration class

factory.setAddress('/' + factory.getAddress());

be changed to be configurable like:

factory.setAddress(env.getProperty("nflow.api.url"));

//Juha

Hello, how are you? I have a problem configuring application.properties to point to an external h2 database, I get this exception Class ""io.nflow.engine.internal.storage.db.H2ModifiedColumnTrigger". My application use h2 as primary database, i'm using version (7.2.0)

I have my application which was earlier running on nflow 4.0.0, Now i have upgrade this 4.2.0 as i need to use some of the current classes which is created in another application which is already running in 4.2.0.

While running the application first i got issue for column workflow_signal does not found in nflow_workflow table.
I have already below script in my nflow database.
https://github.com/NitorCreations/nflow/blob/master/nflow-engine/src/main/resources/scripts/db/update-4.0.0-x/mysql.update.ddl.sql

So i am stuck with an issue where while running my application it starts to give me error as per below screenshot.

Here are some logs

2020-01-31 13:13:35 INFO [nflow-dispatcher] al.executor.WorkflowDispatcher - [] Starting.
2020-01-31 13:13:35 ERROR [nflow-dispatcher] al.executor.WorkflowDispatcher - [] Exception in executing dispatcher - retrying after sleep period (PreparedStatementCallback; uncategorized SQLException for SQL [update nflow_workflow set executor_id = 525, status = 'executing', external_next_activation = null where id = ? and modified = ? and executor_id is null]; SQL state [HY000]; error code [4033]; (conn=2357999) Only row based replication supported for bulk operations; nested exception is java.sql.BatchUpdateException: (conn=2357999) Only row based replication supported for bulk operations)
org.springframework.jdbc.UncategorizedSQLException: PreparedStatementCallback; uncategorized SQLException for SQL [update nflow_workflow set executor_id = 525, status = 'executing', external_next_activation = null where id = ? and modified = ? and executor_id is null]; SQL state [HY000]; error code [4033]; (conn=2357999) Only row based replication supported for bulk operations; nested exception is java.sql.BatchUpdateException: (conn=2357999) Only row based replication supported for bulk operations
at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:84)
at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:81)
at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:81)
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:649)
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:662)
at org.springframework.jdbc.core.JdbcTemplate.batchUpdate(JdbcTemplate.java:950)
at org.springframework.jdbc.core.BatchUpdateUtils.executeBatchUpdate(BatchUpdateUtils.java:32)
at org.springframework.jdbc.core.JdbcTemplate.batchUpdate(JdbcTemplate.java:1000)

My current database version is
ysql Ver 15.1 Distrib 10.1.13-MariaDB,

I am using driver com.mysql.jdbc.Driver

I can try to upgrade my nflow application on latest version but i need to still analyze about the efforts needed and challenges.

It's not possible to create pure Kotlin workflows extending WorkflowDefinition because WorkflowState must be and enum. This creates a clash between name property in Kotlin enum and name() method in WorkflowState interface. Example of non-functioning code:

enum class State(private val _type: WorkflowStateType, private val _desc: String) : WorkflowState {
    repeat(WorkflowStateType.start, "Start state"),
    error(WorkflowStateType.manual, "Error state");

    override fun getDescription() = _desc
    override fun getType() = _type
}

If I use generics in Mutable StateVars:

      @StateVar(value = "hoplaa", readOnly = true) Mutable<Map<Integer, Integer>> hoplaa,

I get the following error in startup:

Caused by: java.lang.ClassCastException: class sun.reflect.generics.reflectiveObjects.ParameterizedTypeImpl cannot be cast to class java.lang.Class (sun.reflect.generics.reflectiveObjects.ParameterizedTypeImpl and java.lang.Class are in module java.base of loader 'bootstrap')
	at io.nflow.engine.internal.workflow.WorkflowDefinitionScanner.lambda$getStateMethods$0(WorkflowDefinitionScanner.java:72) ~[nflow-engine-7.4.0.jar:na]
	at org.springframework.util.ReflectionUtils.doWithMethods(ReflectionUtils.java:367) ~[spring-core-5.3.20.jar:5.3.20]
	at io.nflow.engine.internal.workflow.WorkflowDefinitionScanner.getStateMethods(WorkflowDefinitionScanner.java:56) ~[nflow-engine-7.4.0.jar:na]
	at io.nflow.engine.workflow.definition.AbstractWorkflowDefinition.<init>(AbstractWorkflowDefinition.java:55) ~[nflow-engine-7.4.0.jar:na]
	at io.nflow.engine.workflow.definition.AbstractWorkflowDefinition.<init>(AbstractWorkflowDefinition.java:40) ~[nflow-engine-7.4.0.jar:na]
	at io.nflow.engine.workflow.definition.WorkflowDefinition.<init>(WorkflowDefinition.java:37) ~[nflow-engine-7.4.0.jar:na]
	at fi.mybusiness.workflow.MyWorkflow.<init>(MyWorkflow.java:105) ~[main/:na]
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[na:na]
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[na:na]
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[na:na]
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490) ~[na:na]
	at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:211) ~[spring-beans-5.3.20.jar:5.3.20]
	... 65 common frames omitted

Observed using OpenJDK 11.0.15.

Hi, are you planning to add MSSQL support, and is there a chance that it will work now using one of other profiles?

Github legacy plans will not have access to GitHub Actions when it becomes generally available on November 13, 2019.

So maybe start using travis-ci?

• Exclude nflow-tests and nflow-perf-test modules
• Not mandatory to publish JUnit test reports in Github (just fail the build on errors)
• No need to keep the build artifacts

Hi!

The nflow explorer demo is not up.

/Mikael

When starting the nflow service in something other than local host the javascript keeps failing with get errors:

Possibly unhandled rejection: {"data":{"timestamp":"2022-01-05T21:38:13.451+00:00","status":404,"error":"Not Found","path":"/explorer/undefined/v1/workflow-executor"},"status":404,"config":{"method":"GET","transformRequest":[null],"transformResponse":[null],"jsonpCallbackParam":"callback","withCredentials":false,"url":"undefined/v1/workflow-executor","headers":{"Accept":"application/json, text/plain, /","X-XSRF-TOKEN":"d385c9e7-3c09-469a-b55f-b12dd12af0b2"}},"statusText":"","xhrStatus":"complete","resource":[]}

The undefined seems to indicate a failed expectation but the javascript is impossible to debug on the explorer:

ionConfigWebApplicationContext - [] Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanDefinitionStoreException: Failed to process import candidates for configuration class [io.nflow.jetty.config.NflowJettyConfiguration]; nested exception is org.springframework.context.annotation.ConflictingBeanDefinitionException: Annotation-specified bean name 'statisticsResource' for bean class [io.nflow.rest.v1.jaxrs.StatisticsResource] conflicts with existing, non-compatible bean definition of same name and class [io.nflow.rest.v1.springweb.StatisticsResource]

getting above error when trying to add nflow jetty server to a spring boot application. can someone help me here ?

Currently it is not possible to execute most of the database create scripts to empty database, because they contain commands that assume that some objects already exists.

For example
https://github.com/NitorCreations/nflow/blob/master/nflow-engine/src/main/resources/scripts/db/postgresql.create.ddl.sql#L39
contains this:

drop index nflow_workflow_activation;
create index nflow_workflow_activation on nflow_workflow(next_activation, modified);

The drop will fail if that index does not already exists.

The tests are executing these scripts but they explicitly ignore failed drop statements.
https://github.com/NitorCreations/nflow/blob/master/nflow-engine/src/test/java/io/nflow/engine/internal/dao/BaseDaoTest.java#L45

These scripts should be runnable as-is so that the end user doesn't have to modify them.

We've had this idea/request for showing something useful immediately when you land on Explorer:

Workflow instances should maybe show something initially instead of a blank page? E.g. the most recently modified workflows, to make it quicker to access those.

I'd be happy to implement it to nflow-explorer-ng, but the REST API does not support sorting by any field. There could be sorting for other workflow instance field as well.

org.springframework.dao.DataIntegrityViolationException: PreparedStatementCallback; SQL [update nflow_executor set active=current_timestamp, expires=date_add(current_timestamp, interval 900 second) where id = 90]; Data truncation: Incorrect datetime value: '2017-
03-26 03:14:38.000' for column 'expires' at row 1; nested exception is com.mysql.jdbc.MysqlDataTruncation: Data truncation: Incorrect datetime value: '2017-03-26 03:14:38.000' for column 'expires' at row 1
at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:102)
at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:73)
at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:81)
at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:81)
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:649)
at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:870)
at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:931)
at io.nflow.engine.internal.dao.ExecutorDao.updateWithPreparedStatement(ExecutorDao.java:157)
at io.nflow.engine.internal.dao.ExecutorDao.updateActiveTimestamp(ExecutorDao.java:151)
at io.nflow.engine.internal.dao.ExecutorDao.tick(ExecutorDao.java:92)

Hi there
Currently trying to do an implementation of nflow[7.4.0+] in Springboot
The example fullstack application seems to work however when trying to switch from h2 to postgresql I'm getting the following:

Consider defining a bean of type 'io.nflow.engine.internal.storage.db.SQLVariants' in your configuration.

My application.properties looks like this:

spring.profiles.active=nflow.db.postgres
logging.level.root=WARN

My nflow-engine.properties looks like this:

nflow.db.postgresql.driver=org.postgresql.Driver
nflow.db.postgresql.url=jdbc:postgresql://localhost/nflow
nflow.db.postgresql.user=postgres
nflow.db.postgresql.password=*PASS*

Is this a known issue or am I missing something?

EDIT: I am using java 11 if that means anything

Use old nflow-explorer as reference.

• show traversed path and current state in bold (workflow instance)
• select state from graph -> make incoming and outgoing edges bold
• select state from action history -> highlight the corresponding state from the graph