Service description

• name: Webhook Remote Trigger
• description: Triggers a remote Webhook.
• type: action
• provider name: HTTP Service Provider
• step config options:
  • ...

• output

Further requirements

• testing the step configuration

  • preconditions

  • test success criteria

Steps to reproduce:

1. Create a new flow
2. Deploy the flow
3. Trigger the flow and immediately afterwards view flow logs

Service description

• name: New Item in Feed
• description: triggers on new RSS and Atom feed items/entries
• type: trigger
• provider name: Feed Service Provider
• step config options:
  • Feed URL (required).
    • Note that feed format should not need to be set explicitly. If both RSS and Atom are offered in one feed, pick RSS per default for downstream processing.
  • Refresh interval (optional). Input given in minutes. Defaults to 15.
  • Note that in future the triggering event will need to be configurable and that password protected feeds will need to be supported.
• output (see e.g. Zapier RSS 'New Item in Feed')
  • output every XML child-element of item as parsed content
  • output every XML child-element of item as raw content (prefix element name with raw_)
  • output every attribute of every XML child-element of item as raw content (prefix property name with raw_element_name_)

Further requirements

• ‚new‘ is defined per default as having different GUID/ID/URL. Note that in future triggering events will need to be configurable (e.g. also supporting triggering on different content, triggering by any change in any property, triggering by detected custom value)
• needs to support RSS 1.0, RSS 2.0 and Atom 1.0. Should detect format implicitly.
• keep in mind that in future ‚New Items in Multiple Feeds‘ will need to be supported either as separate service of type trigger, or as config parameter to this service.
• testing the step configuration
  • preconditions
    • at least one recent item in feed
    • at least one item matches selected trigger options (default: different GUID/ID/URL)
  • test success criteria
    • at least one item can be fetched from the feed and parsed successfully
  • sample to display: display raw internal representation (i.e. with all field names that can be used downstream) of parsed feed item

• get all runs from all flowRuns from the given flow
• optionally filter by status
• paginate results
• get total count of (filtered) results

add configSchema to services and a step configParams validator based on these configSchemas.
e.g.

service.configSchema = {
  '1st_line_header': {
     position: 0,
     label: '1st Line Header',
     defaultValue: false,
     type: integer,
     rules: [
       {type: MAX, value: 100, required: true},
       {type: NOT_NEGATIVE}
     ]
   }
}

step.configParams = {
  '1st_line_header': { value: 20 },
}

The taskInitializer should check the run state! If there's an error then the task shouldn't be created and the stateMachine shouldn't stopped.

• write flowRun output to S3
• return output

http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/DynamoDB/DocumentClient.html

to get the metadata and detailed logs of one flow run

getFlowRun(flowId, userId) { logs } 
=> flowRun { logs[] }

When deploying to Dev or Test stage all the old data should be removed and seeded again.

• ASL Generator
• Create StateMachine on createFlowRun
• Start StateMachine on startFlowRun (and createAndStartFlowRun)
• Delete StateMachine on deleteFlowRun

• generate positions upon adding a new step
• regenerate positions upon removing a step

Service description

• name: Act on S3 Object
• description: Creates, modifies or deletes an object in an S3 bucket.
• type: action
• provider name: S3 Service Provider
• step config options:
  • ...

• output

Further requirements

• testing the step configuration

  • preconditions

  • test success criteria

which resolves FlowRun table (#16)

Service description

• name: Write CSV
• description: Serializes CSV formatted content payload from JSON. Note: This service should be able to serialize CSV to the following renditions: CSV string and CSV file (file handle/pointer to temp local storage is passed to a subsequent step).
• type: action
• provider name: CSV Service Provider

• step config options:

• output

Further requirements

• testing the step configuration

  • preconditions

  • test success criteria

Service description

• name: Monitor S3 Bucket
• description: Detects a new, modified or deleted object in an S3 bucket.
• type: trigger
• provider name: S3 Service Provider
• step config options:
  • ...

• output

Further requirements

• testing the step configuration

  • preconditions

  • test success criteria

• Add a ReviewTaskService to the Service-registry
• the Task initializer should create a task instance with the type type REVIEW instead of APPROVE

extend servies with lambda arn

Service description

• name: Read JSON
• description: Parses JSON formatted content payload and extracts data based on JSONPath to a desired JSON structure.
• type: action
• provider name: JSON Service Provider
• step config options:
  • ...

• output

Further requirements

• testing the step configuration

  • preconditions

  • test success criteria

In order to consume RSS feeds or similar it should be possible to execute a TriggerService function in a given interval. E.g. all 10 minutes.
To do this we need to create a CloudWatch-Event that calls the TriggerService function in a configured time interval with a configured payload.

• add CloudWatch-Event rule creator for Scheduled-Trigger-Services
• enhance service-registry to mark scheduled and manual triggers
• remove events and lambda event permissions when deleting a flowRun
• add the pissibility to activate and deactivate events (when deactivating/activating flows)

Service description

• name: Read Text File
• description: Parses text file.
• type: action
• provider name: File Service Provider
• step config options:
  • ...

• output

Further requirements

• testing the step configuration

  • preconditions

  • test success criteria

Specs

• Should set flow.draft=true upon creating a new flow
• Should set flow.draft=true upon mutating a flow e.g. add a step
• Should set flow.draft=false upon creating a new flowRun
• I can discard a flow draft and restore the previous flow model from the last flowRun
• Should set flow.draft=false upon discarding a flow draft

related to #35

Add a Service that pushes the flow output to a configurable S3 Bucket

• Related to Issue #72
• Pull-Request #73

Runs metadata should being persisted in DynamoDB. Only the result of the run should being saved in the S3 Bucket.

• save runs metadata in dynamoDB (flowRun table)
• improve performance of getRunsForFlow resolver. use dynamoDB query or scan

A FlowRun instance contains all the required metadata for a flow execution + a complete copy of the flow which created the FlowRun instance.
besides that it contains detailed logs about the execution. this includes the current status of the flow run and the logs from each steps within the flow run.
e.g.

logs: {
	status: 'running' || 'error' || 'success'
	message: String,
	steps: [
		{
			stepId: String,
			status: 'error' || 'success'
			message: String,
		}
	]	
}

Service description

Note: might be merged with #59

• name: Write JSON
• description: Serializes JSON formatted content payload and extracts data based on JSONPath to a desired JSON structure as string.
• type: action
• provider name: JSON Service Provider
• step config options:
  • ...

• output

Further requirements

• testing the step configuration

  • preconditions

  • test success criteria

Service description

• name: Create Item in Feed
• description: makes available new items in a feed under a customizable url
• type: action
• provider name: Feed Service Provider
• step config options:
  • Feed URL (required). Note that only part of the path can be customized.
    • A path to the feed looks like this: /flows/[flow_id]/steps/[step_id]/out/[rss|atom]/[configurable_string].
    • The path segment /flows/[flow_id]/steps/[step_id]/out/ cannot be configured.
  • Feed format: RSS 2.0 (default) or Atom 1.0 can be configured
  • Feed title
  • Max items (default is 10)
  • Item title (required)
  • Source URL (required)
  • Content (required)
  • PupDate
• Important: Note that field content can be set to be coming from any upstream steps properties and can be chained using any or none characters in between or as pre- or postfixes.
• output
  • Standard conformant feed containing data from the configured upstream properties.

Further requirements

• ‚new‘ is defined by upstream events passing data to step
• testing the step configuration
  • preconditions
    • at least one sample item containg data for all required fields. Will be offered automatically based on available data in flow (last five incoming events). Can be created manually using form fields from step config.
  • test success criteria
    • item in feed can be created successfully based on available data
  • sample to display: display raw generated output.

The API should handle the following user errors:

• Flow – Access Denied
• Flow – Not found
• Task – Access Denied
• Task – Not found

Prototype https://gist.github.com/johipsum/a9b1a998e24f10ab9308e7a49422eb5b (deprecated)

• userPool google integration
• secure graphql api endpoint
• restrict access to flow instances
• restrict access to task instances

Service description

• name: Webhook Trigger
• description: Provides an HTTP endpoint for triggering a Flow.
• type: trigger
• provider name: HTTP Service Provider
• step config options:
  • ...

• output

Further requirements

• testing the step configuration

  • preconditions

  • test success criteria

Service description

• name: Only continue if...
• description: Define rules to specify when this Flow can continue running.
• type: action
• provider name: Filter Service Provider
• step config options:
  • rule definition consisting of
    • setting the field/property to which the condition is to be applied
    • setting a condition type (select list of options)
    • setting a condition definition (i.e. a string, number, date). This setting is optional and is available only if it makes sense (i.e. not for boolean or existential type of conditions)
  • rules can be chained using boolean operators (AND or OR)
  • only the following structure is permitted: one OR clause can contain multiple AND clauses, i.e. conditions can be nested only one level deep, e.g. cond1 AND cond2 AND cond3 OR cond4 AND cond5 OR cond6
  • negations are given as condition type options. There is no explicit NOT operator available.
• output
  • the filtered data as consumed

Further requirements

• Conditions match for Text, Numbers, Datetime, Boolean and Existence and their negations in the case of testing for the existence and Text
  • raw string matching options: REGEX
  • text matching options: containment matches, exact matches, prefix and suffix string matches, matches against set of strings
  • datetime matching options: Equality, before and after
  • numbers matching options: comparison operators: Equality, gt, lt
• testing the step configuration: none

Service description

• name: Read CSV
• description: Parses CSV formatted content payload to JSON. Note: This service should be able to parse CSV in the following renditions: CSV string, CSV stream, CSV url and from CSV file (file handle/pointer is to temp local storage is passed in from a previous step).
• type: action
• provider name: CSV Service Provider
• step config options:
  • separator (string; required; default is ',')
  • include header? (boolean; required; default is 'false')

• output

Further requirements

• testing the step configuration

  • preconditions

  • test success criteria

It should be possible to Activate and Deactivate Flows. Deactivated Flows are not startable.

• Add a active flag to the Flows model
• enable / disable the cloudwatch event from the scheduled trigger when activating / disabling a flow

Description

Expected Behavior

Given a correctly configured flow with the steps

1. URL Trigger (with valid URL)
2. Fetch Article

When I trigger a flow run, then the run should get the status "success".

Actual Behavior

The run remains in status "running".

Possible Fix

Steps to Reproduce

Context

Your Environment

• Environment used: Testing
• Browser Name and version: Chrome (latest)
• Operating System and version (desktop or mobile): macOS High Sierra

a flow needs to contain a status to inform the user if the last flow run was successful or not

• dynDB model
• graphQL resolver