elasticio / webhook-component Goto Github PK

View Code? Open in Web Editor NEW

1.0 14.0 4.0 620 KB

Webhook component for the elastic.io platform

License: Apache License 2.0

JavaScript 100.00%

nodejs platform ipaas integration webhook elasticio

webhook-component's Introduction

webhook-component

General information
- Description
- Purpose
Credentials
Triggers
- Receive
Actions
- Send data
Known Limitations

Description

Purpose

An open source component for sending and receiving WebHooks on elastic.io platform.

Credentials

Webhook component supports the following authorisation types:

No Auth - use this method to work with any open REST API
Basic Auth - use it to provide login credentials like username/password
API Key Auth - use it to provide API Key to access the resource
HMAC verification shared secret - use it to verify via a shared secret

Triggers

Receive

Simple webhook trigger which receives data as an input and starts the flow execution after this.

Expected output metadata

Output schema

Example:

{
 "recievedBody": "recievedBody",
 "_query": {},
 "_headers": {
   "content-type": "application/json",
   "accept": "*/*",
   "accept-encoding": "gzip, deflate"
 },
 "_method": "POST",
 "_url": "/hook/5d691738cb5a286adc1e68e2"
}

Actions

Send data

Simply sends data it receives as an input to a URL provided.

WebHook action can also be used to troubleshoot many processes to see the outcome. For example one could create Invoices (in Salesforce) to Webhook flow and configure the Webhook with a url created in https://webhook.site or with any similar services.

List of Expected Config fields

[required] HTTP Verb
- POST. The WebHook component can POST information to preconfigured WebHook address. This action could be used for different purposes. For example WebHook can be used to inform your custom connector about an event which it waits to work.
- PUT. The WebHook component can also PUT a specific preconfigured JSON into specific address where the process will not be handled by the server. For this reason the "Output JSON Sample" field can be used.
[required] URI. This is the address to send WebHook.
[not required] Secret. This is an optional field to authenticate WebHook POST. There maybe cases when a special password or a secret might be required. For example the WebHook address was generated explicitly with a password so that to prevent any third parties to use it. This could be your specific WebHook address that you use to send your Wordpress posts into your server.

Expected output metadata

Output schema

Example:

{
   "recievedBody": "recievedBody",
   "_query": {},
   "_headers": {
     "content-type": "application/json",
     "accept": "*/*",
     "accept-encoding": "gzip, deflate"
   },
   "_method": "POST",
   "_url": "/hook/5d691738cb5a286adc1e68e2"
 }

Known limitations

Maximal possible size for an attachment is 10 MB.
Attachments mechanism does not work with Local Agent Installation

webhook-component's People

Contributors

Stargazers

Watchers

Forkers

sahas- flowground onlim

webhook-component's Issues

Cross origin error

Bug Report

Description

After executing the following query in browser's console

    cache: 'no-cache',
    credentials: 'include',
    headers: {
      'Authorization': 'Basic XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'
    },
    method: 'get',
    mode: 'cors',
    redirect: 'follow',
    referrer: 'no-referrer'
  };

  fetch('https://in.elastic.io/hook/5ea6a4c55d7ddd5e75eb4191', request);

it is throwing the following error message:

Access to fetch at 'https://in.elastic.io/hook/5ea6a4c55d7ddd5e75eb4191' from origin 'https://app.elastic.io' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.

in.elastic.io/hook/5ea6a4c55d7ddd5e75eb4191:1 Failed to load resource: net::ERR_FAILED
5b62c919fd98ea00112d52e7#/w/58dcfe0f52f01e0019e7484e:1 Uncaught (in promise) TypeError: Failed to fetch

Preconditions

Create a new flow.
Add Webhook component as trigger
Add http reply component
Add custom headers to http reply component
Access-Control-Allow-Origin: "*"
Access-Control-Allow-Methods: "GET"
Turn on 'real time' flow
Start the flow

Environment

Great Moraq (development)
Essos (stage)
Westeros (production)

Steps to Reproduce

Run the created flow.
Execute early mentioned query in browser's console with the authorization header as in Webhook step.
Observe the error message.

Actual Result

Access to fetch at 'https://in.elastic.io/hook/5ea6a4c55d7ddd5e75eb4191' from origin 'https://app.elastic.io' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.

in.elastic.io/hook/5ea6a4c55d7ddd5e75eb4191:1 Failed to load resource: net::ERR_FAILED
5b62c919fd98ea00112d52e7#/w/58dcfe0f52f01e0019e7484e:1 Uncaught (in promise) TypeError: Failed to fetch

Expected Result

Status code 200 OPTIONS

Update Sailor to 2.6.7

PR #34

HMAC for Webhooks not working. "The provided request signature does not match. Please check your signature key"

HMAC is generated by:

printf %s "$uri" | openssl dgst -sha256 -hmac "$secret"| sed 's/^.*= //'

SHA1, SHA512, SHA384 and MD5 any of these throws the message

Here is an example:

curl -X POST -H 'Content-type: application/json' -H "x-eio-signature: 9581ad63eb5d4da4e387059a1489f892e18025dd7fa5778569ee85c3202fbf11" --data '{"text":"Hello World!"}' https://in.elastic.io/hook/5d763a65f7ded16954750006
{"error":"The provided request signature does not match. Please check your signature key"}

Update the documentation in Readme

ATM: the README file contains very outdated and irrelevant information about this component. We have only two lines of useful information which is in the end.

The proposal is to write the documentation according to the current webhook component trigger and include all the nuances there.

Webhook should include addtional information in the call

Create the following flow:

Webhook: No Auth
Code Component: Default code

// Please note only Node.js code is supported here
async function run(msg) {
	console.log('Incoming message is %s', JSON.stringify(msg));
	const body = { result : 'Hello world!' };
	// You can emit as many data messages as required
	await this.emit('data', { body });
	console.log('Execution finished');
}

Publish and start the flow and then call the webhook with:
https://in.elastic.io/hook/5d25e4598370bfb1c7c4696a/Something?abc=def
Observe the incoming message in the code component. It is:

{
  "id": "c88fa000-a314-11e9-a435-dddac8c424fb",
  "attachments": {
    
  },
  "body": {
    "abc": "def",
    "_query": {
      "abc": "def"
    }
  },
  "headers": {
    "host": "in.elastic.io",
    "x-request-id": "c2ddbdbc20c95ff4518e065a18892e82",
    "x-real-ip": "88.130.157.228",
    "x-forwarded-for": "88.130.157.228",
    "x-forwarded-host": "in.elastic.io",
    "x-forwarded-port": "443",
    "x-forwarded-proto": "https",
    "x-original-uri": "\/hook\/5d25e4598370bfb1c7c4696a\/Something?abc=def",
    "x-scheme": "https",
    "upgrade-insecure-requests": "1",
    "user-agent": "Mozilla\/5.0 (X11; Linux x86_64) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/75.0.3770.100 Safari\/537.36",
    "accept": "text\/html,application\/xhtml+xml,application\/xml;q=0.9,image\/webp,image\/apng,*\/*;q=0.8,application\/signed-exchange;v=b3",
    "accept-encoding": "gzip, deflate, br",
    "accept-language": "en-US,en;q=0.9,de-DE;q=0.8,de;q=0.7,fr;q=0.6"
  },
  "metadata": {
    
  },
  "url": "\/hook\/5d25e4598370bfb1c7c4696a\/Something?abc=def",
  "method": "GET",
  "originalUrl": "\/hook\/5d25e4598370bfb1c7c4696a\/Something?abc=def",
  "query": {
    "abc": "def"
  },
  "params": {
    
  },
  "pathSuffix": "\/hook\/5d25e4598370bfb1c7c4696a\/Something",
  "taskId": "5d25e4598370bfb1c7c4696a",
  "passthrough": {
    "step_1": {
      "id": "c88fa000-a314-11e9-a435-dddac8c424fb",
      "attachments": {
        
      },
      "body": {
        "abc": "def",
        "_query": {
          "abc": "def"
        }
      },
      "headers": {
        "host": "in.elastic.io",
        "x-request-id": "c2ddbdbc20c95ff4518e065a18892e82",
        "x-real-ip": "88.130.157.228",
        "x-forwarded-for": "88.130.157.228",
        "x-forwarded-host": "in.elastic.io",
        "x-forwarded-port": "443",
        "x-forwarded-proto": "https",
        "x-original-uri": "\/hook\/5d25e4598370bfb1c7c4696a\/Something?abc=def",
        "x-scheme": "https",
        "upgrade-insecure-requests": "1",
        "user-agent": "Mozilla\/5.0 (X11; Linux x86_64) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/75.0.3770.100 Safari\/537.36",
        "accept": "text\/html,application\/xhtml+xml,application\/xml;q=0.9,image\/webp,image\/apng,*\/*;q=0.8,application\/signed-exchange;v=b3",
        "accept-encoding": "gzip, deflate, br",
        "accept-language": "en-US,en;q=0.9,de-DE;q=0.8,de;q=0.7,fr;q=0.6"
      },
      "metadata": {
        
      },
      "url": "\/hook\/5d25e4598370bfb1c7c4696a\/Something?abc=def",
      "method": "GET",
      "originalUrl": "\/hook\/5d25e4598370bfb1c7c4696a\/Something?abc=def",
      "query": {
        "abc": "def"
      },
      "params": {
        
      },
      "pathSuffix": "\/hook\/5d25e4598370bfb1c7c4696a\/Something",
      "taskId": "5d25e4598370bfb1c7c4696a"
    }
  }
}

Observe that

msg.headers contains all the HTTP headers sent in the request
msg.url contains the URL that was called, including additional data as part of the path
msg.method contains the HTTP verb of the call

Ideally, all of that should be added to the body of the message emitted from webhook so that it is instead:

{
    "abc": "def",
    "_query": {
      "abc": "def"
    },
    "_headers: {
        ...
    },
    "_method": "GET",
    "_url": "\/hook\/5d25e4598370bfb1c7c4696a\/Something?abc=def",
    "_additionalUrlPath": "\/Something"
}

[webhook-component] Make sure sensitive data is not logged in components

Component should be able to handle attachment information via the message body

Deprecate POST action

WebHook Action POST should be depricated and phased out. Please use REST API Client instead.

Naked arrays in request body erase meta-information.

Bug Report

Description

When the body of a webhook request has a naked array, then other information in the request is not included in the outbound message.

Steps to reproduce

Create a flow of the following form. Publish and start this flow.

Webhook to code component
No Auth for the webhook component
Use {} as sample data for both components.
Use the following code for code component:

// Please note only Node.js code is supported here
async function run(msg, cfg, snapshot) {
 this.logger.info('Incoming message is %s', JSON.stringify(msg));
 const body = { result : 'Hello world!' };
 // You can emit as many data messages as required
 await this.emit('data', { body });
 this.logger.info('Execution finished');
}

Send a POST request with a JSON body of the following two forms and observe the data that is logged by the code component.

{"foo":"bar"}
[{"foo":"bar"}]

Actual result

Object Case

``` { "id": "3b69ae10-832d-11ea-9e1e-dd9d4ea250fb", "attachments": {}, "body": { "foo": "bar", "_query": {}, "_headers": { "host": "in.elastic.io", "x-request-id": "6b26ff54e5f7ad104c7a59e8f97be954", "x-real-ip": "76.64.72.41", "x-forwarded-for": "76.64.72.41", "x-forwarded-host": "in.elastic.io", "x-forwarded-port": "443", "x-forwarded-proto": "https", "x-original-uri": "/hook/5e9dc879923c3668e3160de8", "x-scheme": "https", "content-length": "17", "content-type": "application/json", "user-agent": "PostmanRuntime/7.24.1", "accept": "*/*", "cache-control": "no-cache", "postman-token": "e8757d6f-1f45-4482-aa04-9a1176a142a8", "accept-encoding": "gzip, deflate, br" }, "_method": "POST", "_url": "/hook/5e9dc879923c3668e3160de8" }, "headers": { "host": "in.elastic.io", "x-request-id": "6b26ff54e5f7ad104c7a59e8f97be954", "x-real-ip": "76.64.72.41", "x-forwarded-for": "76.64.72.41", "x-forwarded-host": "in.elastic.io", "x-forwarded-port": "443", "x-forwarded-proto": "https", "x-original-uri": "/hook/5e9dc879923c3668e3160de8", "x-scheme": "https", "content-length": "17", "content-type": "application/json", "user-agent": "PostmanRuntime/7.24.1", "accept": "*/*", "cache-control": "no-cache", "postman-token": "e8757d6f-1f45-4482-aa04-9a1176a142a8", "accept-encoding": "gzip, deflate, br" }, "metadata": {}, "url": "/hook/5e9dc879923c3668e3160de8", "method": "POST", "originalUrl": "/hook/5e9dc879923c3668e3160de8", "query": {}, "params": {}, "pathSuffix": "/hook/5e9dc879923c3668e3160de8", "taskId": "5e9dc879923c3668e3160de8", "passthrough": { "step_1": { "id": "3b69ae10-832d-11ea-9e1e-dd9d4ea250fb", "attachments": {}, "body": { "foo": "bar", "_query": {}, "_headers": { "host": "in.elastic.io", "x-request-id": "6b26ff54e5f7ad104c7a59e8f97be954", "x-real-ip": "76.64.72.41", "x-forwarded-for": "76.64.72.41", "x-forwarded-host": "in.elastic.io", "x-forwarded-port": "443", "x-forwarded-proto": "https", "x-original-uri": "/hook/5e9dc879923c3668e3160de8", "x-scheme": "https", "content-length": "17", "content-type": "application/json", "user-agent": "PostmanRuntime/7.24.1", "accept": "*/*", "cache-control": "no-cache", "postman-token": "e8757d6f-1f45-4482-aa04-9a1176a142a8", "accept-encoding": "gzip, deflate, br" }, "_method": "POST", "_url": "/hook/5e9dc879923c3668e3160de8" }, "headers": { "host": "in.elastic.io", "x-request-id": "6b26ff54e5f7ad104c7a59e8f97be954", "x-real-ip": "76.64.72.41", "x-forwarded-for": "76.64.72.41", "x-forwarded-host": "in.elastic.io", "x-forwarded-port": "443", "x-forwarded-proto": "https", "x-original-uri": "/hook/5e9dc879923c3668e3160de8", "x-scheme": "https", "content-length": "17", "content-type": "application/json", "user-agent": "PostmanRuntime/7.24.1", "accept": "*/*", "cache-control": "no-cache", "postman-token": "e8757d6f-1f45-4482-aa04-9a1176a142a8", "accept-encoding": "gzip, deflate, br" }, "metadata": {}, "url": "/hook/5e9dc879923c3668e3160de8", "method": "POST", "originalUrl": "/hook/5e9dc879923c3668e3160de8", "query": {}, "params": {}, "pathSuffix": "/hook/5e9dc879923c3668e3160de8", "taskId": "5e9dc879923c3668e3160de8" } } } ```

Array Case

``` { "id": "65089880-833c-11ea-9e1e-dd9d4ea250fb", "attachments": {}, "body": [ { "foo": "bar" } ], "headers": { "host": "in.elastic.io", "x-request-id": "dabf832f68a8297957a4c3a841357253", "x-real-ip": "76.64.72.41", "x-forwarded-for": "76.64.72.41", "x-forwarded-host": "in.elastic.io", "x-forwarded-port": "443", "x-forwarded-proto": "https", "x-original-uri": "/hook/5e9dc879923c3668e3160de8", "x-scheme": "https", "content-length": "19", "content-type": "application/json", "user-agent": "PostmanRuntime/7.24.1", "accept": "*/*", "cache-control": "no-cache", "postman-token": "6e4220f2-3e54-4957-9809-d55b0e586988", "accept-encoding": "gzip, deflate, br" }, "metadata": {}, "url": "/hook/5e9dc879923c3668e3160de8", "method": "POST", "originalUrl": "/hook/5e9dc879923c3668e3160de8", "query": {}, "params": {}, "pathSuffix": "/hook/5e9dc879923c3668e3160de8", "taskId": "5e9dc879923c3668e3160de8", "passthrough": { "step_1": { "id": "65089880-833c-11ea-9e1e-dd9d4ea250fb", "attachments": {}, "body": [ { "foo": "bar" } ], "headers": { "host": "in.elastic.io", "x-request-id": "dabf832f68a8297957a4c3a841357253", "x-real-ip": "76.64.72.41", "x-forwarded-for": "76.64.72.41", "x-forwarded-host": "in.elastic.io", "x-forwarded-port": "443", "x-forwarded-proto": "https", "x-original-uri": "/hook/5e9dc879923c3668e3160de8", "x-scheme": "https", "content-length": "19", "content-type": "application/json", "user-agent": "PostmanRuntime/7.24.1", "accept": "*/*", "cache-control": "no-cache", "postman-token": "6e4220f2-3e54-4957-9809-d55b0e586988", "accept-encoding": "gzip, deflate, br" }, "metadata": {}, "url": "/hook/5e9dc879923c3668e3160de8", "method": "POST", "originalUrl": "/hook/5e9dc879923c3668e3160de8", "query": {}, "params": {}, "pathSuffix": "/hook/5e9dc879923c3668e3160de8", "taskId": "5e9dc879923c3668e3160de8" } } } ```

Observe the presence of the following fields that are present in the body of the message for the object case but not the array case:

_url
_method
_headers
_query

Expected results

It is possible to read URL, Method, Headers and Query from the body of the message even if the request body is an array.

Proposed Solution

Create a new (backwards incompatible) receive method and depricate the existing method. The message structure of the body of the new method should be:

{
  "body": [{"foo":"bar"}],
  "method": "POST",
  "query": {},
  "attachments": {},
  "url": "....",
  "headers": {...}
}

Wipe webhook credentials from messages that are passed into the flow

Currently, if you create a flow where the first component is a webhook component with authentication all received headers are passed to the next step in the flow (in the body). Ideally, the authentication headers should be removed. Ex:

If authentication type is No Auth -> Nothing to do
If authentication type is Basic Auth -> Remove Authorization header
If authentication type is API Key auth -> Remove the specific key from the header
If authentication type is HMAC -> remove whatever header contains the HMAC value.

HTTP Post with XML file as attachment is not working

STR:

Create flow with Webhook-trigger
Start flow and send request with e.g. pdf file and Content type multipart/form-data

AR:

{
    "error": "getaddrinfo ENOTFOUND steward.marathon.slave.mesos-app steward.marathon.slave.mesos-app:8200"
}

ER: The reply is

{
    "requestId": "b3eb7e50eef7171caaa7a8481d152d8e",
    "message": "thank you"
}

Hide data in logs

Emitting the data to logs is useful for debug and certain use caes, however there are some customers or scenarios when this is not allowed.

Request to add an option to "Suppress logging data" checkbox that would prevent the console.log action.

BUG: I can't choose Format type and help text isn't displayed since version f061c1542b73565a2ee32a6ec95f2d4d64dce387

STR:

Set Webhook component as trigger
Choose latest version
Click Add sample manually button
AR:

ER:

New step-builder UI inconsistency: Empty mapping field

After UI update we have an inconsistency in step-builder. It shows an empty Mapping section:

elasticio / webhook-component Goto Github PK

webhook-component's Introduction

webhook-component

Table of Contents

Description

Purpose

Credentials

Triggers

Receive

Expected output metadata

Actions

Send data

List of Expected Config fields

Expected output metadata

Known limitations

webhook-component's People

Contributors

Stargazers

Watchers

Forkers

webhook-component's Issues

Bug Report

Description

Preconditions

Environment

Steps to Reproduce

Actual Result

Expected Result

Bug Report

Description

Steps to reproduce

Actual result

Expected results

Proposed Solution

Recommend Projects

Recommend Topics

Recommend Org