Code Monkey home page Code Monkey logo

matrix-webhook-gateway's Introduction

matrix-webhook-gateway lets you create and manage webhooks for Matrix channels. The application is configured with a simple and clearly documented configuration file, and webhooks can be added and removed directly in Matrix. A powerful plugin system is available (including several examples), allowing you to easily add handlers for various types of webhooks.

Installation

Docker

The recommended way to run the application service is using Docker. All releases are published as Docker images on https://hub.docker.com/r/geluk/matrix-webhook-gateway.

To get started quickly, an example compose file is provided below:

version: "3.6"
services:
  matrix-webhooks:
    image: geluk/matrix-webhook-gateway:latest
    # Default settings shown here for reference:
    # environment:
    #   WEBHOOK_CONFIG: /config/gateway-config.yaml
    #   WEBHOOK_APPSERVICE_CONFIG: /data/appservice-webhook-gateway.yaml
    volumes:
      # Dynamic data, the application will write to this directory
      - ./data:/data
      # Static configuration, can be mounted read-only
      - ./config:/config:ro
    ports:
        # This port is used to listen for incoming webhooks.
      - 8020:8020
        # This port is used for communication with your homeserver.
      - 8023:8023
    restart: unless-stopped
    # Defaults shown below, these can be uncommented and edited if required:
    #user: 953:953

Without Docker

It is also possible to run the application directly. Download the latest release and extract it. Start the application by running node entry.js Commandline options can be used to alter the default configuration file location. For a list of available options, try node entry.js --help.

Configuration

A new configuration file (/config/gateway-config.yaml in Docker, or ./gateway-config.yaml outside Docker) is generated on first startup. The webhook gateway will use the settings in this file to automatically generate an appservice registration file (default location: /data/appservice-webhook-gateway.yaml in Docker, ./appservice.yaml outside). You should copy this file to your Matrix server, and add it to your Synapse configuration:

app_service_config_files:
  - './appservices/appservice-webhook-gateway.yaml'

If your webhook and Matrix server live on the same host, you can also choose to directly point Matrix to the generated appservice file.

Usage

Set up a webhook

Invite the bot to a new channel:

/invite @_webhook:yourmatrixserver.tld

To create a webhook in the channel, enter the following command:

-hook create <webhook_name>

The bot will create a new user for this webhook, invite it, and send you the webhook URL in a private message.

To list all enabled webhooks in a channel, use:

-hook list

To delete a webhook, use:

-hook delete <id>

Where <id> is the numeric ID of a webhook as returned by -hook list.

A webhook user will automatically update its username and avatar if the relevant fields in the webhook are set. Alternatively, you can use commands to set the profile details of a user.

-hook set name <id> <username>

or:

-hook set avatar <id> <url>

Call the webhook

To call the webhook, send a POST request with a JSON body to the webhook URL. Several formats are supported, such as Slack:

{
  "text": "I am a webhook message",
  "username": "Webhook",
  "icon_url": "https://example.com/hook.png"
}

Discord:

{
  "content": "I am a webhook message",
  "username": "Webhook",
  "icon_url": "https://example.com/hook.png"
}

Apprise:

{
  "version": "1.0",
  "title": "A new message",
  "message": "I am a webhook message"
}

As well as the format exposed by turt2live/matrix-appservice-webhooks:

{
  "text": "Hello world!",
  "format": "plain",
  "displayName": "My Cool Webhook",
  "avatarUrl": "http://i.imgur.com/IDOBtEJ.png"
}

Other formats can be supported by writing a custom plugin, see below.

Webhook user management

When a new webhook is created, the gateway will generate an ID for the webhook user using the pattern specified in app_service.user_pattern in the configuration file.

It is possible to create multiple separate webhooks with the same name in the same channel. Those webhooks will all be handled by the same user.

If you remove the {room} variable from the user_pattern, webhooks with the same name will map to the same user everywhere, which may lead to a single webhook user being active in multiple rooms at once. Despite this, webhooks will still only be posted to the room in which they were created.

Plugins

To support arbitrary JSON POSTs, you can write a custom plugin to interpret the message body and format it into something the webhook gateway understands.

A plugin is a Typescript file dropped into /data/plugins (or ./plugins if you don't use the Docker release), which needs to conform to a specific format.

The format of a plugin file is best explained by looking at an example, which you can find here.

this plugin may also be interesting to look at. It generates a message from an alert notification as sent by Prometheus Alertmanager.

Several plugins are available in the plugin/develop directory. Note that these plugins are intended as a starting point and are usually based on the default configuration for the application they support.

To execute a plugin, you must append its name (as specified by the format key in your plugin definition) to the webhook URL. For instance, to execute the sample plugin, send a POST to https://your-webhook-gateway-url/hook/your-webhook/sample

Writing plugins

If you're just looking to write a simple plugin, the easiest way to get started is by copying the sample plugin, adapting it to your needs, and installing it in the right directory.

If you're writing a more complicated plugin and would like to have access to code analysis, you can clone the repository and write your plugin in the ./plugins/develop directory. This way, features like type hints and automatic imports will work correctly.

Formatting API

To simplify the process of generating messages with both HTML and plaintext content, a formatting API is available. This lets you build messages that will be rendered both in plaintext and as HTML.

For example, creating an ordered list:

ol(a('https://example.com', 'first'), 'second', 'third');

This generates the following plaintext:

1. first (https://example.com)
2. second
3. third

While generating the following HTML content:

  1. first
  2. second
  3. third

Various convenience functions are available. Take a look at the sample plugins to see how the formatting API can be used, and see here for a complete list of all available formatting functions.

Plugin loading process

The loading process of plugins is a bit involved, and not necessarily optimal. In other words, I'm open to suggestions! If you know of a way to streamline it, please do create an issue or a pull request.

Here's how it currently works:

On startup, the directory specified in the configuration file (webhooks.plugin_directory) is scanned recursively, and the plugins in it are loaded one by one. The loading process of a plugin looks like this:

  1. The contents of the plugin file are hashed.
  2. webhooks.plugin_cache_directory is checked for a file with a matching hash. If a file is found, the cached plugin is loaded immediately (see step 5).
  3. If no file is found, the plugin is copied to ./plugins/__workdir and compiled. This path needs to be within the working directory of the application, otherwise type resolutions during compilation will fail.
  4. The compiled plugin is written to webhooks.plugin_cache_directory.
  5. The cached plugin is copied to ./plugins/__workdir to ensure any imports are resolved correctly during the loading process.
  6. The plugin is require()d and some basic integrity checks are performed against the resulting object to validate that it is correctly formed.
  7. The plugin class is instantiated, the init() function is executed, and the plugin is added to the list of active plugins.

Development setup

Run start-matrix.sh (start-matrix.ps on Windows) in ./local-dev to set up a local appservice development environment with Docker. This will start up a Synapse and an Element instance, which you can use for testing the appservice.

cd local-dev
./start-matrix.sh

Go to http://localhost:8009 and log in using the following credentials:

Username: dev Password: appservice-dev

Start the application service:

npm ci
npm run start

Invite the webhook management user using the following Matrix command:

/invite @webhook:matrix.local

Troubleshooting:

  • If the appservice shows an ECONNREFUSED error, it cannot connect to the Synapse server. Make sure Synapse is accessible at http://127.0.0.1:8008 or modify the configuration file to point it to the right URL.
  • If the appservice starts successfully, but doesn't respond to invitations, Synapse can't reach the appservice. Make sure the appservice can be reached from the Synapse container. You can test this by executing the following command within the local-dev directory: docker-compose exec synapse curl 172.31.0.1:8023/health On Windows, if you're using Docker with the WSL backend, you may need to update the IP address in local-dev/synapse-config/appservices/webhook-dev.yml from http://172.31.0.1:8023 (the default gateway of the Docker network) to your computer's IP address.

References

  • apprise, a platform-independent notification framework.
  • turt2live, whose project inspired me to develop this application.

Useful documentation

matrix-webhook-gateway's People

Contributors

geluk avatar snyk-bot avatar wfdewith avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

matrix-webhook-gateway's Issues

Plugin formatting fuctionn ol() doesn't work

Hi,

first of all: thanks for this amazing webhook gateway. The plugin building is great.
No more E-Mail notifications about my infrastructure and or app updates. Thanks for this!

Today I try the ol() function to create bullet point list but the log shows me this error:

2021-09-29 22:32:00.617.000         ERROR         [webhook-srv]                    Failed to handle webhook invocation:
Error         entries.map is not a function

Thanks

Gitea Plugin

Hello,

I tried the gitea.ts plugin on my hosted gitea but webhook-gateway shows the error:
2021-10-16 10:00:46.810.000 INFO [plg-gitea] Unknown Gitea action: 'undefined'

gitea config:
image

last webhook:
image

do you need more informations to debug?

regards

TypeError since last Node.js v18.17.1

Hello,

since the last node update I got this error message:

root@COM:/opt/webhook-gateway# node entry.js
/opt/webhook-gateway/node_modules/ttypescript/lib/loadTypescript.js:13
        function __() { this.constructor = d; }
                                         ^

TypeError: Cannot set property constructor of [object Object] which has only a getter
    at new __ (/opt/webhook-gateway/node_modules/ttypescript/lib/loadTypescript.js:13:42)
    at __extends (/opt/webhook-gateway/node_modules/ttypescript/lib/loadTypescript.js:14:84)
    at /opt/webhook-gateway/node_modules/ttypescript/lib/loadTypescript.js:65:5
    at Object.<anonymous> (/opt/webhook-gateway/node_modules/ttypescript/lib/loadTypescript.js:100:2)
    at Module._compile (node:internal/modules/cjs/loader:1256:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1310:10)
    at Module.load (node:internal/modules/cjs/loader:1119:32)
    at Module._load (node:internal/modules/cjs/loader:960:12)
    at Module.require (node:internal/modules/cjs/loader:1143:19)
    at require (node:internal/modules/cjs/helpers:121:18)

Node.js v18.17.1

webhook hangs at webserver "Received an unrecognised webhook"

Hello
hope I'm not overseeing something.
I setup the gateway without docker. After starting it, I'm getting a warning - which might be the problem - but I'm not sure:

2021-10-28 20:17:48.577.000 WARN [bridge] [-] POST http://127.0.0.1:8008/_matrix/client/r0/register (AS) HTTP 400 Error: "{\"errcode\":\"M_USER_IN_USE\",\"error\":\"User ID already taken.\"}"

I also took a look at the homeserver.log, unfortunately there's just the same error. So I'm unsure whether this is critical.

Webhook creation worked fine in Matrix, but if I want to send some test messages, e.g.

 curl -X POST 127.0.0.1:8020/hook/xxxxx -H "Content-Type: application/json" -d '{"Test":"test"}'

I'm seeing this output and the hook is not being passed to the room:

2021-10-28 20:25:20.570.000      WARN   [webhook-srv]           Received an unrecognised webhook:
{
 Test: 'test'
}

Would be nice if you have some idea about that.
Thanks!

Object being shown in apprise format

Hello, im looking at the features of different webhook formats and noticed that using apprise with any valid "type" in the request the bot prints [Object object] in the first line of the message

stop working after Matrix v1.32.2 update

Hello,

after I've updated matrix to v1.32.2 my matrix-webhook-gateway doesn't work anymore

npm run start

> [email protected] start /opt/matrix-webhook-gateway
> ts-node --compiler ttypescript entry.ts -v

2021-04-26 13:40:39.945.000      DEBUG  [webhook-srv]           Loading configuration file: './gateway-config.yaml'

2021-04-26 13:40:39.981.000      DEBUG  [webhook-srv]           Generating appservice.yaml
2021-04-26 13:40:39.986.000      DEBUG  [webhook-srv]           Creating DB connection with sqlite3
2021-04-26 13:40:40.99.000       DEBUG  [webhook-srv]           There are no pending migrations
2021-04-26 13:40:40.116.000      INFO   [webhook-srv]           Matrix bridge running on 127.0.0.1:8023
2021-04-26 13:40:40.145.000      WARN   [bridge]                [-] POST http://127.0.0.1:8008/_matrix/client/r0/register (AS) HTTP 400 Error: "{\"errcode\":\"M_UNKNOWN\",\"error\":\"An access token should not be provided on requests to /register (except if type is m.login.application_service)\"}"

 M_UNKNOWN      An access token should not be provided on requests to /register (except if type is m.login.application_service)

systemd

[Unit]
Description=Matrix-Webhook daemon
After=matrix-synapse.service
BindsTo=matrix-synapse.service

[Service]
Type=simple
WorkingDirectory=/opt/matrix-webhook-gateway
ExecStart=npm run start
Restart=always

[Install]
WantedBy=matrix.target

Prometheus webhooks not working with prometheus.ts plugin

Hello,

the prometheus.ts plugin is not working with my test alert rule send via alert manager.
alert rule in prometheus:

groups:
- name: test

  rules:
  - alert: TEST ALERT FROM PROMETHEUS PLEASE ACKNOWLEDGE
    expr: prometheus_build_info{instance="localhost:9090"} == 1
    for: 10s
    labels:
      severity: warning
    annotations:
      action: TESTING PLEASE ACKNOWLEDGE, NO FURTHER ACTION REQUIRED ONLY A TEST
      description: TEST ALERT FROM {{ $labels.instance }}

matrix-webhook-gateway.service

Feb 02 23:25:13 COM node[1102626]: 2022-02-02 23:25:13.264  INFO [webhook-srv] Web server running on 0.0.0.0:8020
Feb 02 23:34:29 COM node[1102626]: 2022-02-02 23:34:29.592  ERROR [plg-prometheus] MatrixHttpClient (REQ-5)
Feb 02 23:34:29 COM node[1102626]: {
Feb 02 23:34:29 COM node[1102626]:   errcode: 'M_USER_IN_USE',
Feb 02 23:34:29 COM node[1102626]:   error: 'User ID already taken.'
Feb 02 23:34:29 COM node[1102626]: }
Feb 02 23:34:29 COM node[1102626]: 2022-02-02 23:34:29.593  ERROR [plg-prometheus] Appservice Error registering user: User ID is in use
Feb 02 23:34:29 COM node[1102626]: 2022-02-02 23:34:29.836  ERROR [webhook-srv] Failed to handle webhook invocation:
Feb 02 23:34:29 COM node[1102626]:  TypeError  Cannot read properties of undefined (reading 'formatPlain')
Feb 02 23:34:29 COM node[1102626]: error stack:
Feb 02 23:34:29 COM node[1102626]: • formatting.ts:74 toPlain
Feb 02 23:34:29 COM node[1102626]:     src/formatting/formatting.ts:74:24
Feb 02 23:34:29 COM node[1102626]: • formatting.ts:92 formatPlain
Feb 02 23:34:29 COM node[1102626]:     src/formatting/formatting.ts:92:30
Feb 02 23:34:29 COM node[1102626]: • formatting.ts:74 toPlain
Feb 02 23:34:29 COM node[1102626]:     src/formatting/formatting.ts:74:25
Feb 02 23:34:29 COM node[1102626]: • formatting.ts:361 <anonymous>
Feb 02 23:34:29 COM node[1102626]:     src/formatting/formatting.ts:361:49
Feb 02 23:34:29 COM node[1102626]: • formatting.ts:361 formatPlain
Feb 02 23:34:29 COM node[1102626]:     src/formatting/formatting.ts:361:32
Feb 02 23:34:29 COM node[1102626]: • formatting.ts:74 toPlain
Feb 02 23:34:29 COM node[1102626]:     src/formatting/formatting.ts:74:25
Feb 02 23:34:29 COM node[1102626]: • formatting.ts:92 formatPlain
Feb 02 23:34:29 COM node[1102626]:     src/formatting/formatting.ts:92:30
Feb 03 03:12:31 COM node[1102626]: 2022-02-03 03:12:31.817  ERROR [plg-prometheus] MatrixHttpClient (REQ-8)
Feb 03 03:12:31 COM node[1102626]: {
Feb 03 03:12:31 COM node[1102626]:   errcode: 'M_USER_IN_USE',
Feb 03 03:12:31 COM node[1102626]:   error: 'User ID already taken.'
Feb 03 03:12:31 COM node[1102626]: }

Is there specification for the alert rules to work with the prometheus plugin?

starting from docker fails: connect ECONNREFUSED 127.0.0.1:8008

Hello
I am just trying to move from "normal" installation to the docker one, while learing about the docker structure in parallel.
From my understanding the docker compose file is only for reference if you want to setup multiple docker containers containing this webhook gateway?

So basically I just installed docker and the dependencies, pulled the webhook gateway and tried to start it - unfortunately I'm getting this error and have no idea what could be the problem.
I'd be really appreciated If you have a minute and help me out a little, here is the error:

https://pastebin.com/EVxtH6E1

Best regards

Cannot read properties of undefined (reading 'split')

Hello
Since some days that nice plugin doesn't start anymore.
I'm getting this error:

/home/user/scripts/webhook-gateway/node_modules/ttypescript/lib/loadTypescript.js:29
    var _e = ts.versionMajorMinor.split('.'), major = _e[0], minor = _e[1];
                                  ^

TypeError: Cannot read properties of undefined (reading 'split')
    at Object.loadTypeScript (/home/user/scripts/webhook-gateway/node_modules/ttypescript/lib/loadTypescript.js:29:35)
    at Object.<anonymous> (/home/user/scripts/webhook-gateway/node_modules/ttypescript/lib/typescript.js:3:27)
    at Module._compile (node:internal/modules/cjs/loader:1155:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1209:10)
    at Module.load (node:internal/modules/cjs/loader:1033:32)
    at Function.Module._load (node:internal/modules/cjs/loader:868:12)
    at Module.require (node:internal/modules/cjs/loader:1057:19)
    at require (node:internal/modules/cjs/helpers:103:18)
    at Object.<anonymous> (/home/user/scripts/webhook-gateway/src/webhooks/PluginCollection.js:61:26)
    at Module._compile (node:internal/modules/cjs/loader:1155:14)

Anybody got an idea how to fix that?

Retrieve existing hook URLs

There should be a command to retrieve URLs of existing webhooks. I propose something like: -hook listfull, which will send a PM with all hooks in a room, if the user is allowed to see them.

Running in docker - HTML formating

I have running wegnook-gateway in docker. All works, but HTML formating no. If I try:

curl -k -X POST -H 'Content-type: application/json' --data '{"text":"<b>Hello world !</b>","format":"html"}' URL

Then I have message:

<b>Hello world !</b>

Is it some issue with HTML formating?

It was working in older webhook bridge from turt2live.

HTML formating in docker

I have running wegnook-gateway in docker. All works, but HTML formating no. If I try:

curl -k -X POST -H 'Content-type: application/json' --data '{"text":"<b>Hello world !</b>","format":"html"}' URL

Then I have message:

<b>Hello world !</b>

Is it some issue with HTML formating?

It was working in older webhook bridge from turt2live.

How to make hooks persistent?

Hi,

I'm using the docker version of the software but cannot figure out how to make the hooks persistent after a restart.
The bot stays present in the room, but it loses its hooks. I have to recreate them each time.

Best regards,

O.

Bot didn't join the conversation/room

Hello,

I installed the bot without docker (tried also with docker). When I invite the bot to a room/conversation the bot didn't join. Status keeps invitation pending. I didn't found any errors in the homeserver.log.

Thanks in advance.

Emoji support

Is there any plan on integrating the ability to use emojis?

One use of webhooks for me is fail2ban notifications using ipinfo for showing flags (:us:).

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.