Code Monkey home page Code Monkey logo

webhookrelay-operator's Introduction

Webhook Relay Kubernetes Operator

Build Status

Webhook Relay Operator provides an easy way to receive webhooks to an internal Kubernetes cluster without configuring public IP or load balancer. Perfect for:

  • On-premise deployments
  • Cloud deployments where public load balancer is not required (single endpoint receiving webhooks and no need to expose the whole server)
  • Edge deployments
  • IoT & Edge computing with https://k3s.io/

Operator can manage buckets, configure your public endpoints that accept webhooks/API requests and sets up forwarding destinations (where HTTP requests will be sent).

Features

Current operator project scope:

  • Deploy webhook forwarding agents with configured buckets
  • Read credentials from secrets and mount secrets to webhookrelayd containers
  • Ensure buckets are created
  • Ensure inputs are configured (public endpoints)
  • Ensure outputs are configured (forwarding destinations)
  • K8s events on taken actions
  • Updates CR status

Roadmap

  • Create & manage Functions that transform webhook requests and responses
  • Manage Function configuration through Kubernetes secrets
  • Provision separate access tokens for webhookrelayd containers with disabled API access (only subscribe capability). CR should have a finalizer that would ensure that the secret is removed together with the agent configuration.
  • Deploy Webhook Relay ingress controller (separate CRD)
  • Expose webhookrelayd agent forwarding metrics
  • Configure notification integrations via CRDs

Installation

Prerequisites:

You need to add this Chart repo to Helm:

helm repo add webhookrelay https://charts.webhookrelay.com
helm repo update

Get access token from here. Once you click on 'Create Token', it will generate it and show a helper to set environment variables:

export RELAY_KEY=*****-****-****-****-*********
export RELAY_SECRET=**********

Install through Helm:

helm upgrade --install webhookrelay-operator --namespace=default webhookrelay/webhookrelay-operator \
  --set credentials.key=$RELAY_KEY --set credentials.secret=$RELAY_SECRET

Usage

Operator works as a manager to configure your public endpoints and forwarding destinations. To start receiving webhooks you will need to create a Custom Resource (usually called just 'CR'). It's a short yaml file that describes your public endpoint characteristics and specifies where to forward the webhooks:

# cr.yaml
apiVersion: forward.webhookrelay.com/v1
kind: WebhookRelayForward
metadata:
  name: example-forward
spec:
  buckets:
  - name: k8s-operator
    inputs:
    - name: public-endpoint
      description: "Public endpoint, supply this to the webhook producer"
      responseBody: "OK"
      responseStatusCode: 200
    outputs:
    - name: webhook-receiver
      lockPath: true  # set to 'false' to reuse any extra path WHR received
      disabled: false # set to 'true' to disable output
      destination: http://destination:5050/webhooks
kubectl apply -f cr.yaml

Now, to view CR status which will display our public endpoints:

# get available CRs
$ kubectl get webhookrelayforwards.forward.webhookrelay.com
# get our example forward status
$ kubectl describe webhookrelayforwards.forward.webhookrelay.com example-forward
Name:         example-forward
Namespace:    default
Labels:       <none>
Annotations:  API Version:  forward.webhookrelay.com/v1
Kind:         WebhookRelayForward
Metadata:
  Creation Timestamp:  2020-06-18T23:05:33Z
  Generation:          1
  Resource Version:    118902
  Self Link:           /apis/forward.webhookrelay.com/v1/namespaces/default/webhookrelayforwards/example-forward
  UID:                 998b0fca-f975-40dd-b2b5-91abd1edaee0
Spec:
  Buckets:
    Inputs:
      Description:           Public endpoint, supply this to the webhook producer
      Name:                  public-endpoint
      Response Body:         OK
      Response Status Code:  200
    Name:                    k8s-operator
    Outputs:
      Destination:       http://destination:5050/webhooks
      Name:              webhook-receiver
  Secret Ref Name:       whr-credentials
  Secret Ref Namespace:  
Status:
  Agent Status:  Running
  Public Endpoints:
    https://my.webhookrelay.com/v1/webhooks/92582560-738a-4eae-94b1-23299ed20b3c
  Ready:           true
  Routing Status:  Configured
Events:            <none>

Here we can see our public endpoints.

Advanced Usage (multi-tenant, credentials per CR)

If more than one user is using the operator, it's possible to skip credentials setting during Helm install and just specify the access token key & secret in the CR itself:

# access_token.yaml
apiVersion: v1
kind: Secret
metadata:
  name: whr-credentials
type: Opaque
stringData:
  key: XXX    # your access token key
  secret: YYY # your access token secret

Create it:

kubectl apply -f access_token.yaml

Specify the secret ref in the CR as secretRefName and secretRefNamespace (this one is optional):

# cr.yaml
apiVersion: forward.webhookrelay.com/v1
kind: WebhookRelayForward
metadata:
  name: example-forward
spec:
  secretRefName: whr-credentials # Secret 
  secretRefNamespace: ""
  buckets:
  - name: k8s-operator
    inputs:
    - name: public-endpoint
      description: "Public endpoint, supply this to the webhook producer"
      responseBody: "OK"
      responseStatusCode: 200
    outputs:
    - name: webhook-receiver
      lockPath: true  # set to 'false' to reuse any extra path WHR received
      disabled: false # set to 'true' to disable output
      destination: http://destination:5050/webhooks
  # Use custom Docker image
  #image: "quay.io/your-custom/image:latest"
  # Add custom env variables to the agent container
  extraEnvVars:
  - name: WEBSOCKET_TRANSPORT
    value: "true"

Create the CR:

kubectl apply -f cr.yaml

HTTP Proxy settings

If your outgoing connections are intercepted by an HTTP/HTTPS proxy - you will need to supply connection details with --set httpProxy or --set httpsProxy Helm values:

helm upgrade --install webhookrelay-operator --namespace=default webhookrelay/webhookrelay-operator \
  --set credentials.key=$RELAY_KEY --set credentials.secret=$RELAY_SECRET \
  --set httpsProxy="https://example-proxy.com"

This will set environment variables for the operator and operator will propagate them to the deployed agent.

webhookrelay-operator's People

Contributors

rimusz avatar rusenask avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

Forkers

rimusz

webhookrelay-operator's Issues

Help to configure atlantis on internal Kubernetes with Webhook Relay

Is using this article to configure Webhook relay for my atlantis lab on kubernetes.

I did folow all intructions but when I configured WebhookRelayForward, I received the follwoing error:

{"level":"error","ts":1688430599.769275,"logger":"controller_webhookrelayforward","msg":"failed to create input","Request.Namespace":"atlantis","Request.Name":"forward-to-atlantis","bucket_name":"github-to-atlantis","bucket_id":"c28be0c1-1fe0-4300-b970-8495236b56ec","error":"HTTP status 402: feature not available for your subscription","errorVerbose":"HTTP status 402: feature not available for your subscription\ngithub.com/webhookrelay/webhookrelay-go.(*API).makeRequestWithAuthTypeAndHeaders\n\tgithub.com/webhookrelay/[email protected]/webhookrelay.go:207\ngithub.com/webhookrelay/webhookrelay-go.(*API).makeRequestWithAuthType\n\tgithub.com/webhookrelay/[email protected]/webhookrelay.go:119\ngithub.com/webhookrelay/webhookrelay-go.(*API).makeRequest\n\tgithub.com/webhookrelay/[email protected]/webhookrelay.go:111\ngithub.com/webhookrelay/webhookrelay-go.(*API).CreateInput\n\tgithub.com/webhookrelay/[email protected]/inputs.go:134\ngithub.com/webhookrelay/webhookrelay-operator/pkg/controller/webhookrelayforward.(*ReconcileWebhookRelayForward).ensureBucketInputs\n\tgithub.com/webhookrelay/webhookrelay-operator/pkg/controller/webhookrelayforward/sync_inputs.go:44\ngithub.com/webhookrelay/webhookrelay-operator/pkg/controller/webhookrelayforward.(*ReconcileWebhookRelayForward).ensureRoutingConfiguration\n\tgithub.com/webhookrelay/webhookrelay-operator/pkg/controller/webhookrelayforward/webhookrelayforward_routing_engine.go:30\ngithub.com/webhookrelay/webhookrelay-operator/pkg/controller/webhookrelayforward.(*ReconcileWebhookRelayForward).Reconcile\n\tgithub.com/webhookrelay/webhookrelay-operator/pkg/controller/webhookrelayforward/webhookrelayforward_controller.go:150\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\tsigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:256\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tsigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:232\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker\n\tsigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:211\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\tk8s.io/[email protected]/pkg/util/wait/wait.go:155\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\tk8s.io/[email protected]/pkg/util/wait/wait.go:156\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\tk8s.io/[email protected]/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.Until\n\tk8s.io/[email protected]/pkg/util/wait/wait.go:90\nruntime.goexit\n\truntime/asm_amd64.s:1594","stacktrace":"github.com/go-logr/zapr.(*zapLogger).Error\n\tgithub.com/go-logr/[email protected]/zapr.go:128\ngithub.com/webhookrelay/webhookrelay-operator/pkg/controller/webhookrelayforward.(*ReconcileWebhookRelayForward).ensureBucketInputs\n\tgithub.com/webhookrelay/webhookrelay-operator/pkg/controller/webhookrelayforward/sync_inputs.go:46\ngithub.com/webhookrelay/webhookrelay-operator/pkg/controller/webhookrelayforward.(*ReconcileWebhookRelayForward).ensureRoutingConfiguration\n\tgithub.com/webhookrelay/webhookrelay-operator/pkg/controller/webhookrelayforward/webhookrelayforward_routing_engine.go:30\ngithub.com/webhookrelay/webhookrelay-operator/pkg/controller/webhookrelayforward.(*ReconcileWebhookRelayForward).Reconcile\n\tgithub.com/webhookrelay/webhookrelay-operator/pkg/controller/webhookrelayforward/webhookrelayforward_controller.go:150\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\tsigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:256\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\tsigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:232\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker\n\tsigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:211\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1\n\tk8s.io/[email protected]/pkg/util/wait/wait.go:155\nk8s.io/apimachinery/pkg/util/wait.BackoffUntil\n\tk8s.io/[email protected]/pkg/util/wait/wait.go:156\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\tk8s.io/[email protected]/pkg/util/wait/wait.go:133\nk8s.io/apimachinery/pkg/util/wait.Until\n\tk8s.io/[email protected]/pkg/util/wait/wait.go:90"}

My webhook-relay-operator.yaml

replicaCount: 1

image:
  repository: webhookrelay/webhookrelay-operator
  pullPolicy: Always
  tag: "0.6.0"

credentials:
  key: my-key
  secret: my-secret

httpsProxy: ""

imagePullSecrets: []
nameOverride: ""
fullnameOverride: ""

serviceAccount:
  # Specifies whether a service account should be created
  create: true
  # Annotations to add to the service account
  annotations: {}
  # The name of the service account to use.
  # If not set and create is true, a name is generated using the fullname template
  name: "webhookrelay-operator"

crd:
  create: false

rbac:
  create: true

podAnnotations:
  prometheus.io/scrape: "true"
  prometheus.io/port: "8383"

podSecurityContext: {}

securityContext: {}

resources:
  limits:
    cpu: 100m
    memory: 256Mi
  requests:
    cpu: 50m
    memory: 128Mi

nodeSelector: {}

tolerations: []

affinity: {}

My webhook-atlantis.yaml

apiVersion: forward.webhookrelay.com/v1
kind: WebhookRelayForward
metadata:
  name: forward-to-atlantis
  namespace: atlantis
spec:
  buckets:
  - name: github-to-atlantis
    inputs:
    - name: public-endpoint
      description: "Endpoint for GitHub"
      responseBody: "OK"
      responseStatusCode: 200
    outputs:
    - name: atlantis-pod
      destination: http://runatlantis.atlantis.svc.cluster.local # My atlantis service internal URL.

When I perform any PR on dashboard of webhook relay:

image

Could you help me please?

Feature Request: Support lock destination path

When we create a bucket via the operator, there isn't a way to lock the destination path. For a lot of our workloads the path we are outputting to matters and should static. We end up need to go into the UI and toggling the Lock destination path options for the destination configurations for a bucket for the vast majority of our buckets. Every time the operator recreates the bucket we need to go back into the UI and manually set that setting. It would a significant improvement if we could lock the path in the CRD.

ARM controller still spawns amd64 forwarder

The k8s deployment created by the operator defaults to an image - presuming it's amd64 - which throws an exec format error on startup. As far as I can see, the helm chart has no way to override this. Manually editing the deployment seems to work but it would be nice not to have to.

Arm64 support

Hello, are you planning to provide a multi arch image for arm64 architectures? Thanks in advance

Forward same request to a service in every namespaces

Hello, thank you for maintaining this tool ๐Ÿ˜Š

My use case is forwarding the request to a service in every namespaces.

I guess I could create a single WebhookRelayForward with multiple outputs like this :

      outputs:
        - name: business-backend-ns1
          destination: http://business-backend.ns1.svc.cluster.local
          internal: true
          lockPath: false
        - name: business-backend-ns2
          destination: http://business-backend.ns2.svc.cluster.local
          internal: true
          lockPath: false

But it means being aware of every namespaces at once, and mines are dynamic (these are ephemeral environments which all needs the requests).

Do I have another solution ? would it work if I created the same object in each namespace ; would it merge the redirection to do all, or would it be overriden each time by the latest ?

Thanks in advance for any input ๐Ÿ™

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.