Code Monkey home page Code Monkey logo

wasm-filters's Introduction

Shows a dark layer5 logo in light mode and a white logo in dark mode

GitHub contributors GitHub GitHub issues by-label Slack Twitter Follow

WebAssembly Filters for Envoy

A collection of WebAssemby filters for Envoy proxy written in C,C++,C# and Rust for exercising different features provided by envoy-wasm.

See the Image Hub as a related project (a sample application). Also, see Meshery's filter management capabitilies.

Weekly Meeting Details

See all community meeting details --> https://meet.layer5.io


Join the Community!

Our projects are community-built and welcome collaboration. πŸ‘ Be sure to see the Layer5 Community Welcome Guide for a tour of resources available to you and jump into our Slack!

Shows an illustrated light mode meshery logo in light color mode and a dark mode meshery logo dark color mode.

Layer5 Community

βœ”οΈ Join community meetings. See details on the Layer5 community calendar.
βœ”οΈ Watch community meeting recordings.
βœ”οΈ Access the Community Drive by completing a community Member Form.
βœ”οΈ Discuss in the Community Forum.

Not sure where to start? Grab an open issue with the help-wanted label.

Β 

About Layer5

Layer5's cloud native application and infrastructure management software enables organizations to expect more from their infrastructure. We embrace developer-defined infrastructure. We empower engineer to change how they write applications, support operators in rethinking how they run modern infrastructure and enable product owners to regain full control over their product portfolio.

Get the Rust toolchain

To compile Rust filters to WASM, the nightly toolchain and support for wasm compilation target is needed. Make sure you have Rust and Cargo installed using Rustup. If you're on a *nix system (Unix, Linux, MacOS), in the project root directory, run:

make rust-toolchain

This will also install wasm-pack for you. Also, take a look at installing wasm-pack for OS other than *nix.

Upstream

Upstream is a webserver which is used by few of the filters mentioned above. It provides a route for :

  • Mock authentication
  • Storing Metrics
  • Retrieving Metrics

Build the docker Image for Upstream before proceeding with the examples.

Build Image:

cd upstream
make

HTTP-Auth

Simulates handling authentication of requests at proxy level. Requests with a header token with value hello are accepted as authorized while the rest unauthorized. The actual authentication is handled by the Upstream server. Whenever the proxy recieves a request it extracts the token header and makes a request to the Upstream server which validates the token and returns a response.

Build and deploy:

cd http-auth
make run-filtered

Test:

curl  -H "token":"hello" 0.0.0.0:18000 -v # Authorized
curl  -H "token":"world" 0.0.0.0:18000 -v # Unauthorized

TCP-Metrics

Collects simple metrics for every TCP packet and logs it.

Build and deploy:

cd tcp-metrics
make run-filtered

Test:

curl 0.0.0.0:18000 -v -d "request body"

Check the logs for the metrics.

TCP-Packet-Parse

Parses the contents of every TCP packet the proxy receives and logs it.

Build and deploy:

cd tcp-packet-parse
make run-filtered

Test:

curl 0.0.0.0:18000 -v -d "request body"

Check the logs for the packet contents.

Singleton-HTTP-Call

An example which depicts an singleton HTTP WASM service which does an HTTP call once every 2 seconds.

Build and deploy:

cd singleton-http-call
make run-filtered

Check the logs for the response of the request.

Metrics-Store

This example showcases communication between a WASM filter and a service via shared queue. It combines the Singleton-HTTP-Call and TCP-Metrics examples. The filter collects metrics and enqueues it onto the queue while the service dequeues it and sends it to upstream server where it is stored.

Build and deploy:

cd metrics-store
make run-filtered

Test:

curl 0.0.0.0:18000 -v -d "request body" # make a few of these calls
curl 0.0.0.0:8080/retrieve -v # Retrieves the stored stats
# x | y | z  === x : downstream bytes, y : upstream bytes, z: the latency for application server to respond 

wasm-filters's People

Contributors

abhijeetgauravm avatar acald-creator avatar adeola-adesoba avatar adithyaakrishna avatar aminoxix avatar ashiscs avatar chadha93 avatar chetak123 avatar delusionaloptimist avatar felixfaisal avatar josephmidura avatar kanishkarj avatar kumarabd avatar leecalcote avatar pranav-bhatt avatar revolyssup avatar shuklaritvik06 avatar subhamkrai avatar thebeginner86 avatar utkarshmishra12 avatar warunicorn19 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

wasm-filters's Issues

[Bug] Envoy proxy container failing

Description
Proxy container(Envoy) container exits due to an issue with configuration

Expected Behavior
Should run the TCP metrics succesfully

Error

proxy_1        | [2022-01-03 16:20:08.763][8][info][main] [external/envoy/source/server/server.cc:372]   envoy.request_id: envoy.request_id.uuid
proxy_1        | [2022-01-03 16:20:08.766][8][critical][main] [external/envoy/source/server/server.cc:114] error initializing configuration '/etc/envoy.yaml': Protobuf message (type envoy.config.bootstrap.v3.Bootstrap reason INVALID_ARGUMENT:(static_resources.clusters[0]) hosts: Cannot find field.) has unknown fields
proxy_1        | [2022-01-03 16:20:08.766][8][info][main] [external/envoy/source/server/server.cc:891] exiting
proxy_1        | [2022-01-03 16:20:08.766][8][debug][main] [external/envoy/source/common/access_log/access_log_manager_impl.cc:21] destroyed access loggers
proxy_1        | [2022-01-03 16:20:08.766][8][debug][main] [external/envoy/source/common/event/dispatcher_impl.cc:82] destroying dispatcher main_thread
proxy_1        | [2022-01-03 16:20:08.766][8][debug][init] [external/envoy/source/common/init/watcher_impl.cc:31] init manager Server destroyed
proxy_1        | Protobuf message (type envoy.config.bootstrap.v3.Bootstrap reason INVALID_ARGUMENT:(static_resources.clusters[0]) hosts: Cannot find field.) has unknown fields
tcp-metrics_proxy_1 exited with code 1

Environment:

  • OS: Ubuntu
  • Version: 21.10
  • Device: Laptop

To Reproduce
Follow the steps for TCPmetrics to reproduce this error,
However I've used make run to see docker logs

Error running the http-auth filter.

Description
Tried running the composition for testing the http-auth filter (which looks a lot like something I've build awhile ago)
but Envoy crashes when trying to load the WASM module:

proxy_1          | [2021-09-13 15:31:21.608][7][critical][main] [external/envoy/source/server/server.cc:96] error initializing configuration '/etc/envoy.yaml': Failed to load WASM module due to an import type mismatch: env.proxy_grpc_call, want: i32 i32 i32 i32 i32 i32 i32 i32 i32 i32 i32 i32 -> i32, but host exports: i32 i32 i32 i32 i32 i32 i32 i32 i32 i32 -> i32

Expected Behavior
Envoy loads WASM code and runs.

Environment:

  • OS: MacOS

[README] Need to fix Slack badge

Description

README.md is the first file one should read when starting a new project. For attractive look, we added some badges also.
But, there's a Slack badge failing, need to fix that.

Example Screenshot

Current Behavior

image

Desired Behavior
image

Environment:

  • OS: any [e.g. Ubuntu]
  • Browser: any [e.g. Chrome, Safari]
  • Version: any [e.g. 22]
  • Device: any [e.g. laptop, iPhone 8]

[README] Need to add `Community Forum`

Description

README.md is the first file one should read when starting a new project. Our community launched Discussion forum for help seekers; needs to publish it officially on README.md for convenience.

Screenshots

Current behaviour:
image

Desired behaviour:
image

Note:
Need to add only last line for Community Forum

Environment:

  • OS: any [e.g. Ubuntu]
  • Browser: any [e.g. Chrome, Safari]
  • Version: any [e.g. 22]
  • Device: any [e.g. laptop, iPhone 8]

Reference:

[Refactor] Eliminate warnings in Rust code

Description
Currently, upon building any of the filters you get rust compilation warnings. We can eliminate these warnings and follow best practices of Rust.

Expected Behavior
Code compiles without any warnings

Screenshots
Image

To Reproduce
Steps to reproduce the behavior:

  • Compile any of the filters

[CI] add label commenter workflow

Description:

This repository should have the label commenter workflow.
It would run whenever an issue/pr is labelled with one of the targeted labels and comment down any relevant information.

Contributor Reference:

The workflow would use: https://github.com/peaceiris/actions-label-commenter

A sample workflow is defined in: https://github.com/meshery/meshery/blob/master/.github/workflows/label-commenter.yml.

It would also need a sample configuration which is defined in: https://github.com/meshery/meshery/blob/master/.github/label-commenter-config.yml

NOTE:

[Filters] wasm filter for extracting data from a JWT

See the filter hub spec for detailed user story

Use Case
A user would like to parse and remove the email address and "sub" claim from an incoming JWT’s payload data and append it to the request headers, so that they can route traffic to the appropriate destination service.

Current Behavior

User has been writing Lua filters for extracting data from the JWTs and appending it to the header, but would like to avoid this.

Desired Behavior

Using a portable and reusable WASM filter instead for achieving the above mentioned functionality.

[Build] use wasm-pack for compiling rust to wasm

This issue addresses the discussions in #10.

Current behaviour
The current build rules compile Rust to Wasm without any optimizations. This results in large sized unoptimized Wasm binaries having unnecessary references and lines linking to packages not needed.

Desired behaviour
As suggested by @pranav-bhatt, we should use wasm-pack for compiling to smaller and optimized Wasm binaries.

This will include updating the current build process for examples by modifying makefiles, docker-compose configs and Cargo.toml manifests.

[CI] Release: Attach build artifacts to GH release entry

Current Behavior

The build artifact wash-filters.tar.gz is being captured and attached as build artifacts to the release workflow.

Desired Behavior

Instead, capture and attach the build artifacts (wasm-filters.tar.gz) to the given release itself.


Contributor Guides and Resources

[README] Update README Community Section

Description
The Community section in the README says "Access the Community Drive". There is a new procedure of having this automated based on the individual filling in a community member form

Expected Behavior
Update "Access Community Drive" to "To Access Community Drive, fill Community Member Form"

Screenshots
img

Resources
Refer to the Contributing guide

Environment:

  • OS: [e.g. Ubuntu]
  • Browser: [e.g. Chrome, Safari]
  • Version: [e.g. 22]
  • Device: [e.g. laptop, iPhone 8]

Error in proxy containers logs when attempting to run http-auth example

I am trying to run the http-auth example using the command make run-filtered which build fine but once compose begins running container the http-auth_proxy_1 container exits due to the following error. It would seem it does not like something in the envoy.yaml config but I am not entirely sure what.

Failed to load WASM module due to an import type mismatch: env.proxy_grpc_call, want: i32 i32 i32 i32 i32 i32 i32 i32 i32 i32 i32 i32 -> i32, but host exports: i32 i32 i32 i32 i32 i32 i32 i32 i32 i32 -> i32

Full logs:

[2021-08-22 15:18:29.952][9][debug][wasm] [bazel-out/k8-opt/bin/external/envoy/source/extensions/common/wasm/_virtual_includes/wasm_vm_base/extensions/common/wasm/wasm_vm_base.h:38] WasmVm created envoy.wasm.runtime.v8 now active
[2021-08-22 15:18:29.952][9][debug][wasm] [external/envoy/source/extensions/common/wasm/wasm.cc:121] Base Wasm created 1 now active
[2021-08-22 15:18:30.067][9][debug][wasm] [external/envoy/source/extensions/common/wasm/wasm.cc:291] ~Wasm 0 remaining active
[2021-08-22 15:18:30.069][9][debug][wasm] [bazel-out/k8-opt/bin/external/envoy/source/extensions/common/wasm/_virtual_includes/wasm_vm_base/extensions/common/wasm/wasm_vm_base.h:42] ~WasmVm envoy.wasm.runtime.v8 0 remaining active
[2021-08-22 15:18:30.069][9][debug][init] [external/envoy/source/common/init/watcher_impl.cc:27] ListenerImpl destroyed
[2021-08-22 15:18:30.069][9][debug][init] [external/envoy/source/common/init/watcher_impl.cc:27] init manager Listener main destroyed
[2021-08-22 15:18:30.070][9][critical][main] [external/envoy/source/server/server.cc:96] error initializing configuration '/etc/envoy.yaml': Failed to load WASM module due to an import type mismatch: env.proxy_grpc_call, want: i32 i32 i32 i32 i32 i32 i32 i32 i32 i32 i32 i32 -> i32, but host exports: i32 i32 i32 i32 i32 i32 i32 i32 i32 i32 -> i32
[2021-08-22 15:18:30.070][9][debug][init] [external/envoy/source/common/init/watcher_impl.cc:27] ClusterImplBase destroyed
[2021-08-22 15:18:30.070][9][debug][init] [external/envoy/source/common/init/watcher_impl.cc:27] init manager Cluster wasm_upstream destroyed
[2021-08-22 15:18:30.070][9][debug][init] [external/envoy/source/common/init/watcher_impl.cc:27] ClusterImplBase destroyed
[2021-08-22 15:18:30.070][9][debug][init] [external/envoy/source/common/init/watcher_impl.cc:27] init manager Cluster web_service destroyed
[2021-08-22 15:18:30.070][9][debug][grpc] [external/envoy/source/common/grpc/google_async_client_impl.cc:35] Joining completionThread
[2021-08-22 15:18:30.070][13][debug][grpc] [external/envoy/source/common/grpc/google_async_client_impl.cc:68] completionThread exiting
[2021-08-22 15:18:30.070][9][debug][grpc] [external/envoy/source/common/grpc/google_async_client_impl.cc:37] Joined completionThread
[2021-08-22 15:18:30.070][9][debug][upstream] [external/envoy/source/common/upstream/cluster_manager_impl.cc:937] shutting down thread local cluster manager
[2021-08-22 15:18:30.070][9][info][main] [external/envoy/source/server/server.cc:619] exiting
[2021-08-22 15:18:30.071][9][debug][init] [external/envoy/source/common/init/watcher_impl.cc:27] init manager Server destroyed
Failed to load WASM module due to an import type mismatch: env.proxy_grpc_call, want: i32 i32 i32 i32 i32 i32 i32 i32 i32 i32 i32 i32 -> i32, but host exports: i32 i32 i32 i32 i32 i32 i32 i32 i32 i32 -> i32

Issue Template Updates for Figma linkΒ 

Current Behavior

The project's current issue templates are missing an open invitation link where new contributors can join the community's Figma team and view user interface designs and other UX projects.

Desired Situation

Each template that has a reference to Figma in its resources section should have an invite link added.

Implementation

- 🎨 Wireframes and [designs for Layer5 site](https://www.figma.com/file/5ZwEkSJwUPitURD59YHMEN/Layer5-Designs) in Figma [(open invite)](https://www.figma.com/team_invite/redeem/qJy1c95qirjgWQODApilR9)

Acceptance Tests
All references to Figma include the "open invite" link.


Contributor Guides and Resources

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.