proxy-wasm / test-framework Goto Github PK

WebAssembly for Proxies (test framework)

License: Apache License 2.0

Rust 100.00%

test-framework's Introduction

WebAssembly for Proxies (test framework)

The Proxy-Wasm ABI and associated SDKs enable developers to build extensions in any of the supported languages and deliver the plugins as WebAssembly modules at run-time. This repository contains a standalone runner which serves as a test harness and simulator for Proxy-Wasm extensions, enabling quick testing in a controlled environment.

Examples

The basic usage of this test-framework is provided in the examples/ folder which contains mocking of proxy-wasm modules provided in the proxy-wasm-rust-sdk examples/.

Supported

Low-level expectation setting over most host-side functions that are consumed immediately
Checking of residual low-level expectations after wasm-function call has completed
Various high-level (simulator defaults) expectations that persist across several host-function calls as opposed to being immediately consumed
Expectation setting over returns from functions exposed on the proxy-wasm module

In Progress

Complete default implementation for all host-side functions
Low-level expectation setting for the remaining host-side functions
High-level expectation setting for the remaining host-side functions
Integration test examples

test-framework's People

Contributors

Stargazers

Watchers

Forkers

agiachris piotrsikora chandlernine revenuecat mccormickt cognitive-services-gateway alexsnaps kuadrant proxywasm ahmed-adam

test-framework's Issues

Add support for Proxy-Wasm ABI v0.2.0

This is needed to support extensions written using C++ SDK.

Add high-level expectations

Provide high-level expectations, e.g.:

http_headers_test
    .http_request(vec![
            (":method", "GET"),
            (":path", "/hello"),
            (":authority", "developer"),
        ],
        None, // request body
        None, // request trailers
    )
    .expect_log(LogLevel::Trace, "#1 -> :method: GET")
    .expect_log(LogLevel::Trace, "#1 -> :path: /hello")
    .expect_log(LogLevel::Trace, "#1 -> :authority: developer")
    .execute_and_expect(ReturnType::Action(Action::Continue))?;

which would be more or less equivalent to this test expressed using currently available low-level expectations:

http_headers_test
    .call_start()
    .execute_and_expect(ReturnType::None)?;

let root_context = 1;
http_headers_test
    .call_proxy_on_context_create(root_context, 0)
    .execute_and_expect(ReturnType::None)?;

let http_context = 2;
http_headers_test
    .call_proxy_on_context_create(http_context, root_context)
    .execute_and_expect(ReturnType::None)?;

http_headers_test
    .call_proxy_on_request_headers(http_context, 0)
    .expect_get_header_map_pairs(MapType::HttpRequestHeaders)
    .returning(vec![
        (":method", "GET"),
            (":path", "/hello"),
            (":authority", "developer"),
        ])
    .expect_log(LogLevel::Trace, "#1 -> :method: GET")
    .expect_log(LogLevel::Trace, "#1 -> :path: /hello")
    .expect_log(LogLevel::Trace, "#1 -> :authority: developer")
    .execute_and_expect(ReturnType::Action(Action::Continue))?;

Provide strict mode

In this mode (or maybe by default?) tests should fail if the extension is making unexpected hostcalls.

Prettify output

The current output is a bit rough and while fine for debugging, it's not very readable output that you want to see when running tests.

Consider using something like this:

[vm->host] function(params...) for host call entry,
[vm<-host] function return: X for host call return,
[host->vm] function(params...) for callback entry,
[host<-vm] function return: X for callback return.

with either OK, FAILED, UNEXPECTED being printed out on the right hand side, depending on the expectations.

How do you use this repository to test?

I tried this
~/gh/proxy-wasm/proxy-wasm-rust-sdk$ bazel build //examples:hello_world

the output looks like this

WARNING: Download from https://github.com/bazelbuild/bazel-skylib/releases/download/1.0.3/bazel-skylib-1.0.3.tar.gz failed: class javax.net.ssl.SSLHandshakeException PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
INFO: Analyzed target //examples:hello_world (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
Target //examples:hello_world up-to-date:
  bazel-bin/examples/hello_world.wasm

Then in the test-framework project I did this

cargo run --example hello_world
    Finished dev [unoptimized + debuginfo] target(s) in 0.05s
     Running `target/debug/examples/hello_world`
error: The following required arguments were not provided:
    <wasm-path>

USAGE:
    hello_world [FLAGS] <wasm-path>

For more information try --help

following that I tried this

cargo run --example hello_world ~/gh/proxy-wasm/proxy-wasm-rust-sdk/bazel-bin/examples/hello_world.wasm

The output looks like this

    Finished dev [unoptimized + debuginfo] target(s) in 0.07s
     Running `target/debug/examples/hello_world /home/janitha/gh/proxy-wasm/proxy-wasm-rust-sdk/bazel-bin/examples/hello_world.wasm`
thread 'main' panicked at 'Error: test-framework does not support proxy-wasm modules of this abi version', src/hostcalls.rs:47:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Why is this saying the abi version is unsupported?

The backtrace implies that I am calling improperly i.e. the wasm is not loaded

RUST_BACKTRACE=1 cargo run --example hello_world /home/janitha/gh/proxy-wasm/proxy-wasm-rust-sdk/bazel-bin/examples/hello_world.wasm
    Finished dev [unoptimized + debuginfo] target(s) in 0.07s
     Running `target/debug/examples/hello_world /home/janitha/gh/proxy-wasm/proxy-wasm-rust-sdk/bazel-bin/examples/hello_world.wasm`
thread 'main' panicked at 'Error: test-framework does not support proxy-wasm modules of this abi version', src/hostcalls.rs:47:9
stack backtrace:
   0: std::panicking::begin_panic
             at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/std/src/panicking.rs:541:12
   1: proxy_wasm_test_framework::hostcalls::get_abi_version
             at ./src/hostcalls.rs:47:9
   2: proxy_wasm_test_framework::tester::mock
             at ./src/tester.rs:47:23
   3: hello_world::main
             at ./examples/hello_world.rs:24:32
   4: core::ops::function::FnOnce::call_once
             at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/core/src/ops/function.rs:227:5

What is the correct way to call it?

Modules compiled with Rust 1.76 seem to require random_get in WASI targets

While trying out the test-framework I've found that WASM modules compiled with Rust 1.76 assume random_get exists:

[host->vm] proxy_on_context_create(root_context_id=1, parent_context_id=0)
[vm->host] random_get() -> (...) status: Unexpected ***THIS IS AN EMPTY PLACEHOLDER***
[vm<-host] random_get() -> (..) return: InternalFailure ***THIS IS AN EMPTY PLACEHOLDER***
[vm->host] proxy_log(level=5, message_data="panicked at library/std/src/sys/wasi/mod.rs:190:37:
random_get failure: Errno { code: 10, name: "BUSY", message: "Device or resource busy." }") status: Unexpected
Error: error while executing at wasm backtrace:
    0: 0x35400f - <unknown>!__rust_start_panic
    1: 0x353dcf - <unknown>!rust_panic
    2: 0x353d02 - <unknown>!std::panicking::rust_panic_with_hook::hc97f08b908247b1d
    3: 0x352f06 - <unknown>!std::panicking::begin_panic_handler::{{closure}}::hdd638bdaba0c6bcc
    4: 0x352e31 - <unknown>!std::sys_common::backtrace::__rust_end_short_backtrace::h5b392607309abb6a
    5: 0x3536d9 - <unknown>!rust_begin_unwind
    6: 0x359a53 - <unknown>!core::panicking::panic_fmt::h9fec598e5939a913
    7: 0x36055b - <unknown>!core::result::unwrap_failed::hf6fa377d4c4189f7
    8: 0x353fcc - <unknown>!std::sys::wasi::hashmap_random_keys::h005380782f2a95d3 ***THIS IS THE RELEVANT ONE***
    9: 0x2f1418 - <unknown>!std::hash::random::RandomState::new::KEYS::__getit::{{closure}}::hcefb3649984e942d
   10: 0x2f110b - <unknown>!std::sys::common::thread_local::lazy::LazyKeyInner<T>::initialize::h29f987aa14a20853
   11: 0x2f7889 - <unknown>!std::sys::common::thread_local::static_local::Key<T>::get::h46125d90ec6d1a87
   12: 0x1eb30 - <unknown>!std::hash::random::RandomState::new::KEYS::__getit::h7058f584453fcfdb
   13: 0x2f7a52 - <unknown>!std::thread::local::LocalKey<T>::try_with::he989749e1311f98b
   14: 0x2f7968 - <unknown>!std::thread::local::LocalKey<T>::with::hf6618a5eeec42590
   15: 0x1ebf5 - <unknown>!<std::hash::random::RandomState as core::default::Default>::default::h9ac44d7f5d33df22
   16: 0x8850 - <unknown>!<std::collections::hash::map::HashMap<K,V,S> as core::default::Default>::default::h915d2d8435b25d9f
   17: 0x36562 - <unknown>!tokenfilter::_initialize::{{closure}}::hed2a02ab21e62030
   18: 0x1847d - <unknown>!core::ops::function::FnOnce::call_once::he7da19782bed26e6
   19: 0x5bf36 - <unknown>!proxy_wasm::dispatcher::Dispatcher::create_root_context::h8b217841f9e5145f
   20: 0x5c87f - <unknown>!proxy_wasm::dispatcher::Dispatcher::on_create_context::he348d9f5cfffda63
   21: 0x618af - <unknown>!proxy_wasm::dispatcher::proxy_on_context_create::{{closure}}::hb251467ca6eabcc7
   22: 0x8a43b - <unknown>!std::thread::local::LocalKey<T>::try_with::h2782b0d9105d5b2d
   23: 0x8a289 - <unknown>!std::thread::local::LocalKey<T>::with::h0f823b49025a9729
   24: 0x6183e - <unknown>!proxy_on_context_create

It appears to try to manipulate a hashmap which now assumes !std::sys::wasi::hashmap_random_keys is usable in wasm32-wasi targets.

Compiling with an older rust version (i.e. 1.67.0) appears to run into a different issue (environ_sizes_get in my case), but looking at https://github.com/rust-lang/rust/blob/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/sys/wasi/mod.rs#L185 it's going to try to make this hostcall no matter what we do.

This pretty much ties in to #2 where 0.2.0 features are getting used more and more often. Mocking out the random_get hostcall does seem to work, but in my trial and error I just always return the same "random" value, so that's not really a usable implementation.

Use macros to avoid some of the repetetion

Adaptive Wasm callbacks based on exports

Combination calls such as http_request(..) combine individual proxy-wasm callbacks in order: proxy_on_http_request_headers(..), proxy_on_http_request_body(..), proxy_on_http_request_trailers(..). In some cases, one of these functions may not exported across the ABI boundary. To prevent test failure, calling of these individual functions should not depend on the default simulator settings, but rather, on the visible functions exported by the wasm module.

We propose that upon initially mocking the wasm module, a check of exported wasm-side functions is done to prevent potential calls to such functions during high-level testing.

proxy-wasm / test-framework Goto Github PK

test-framework's Introduction

WebAssembly for Proxies (test framework)

Examples

Supported

In Progress

test-framework's People

Contributors

Stargazers

Watchers

Forkers

test-framework's Issues

Recommend Projects

Recommend Topics

Recommend Org