Code Monkey home page Code Monkey logo

wasm3 / wasm3 Goto Github PK

View Code? Open in Web Editor NEW
7.0K 128.0 434.0 5.74 MB

๐Ÿš€ A fast WebAssembly interpreter and the most universal WASM runtime

Home Page: https://twitter.com/wasm3_engine

License: MIT License

C 90.39% Shell 0.42% C++ 3.73% CMake 0.96% Python 2.78% JavaScript 0.32% Lua 0.01% WebAssembly 0.11% Makefile 0.73% Assembly 0.11% Java 0.11% Swift 0.24% Zig 0.09%
wasm webassembly embedded iot scripting edge-computing cosmopolitan smart-contracts virtual-machine serverless

wasm3's Introduction

Note

I regret to inform the community that since my house was destroyed by russians who invaded my country, Wasm3 will enter a minimal maintenance phase. At this time, I am unable to continue the development of new features. However, I am committed to keeping the project alive and will actively review and merge incoming Pull Requests. I deeply appreciate your understanding and support during this difficult period. Your contributions to Wasm3 are now more valuable than ever.

Wasm3

StandWithUkraine WAPM GitHub issues Tests status Fuzzing Status GitHub license

A fast WebAssembly interpreter and the most universal WASM runtime.
Based on CoreMark 1.0 and independent benchmarks. Your mileage may vary.

Twitter Discord

Getting Started

Here's a small getting started guide. Click here to start:

LIVE DEMO

Installation

Please follow the installation instructions.

Wasm3 can also be used as a library for:

Python3 โ”‚ Rust โ”‚ C/C++ โ”‚ GoLang โ”‚ Zig โ”‚ Perl
Swift โ”‚ .Net โ”‚ Nim โ”‚ Arduino, PlatformIO, Particle โ”‚ QuickJS

Status

wasm3 passes the WebAssembly spec testsuite and is able to run many WASI apps.

Minimum useful system requirements: ~64Kb for code and ~10Kb RAM

wasm3 runs on a wide range of architectures (x86, x86_64, ARM, RISC-V, PowerPC, MIPS, Xtensa, ARC32, ...) and platforms:

  • Linux, Windows, OS X, FreeBSD, Android, iOS
  • OpenWrt, Yocto, Buildroot (routers, modems, etc.)
  • Raspberry Pi, Orange Pi and other SBCs
  • MCUs: Arduino, ESP8266, ESP32, Particle, ... see full list
  • Browsers. Yes, using WebAssembly itself!
  • wasm3 can execute wasm3 (self-hosting)

Features

Webassembly Core Proposals Extra
โ˜‘ Import/Export of Mutable Globals โ˜‘ Structured execution tracing
โ˜‘ Non-trapping float-to-int conversions โ˜‘ Big-Endian systems support
โ˜‘ Sign-extension operators โ˜‘ Wasm and WASI self-hosting
โ˜‘ Multi-value โ˜‘ Gas metering
โ˜‘ Bulk memory operations (partial support) โ˜‘ Linear memory limit (< 64KiB)
โ˜ Multiple memories
โ˜ Reference types
โ˜ Tail call optimization
โ˜ Fixed-width SIMD
โ˜ Exception handling

Motivation

Why use a "slow interpreter" versus a "fast JIT"?

In many situations, speed is not the main concern. Runtime executable size, memory usage, startup latency can be improved with the interpreter approach. Portability and security are much easier to achieve and maintain. Additionally, development impedance is much lower. A simple library like Wasm3 is easy to compile and integrate into an existing project. (Wasm3 builds in a just few seconds). Finally, on some platforms (i.e. iOS and WebAssembly itself) you can't generate executable code pages in runtime, so JIT is unavailable.

Why would you want to run WASM on embedded devices?

Wasm3 started as a research project and remains so by any means. Evaluating the engine in different environments is part of the research. Given that we have Lua, JS, Python, Lisp, ... running on MCUs, WebAssembly is a promising alternative. It provides toolchain decoupling as well as a completely sandboxed, well-defined, predictable environment. Among practical use cases we can list edge computing, scripting, plugin systems, running IoT rules, smart contracts, etc.

Used by

ใ€€ ใ€€ ใ€€ ใ€€ ใ€€ ใ€€ ใ€€ ใ€€ ใ€€ ใ€€ ใ€€ ใ€€

Further Resources

Demos
Installation instructions
Cookbook
Troubleshooting
Build and Development instructions
Supported Hardware
Testing & Fuzzing
Performance
Interpreter Architecture
Logging
Awesome WebAssembly Tools

License

This project is released under The MIT License (MIT)

wasm3's People

Contributors

abalkin avatar atdrendel avatar axic avatar cclauss avatar cjihrig avatar coderzh avatar fgasper avatar gibachan avatar guusw avatar hailtododongo avatar iddm avatar igrr avatar kateinoigakukun avatar maxgraey avatar mithro avatar nickwanninger avatar nlordell avatar petersalomonsen avatar phcoder avatar robinvanemden avatar ryantrem avatar saghul avatar sinkingsugar avatar skrphv avatar soundandform avatar syrusakbary avatar t-veor avatar vlad-ivanov-name avatar vshymanskyy avatar zhuxingwei avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

wasm3's Issues

m3_GetErrorInfo broken

M3ErrorInfo originally used a fixed string array to simplify & avoid memory ownership issues. The pointer mechanism in use now is getting nullified when m3_GetErrorInfo is called so no actual error information is show.

Improve performance further (perform profiling, etc)

Wasm3 is fast, but we didn't even get to profiling yet.

As noted by @dstogov:

Comparing wasm3 performance with PHP and converting this test to PHP... Actually, php master branch, compiled by GCC, with opcache/optimizer enabled (without JIT) is a bit (~15%) faster. I'm a bit surprised, because our VMs achieve very similar VM assembler code (by different ways), WASM code and data are simpler to interpret, and you made additional optimization for accumulator register. For some reason, wasm3 suffers from branch miss-prediction (more than 10 time more misses than PHP) and cache misses (almost 100 times more L1 DCache misses). Measured with Linux perf stat.
$ perf record -e L1-dcache-load-misses ./wasm3 mandel.wasm
$ perf report -n
It looks like, your virtual code segment evicted from CPU cache too often.
Fixing this, should significantly improve performance.
However, this also may be related to bad indirect branch prediction and speculative execution.

If you have any other ideas on how to improve performance, please comment here.

RFC: add CMakeLists.txt to `source` subdirectory, make it useable as a library

For projects built with CMake build system, the natural way of using wasm3 would be to call add_subdirectory to include wasm3 as a static library. However the root CMakeLists file contains a lot of logic related to host app builds, so it may be not suitable for the purpose.

It seems that one way to expose a static library would be to add a CMakeLists.txt file into the source directory, then add this library using add_subdirectory and target_link_libraries in the CMakeLists files which want to use this library. This is in line with the "modern CMake" approach to structuring CMake projects. It helps eliminate hacks like symlinks in platform-specific examples (at least the ones which are built using CMake).

If noone sees a problem with this, I will open a PR for such refactoring.

Wasmroutines

How hard would be to add ability to execute multithreaded code (from WebAssembly app point of view) by a single system thread? Something like WebAssembly/design#1321?

Remove -w flag, clean up warnings

I understand that the project is moving quite quickly, so some number of warnings may be acceptable. However I've already bumped into an issue which would have been very easy to spot, would it be not for the -w build flag.

I think selectively disabling certain warnings known for specific source files will be safer than disabling all the warnings for all the source files.

Opening this issue to track the clean-up and re-enabling of warnings.

wasi exit should exit with return code instead of logging trap errors

If the WASI exit API is invoked, the runtime currently traps and prints an error and ignores the return code. The behavior (at least when not executing in REPL mode) should be to exit the runtime and return the code value used to call.

Here is a half solution that doesn't error and always returns 0. I'm not sure how to correctly read the top of the stack in main:

diff --git a/platforms/app/main.c b/platforms/app/main.c
index 97ead22..f633aa4 100644
--- a/platforms/app/main.c
+++ b/platforms/app/main.c
@@ -273,6 +273,11 @@ int  main  (int i_argc, const char* i_argv[])
             } else {
                 result = repl_call(runtime, argFunc, i_argc, i_argv);
             }
+            if (result && result == m3Err_trapExit) {
+                // TODO: correct thing here
+                // int ret = ((m3reg_t*)(runtime->stack))[runtime->stackSize];
+                return 0;
+            }
             if (result) FATAL("repl_call: %s", result);
         }
     }
diff --git a/source/m3_api_defs.h b/source/m3_api_defs.h
index fb9e7af..56df9f5 100644
--- a/source/m3_api_defs.h
+++ b/source/m3_api_defs.h
@@ -20,6 +20,7 @@
 
 #define m3ApiRawFunction(NAME)     const void * NAME (IM3Runtime runtime, uint64_t * _sp, void * _mem)
 #define m3ApiReturn(VALUE)         { *raw_return = (VALUE); return m3Err_none; }
+#define m3ApiExit(VALUE)           { *raw_return = (VALUE); return m3Err_trapExit; }
 #define m3ApiTrap(VALUE)           { return VALUE; }
 
 #endif // m3_api_defs_h
diff --git a/source/m3_api_wasi.c b/source/m3_api_wasi.c
index 437cb61..ad1639f 100644
--- a/source/m3_api_wasi.c
+++ b/source/m3_api_wasi.c
@@ -623,7 +623,7 @@ m3ApiRawFunction(m3_wasi_unstable_proc_exit)
         fprintf(stderr, M3_ARCH "-wasi: exit(%d)\n", code);
     }
 
-    m3ApiTrap(m3Err_trapExit);
+    m3ApiExit(code);
 }
 
 

Implement stack access API

We need to come up with some stack access API, as currently we can only get function results by tampering with the stack directly. I.e. this has to be revised:

result = m3_CallWithArgs (f, 1, i_argv);
if (result) FATAL("m3_CallWithArgs", result);
long value = *(uint64_t*)(runtime->stack);
Serial.print("Result: ");
Serial.println(value);

Please comment if you have any suggestions on the API.

How to access linear memory

To access non-integer results from WebAssembly modules, I generally first retrieve respectively the location and the length of a result variable as previously allocated to the linear memory of the WebAssembly instance. This enables us to consequently read (or write) to that memory area. For a Golang example, see for instance:

https://github.com/wasmerio/go-ext-wasm#read-the-memory

memory := instance.Memory.Data()
// Reads the memory.
result := memory[pointer : pointer+length]
// now you can parse "result" 

I presume that in wasm3, I would need to access M3Memory? I am, however, unsure how to interface with it. Could you suggest how best to proceed to obtain access to linear memory from wasm3?

Document configuration options

Hi, while working on #38 I did some experiments like building libxml2 as a WASM module.
I'm not able to load it using the current master as it seems to trigger typeListOverflow. Not sure if this is temporary or something related to the specs.

If you want to try it out, you can grab libxml2.wasm from here. In the meantime I'm able to use the module by tweaking d_m3MaxNumFunctionArgs in m3_config.h:

index a10cf61..e5bd84d 100644
--- a/source/m3_config.h
+++ b/source/m3_config.h
@@ -11,7 +11,7 @@
 #include "m3_config_platforms.h"
 
 # ifndef d_m3MaxNumFunctionArgs
-#   define d_m3MaxNumFunctionArgs               16
+#   define d_m3MaxNumFunctionArgs               32
 # endif
 
 # ifndef d_m3CodePageAlignSize

Regards.

Run CoreMark

Bring wasm3 to the level where it can run CoreMark, so we have more reliable numbers of it's performance.
In general we need to:

  • Build CoreMark to WASM
  • Benchmark some popular wasm runtimes using CoreMark
  • Implement the needed imported functions for wasi or emscripten interfaces
  • Improve implementation (mostly by running spec tests and fixing inconsistencies)

Rust bindings

Implement Rust bindings, so wasm3 can be easily used from Rust.

NodeMCU examples throw errors

About two weeks ago, we were able to run several WebAssembly modules on NodeMCU ESP microcontrollers following the esp32-pio and esp8266 platform examples.

In the meanwhile, work on wasm3 has clearly continued - we are particularly happy with the WASI support!

I am, however, now unable to run our WASM modules using the latest code base. For instance, on adapting the esp8266 example as follows:

#include <stdio.h>

#include "Arduino.h"

#define FATAL(msg, ...) { printf("Fatal: " msg "\n", ##__VA_ARGS__); return; }

#include "m3/m3.h" 
#include "m3/m3_env.h"

#include "m3/extra/fib32.wasm.h"

void run_wasm()
{
    M3Result result = c_m3Err_none;

    uint8_t* wasm = (uint8_t*)fib32_wasm;
    size_t fsize = fib32_wasm_len-1;

    IM3Environment env = m3_NewEnvironment ();
    if (!env) FATAL("m3_NewEnvironment failed");

    IM3Runtime runtime = m3_NewRuntime (env, 64*1024, NULL);
    if (!runtime) FATAL("m3_NewRuntime failed");

    IM3Module module;
    result = m3_ParseModule (env, &module, wasm, fsize);
    if (result) FATAL("m3_ParseModule: %s", result);

    result = m3_LoadModule (runtime, module);
    if (result) FATAL("m3_LoadModule: %s", result);
    
    /*result = m3_LinkWASI (runtime->modules);
    if (result) FATAL("m3_LinkWASI: %s", result); 
    
    result = m3_LinkLibC (runtime->modules);
    if (result) FATAL("m3_LinkLibC: %s", result);*/
    
    IM3Function f;
    result = m3_FindFunction (&f, runtime, "fib");
    if (result) FATAL("m3_FindFunction: %s", result);

    const char* i_argv[2] = { "3", NULL };
    result = m3_CallWithArgs (f, 1, i_argv);

    if (result) FATAL("Call: %s", result);

    long value = *(long*)(runtime->stack);

    Serial.println(value);

}

void setup()
{
  Serial.begin(115200);
  delay(10);

  Serial.print("\nwasm3 on ESP8266, build " __DATE__ " " __TIME__ "\n");

  u32 start = millis();
  run_wasm();
  u32 end = millis();

  Serial.print(String("Elapsed: ") +  (end - start) + " ms\n");
}

void loop()
{
  delay(100);
}

... one of our ESP8266's throws the following error:

wasm3 on ESP8266, build Dec 27 2019 18:59:05
Fatal: m3_NewRuntime failed
Elapsed: 6 ms

Also, updating the code of the esp32-pio example along the same lines throws a Guru Meditation error, followed by a boot loop.

Might it be possible to provide a minimal working example that runs, for instance, fib32_wasm on ESP8266 and ESP32 using the latest version of wasm3?

(Separately, linking WASI gives rise to the following error on the ESP8266:
/home/earle/src/esp-quick-toolchain/repo/newlib/newlib/libc/time/clock.c:62: undefined reference to _times_r
... but that is of course a different issue)

GoLang bindings

Implement Go bindings, so wasm3 can be easily used from Go.

Update spec tests to the official W3C 1.0 recommendation

The wast2json tool was updated (WebAssembly/wabt#1275), and we can now switch to the latest spec tests release: https://github.com/WebAssembly/spec/releases/tag/v1.1
I did some preliminary tests and everything looks good except a few minor issues:

  • in call.0.wasm, we now have a function with 100 arguments, so the whole module fails to load due to d_m3MaxNumFunctionArgs=16. Increasing the limit helps. I think we need to allow loading such modules, and fail on the event of function call.
  • names.wast now checks more unicode characters => escaping mechanism has to be updated

This ticket is created to track the progress.

Answers to questions in the README.md

Hello, I was reading the README.md and saw the Thoughts & Questions about WebAssembly section asking for some answers.

Linear memory: Modules seem hardcoded to request 16MB of memory. What? Why?

This is just the default settings of the toolchain produced the modules. Most toolchains have flags for changing the memory size.

As the host, where can I safely allocate things in linear memory? Who's in charge of this 16MB?

In general, you can't. The application is in charge, and hosts shouldn't make any assumptions about how applications manage their memory. Often the way to avoid needing to do this is to ask the application to do an allocation, and pass the VM a pointer to it.

Traps: The spec says "Signed and unsigned operators trap whenever the result cannot be represented in the result type." Really? That's cool, but how can the behavior of source languages be modeled (efficiently)? C integer operations wrap and sometimes that's what you want.

It's admittedly subtle, but wasm's add, sub, and mul aren't considered signed or unsigned, so that rule doesn't apply to them and they wrap rather than trap.

esp32-idf example fails when -DESP32 is defined

I've noticed that esp32-idf example doesn't define -DESP32, so d_m3MaxFunctionStackHeight is not overridden in m3_config_platforms.h:

# elif defined(ESP32)
# define d_m3MaxFunctionStackHeight 256

Once I add -DESP32, the example stops working, and crashes with null pointer dereference at

long value = *(uint64_t*)(runtime->stack);

Not sure how to interpret this, as it was reported in #26 (comment) that esp32-pio example (which I assume defines -DESP32) works okay. I don't have a working PIO environment so can't verify that.

[WASI] implement passing args to main()

Basically, just need to implement 2 wasi functions:

wasm3/source/m3_api_wasi.c

Lines 124 to 145 in be51568

uint32_t m3_wasi_unstable_args_get(IM3Runtime runtime,
uint32_t argv_offset,
uint32_t argv_buf_offset)
{
if (runtime == NULL) { return __WASI_EINVAL; }
// TODO
return __WASI_ESUCCESS;
}
uint32_t m3_wasi_unstable_args_sizes_get(IM3Runtime runtime,
uint32_t argc_offset,
uint32_t argv_buf_size_offset)
{
if (runtime == NULL) { return __WASI_EINVAL; }
__wasi_size_t *argc = offset2addr(runtime, argc_offset);
__wasi_size_t *argv_buf_size = offset2addr(runtime, argv_buf_size_offset);
*argc = 0;
*argv_buf_size = 0;
return __WASI_ESUCCESS;
}

Improve memory bounds checks

Currently, memory addresses in Load and Store operations are cast to u64 to workaround address overflow (i.e. on 32-bit platforms):

wasm3/source/m3_exec.h

Lines 751 to 752 in e40383b

u64 src8 = (u64)_mem + operand + offset; \
u64 end = ((M3MemoryHeader*)(_mem) - 1)->end; \

There are several things we can improve here:

  • Find more efficient way of checking overflows, that would also work for 64-bit address space in future
  • Add compile-time flag to skip those checks (i.e. for benchmarks)

Add support for assert_exhaustion, action in spec tests

Currently, when running run-spec-test.py, assert_exhaustion and action is just skipped.
To support this, we need:

  1. Modify wasm3 to fail gracefully when running out of resources (stack, for now?)
  2. Update run-spec-test.py to run/verify those tests

Test:

./run-spec-test.py ./core/fac.json --line 89

Actual Result:

Warning: Skipped fac.wast:89 assert_exhaustion

Expected Result: test is passed

[spec tests] Check stack state properly

The "Empty Stack" case is not correctly verified by the test script.
Currently "Empty Stack" is printed every time when the function has no return values (but this is not an indicator of an empty stack). I think we can add a new repl command to print the actual stack state.

[WASI] implement passing environment variables

Basically, just need to implement 2 wasi functions:

wasm3/source/m3_api_wasi.c

Lines 147 to 166 in be51568

uint32_t m3_wasi_unstable_environ_get(IM3Runtime runtime,
uint32_t environ_ptrs_offset,
uint32_t environ_strs_offset)
{
if (runtime == NULL) { return __WASI_EINVAL; }
// TODO
return __WASI_ESUCCESS;
}
uint32_t m3_wasi_unstable_environ_sizes_get(IM3Runtime runtime,
uint32_t environ_count_offset,
uint32_t environ_buf_size_offset)
{
if (runtime == NULL) { return __WASI_EINVAL; }
__wasi_size_t *environ_count = offset2addr(runtime, environ_count_offset);
__wasi_size_t *environ_buf_size = offset2addr(runtime, environ_buf_size_offset);
*environ_count = 0; // TODO
*environ_buf_size = 0; // TODO
return __WASI_ESUCCESS;
}

getrandom(2) does not exist in Ubuntu 16.04

In Ubuntu Xenial 16.04 it does not have /usr/include/sys (there is a /usr/include/linux/random.h, but no getrandom defined)

It appears in Bionic (18.04) however.

What is the minimum version of glibc is recommended?

Do you intend to support 16.04 ?

Example interacting with hardware peripherals

All the platforms use the same fib32.wasm.h but I haven't found an example that interacts with the hardware peripherals like GPIO. Is that possible, even if with a bit of wasm<>C++ glue code? I could imagine something like wasm-bindgen be helpful here, or Interface Types when that's ready. But for now blinking an LED would be great :)

Self-hosting

Bring wasm3 to a level where it can run itself. JFF ;)

Actions list:

  • Improve interpreter so it can execute itself
  • Build in 32-bit mode (normal native x86 build) and fix spec tests
  • Build wasm3.wasm
  • Improve test script so we can run spec tests on wasm3.wasm
  • Fix spec tests for wasm3.wasm
  • Fix imported (native) function calls
  • Implement MetaWASI (redirect WASI calls to the host WASI environment)
  • Run WASI apps in self-hosted mode
  • Run spec tests in self-hosted mode

Easy embedding in other programs

I have been trying to embed WASM in vim as default. I had send one PR on embedding JS to vim vim/vim#5198.

Is it possible to just have 2 files wasm3.h and wasm3.c in the output similar to how duktape does it to make it easy to embed in other programs? This way I can just #include "wasm3.h" and don't have to worry about it.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.