Code Monkey home page Code Monkey logo

wasmtime-py's Introduction

wasmtime-py

Python embedding of Wasmtime

A Bytecode Alliance project

CI status Latest Version Latest Version Documentation Code Coverage

Installation

To install wasmtime-py, run this command in your terminal:

$ pip install wasmtime

The package currently supports 64-bit builds of Python 3.8+ on x86_64 Windows, macOS, and Linux, as well as on arm64 macOS and Linux.

Versioning

wasmtime-py follows the Wasmtime versioning scheme, with a new major version being released every month. As with Wasmtime itself, new major versions of wasmtime-py can contain changes that break code written against the previous major version.

Since every installed Python package needs to agree on a single version of wasmtime-py, to use the upper bound on the major version in the dependency requirement should be bumped reguarly, ideally as soon as a new wasmtime-py version is released. To automate this process it is possible to use the whitequark/track-pypi-dependency-version script. YoWASP/runtime is an example of a project that automatically publishes releases on PyPI once a new version of wasmtime-py is released if it passes the testsuite.

Usage

In this example, we compile and instantiate a WebAssembly module and use it from Python:

from wasmtime import Store, Module, Instance, Func, FuncType

store = Store()
module = Module(store.engine, """
  (module
    (func $hello (import "" "hello"))
    (func (export "run") (call $hello))
  )
""")

def say_hello():
    print("Hello from Python!")
hello = Func(store, FuncType([], []), say_hello)

instance = Instance(store, module, [hello])
run = instance.exports(store)["run"]
run(store)

Be sure to check out the examples directory, which has other usage patterns as well as the full API documentation of the wasmtime-py package.

If your WebAssembly module works this way, then you can also import the WebAssembly module directly into Python without explicitly compiling and instantiating it yourself:

# Import the custom loader for `*.wasm` files
import wasmtime.loader

# Assuming `your_wasm_file.wasm` is in the python load path...
import your_wasm_file

# Now you're compiled and instantiated and ready to go!
print(your_wasm_file.run())

Components

The wasmtime package has initial support for running WebAssembly components in Python with high-level bindings. WebAssembly components are defined by the component model and are a flagship feature under development for Wasmtime and its bindings. Components enable communication between the host and WebAssembly guests with richer types than the numerical primitives supported by core WebAssembly. For example with a component Python can pass a string to wasm and back.

Components are represented as *.wasm binaries in the same manner as core WebAssembly modules. With a component binary you can generate Python bindings with:

$ python -m wasmtime.bindgen the-component.wasm --out-dir the-bindings

An example of using this can be done with the wasm-tools repository. For example with this core wasm module at demo.wat:

(module
  (import "python" "print" (func $print (param i32 i32)))
  (memory (export "memory") 1)

  (func (export "run")
    i32.const 100   ;; base pointer of string
    i32.const 13    ;; length of string
    call $print)

  (data (i32.const 100) "Hello, world!")
)

and with this *.wit interface at demo.wit:

package my:demo

world demo {
  import python: interface {
    print: func(s: string)
  }

  export run: func()
}

And this demo.py script

from demo import Root, RootImports, imports
from wasmtime import Store

class Host(imports.Python):
    def print(self, s: str):
        print(s)

def main():
    store = Store()
    demo = Root(store, RootImports(Host()))
    demo.run(store)

if __name__ == '__main__':
    main()
$ wasm-tools component embed demo.wit demo.wat -o demo.wasm
$ wasm-tools component new demo.wasm -o demo.component.wasm
$ python -m wasmtime.bindgen demo.component.wasm --out-dir demo
$ python demo.py
Hello, world!

The generated package demo has all of the requisite exports/imports into the component bound. The demo package is additionally annotated with types to assist with type-checking and self-documentation as much as possible.

Contributing

See CONTRIBUTING.md.

wasmtime-py's People

Contributors

adammohammed avatar alexcrichton avatar cclauss avatar danbev avatar dependabot[bot] avatar dicej avatar esoterra avatar fitzgen avatar glebpom avatar guybedford avatar hoodmane avatar igrr avatar jbourassa avatar john-g-g avatar kulakowski-wasm avatar mossaka avatar muayyad-alsadi avatar niraj-kamdar avatar pchickey avatar sunfishcode avatar thetianshuhuang avatar thewtex avatar tkat0 avatar viniarck avatar vrandezo avatar whitequark avatar willkg avatar womeier avatar zifeo avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

wasmtime-py's Issues

Running WASI binaries

Let me start off by saying how much I like wasmtime. I always liked the concept, but six months ago the implementation wasn't quite there. I was pretty sure it'll get better, and today I ran the exact same binary I was trying to run back then in no time and it works better than my most optimistic expectations. Right now it seems like wasmtime is going to solve the single biggest obstacle new nMigen developers encounter, especially on Windows but elsewhere too.

That aside, is there any way I'm missing to run WASI binaries with wasmtime-py? I really only need stdin and stdout, though the binary I'm interested in, I think, is expecting to be able to load from $PREFIX/share/ so some sort of virtualized fileystem would be ideal.

Is this something I can easily implement myself? (I'm only really familiar with WASM/WASI as a toolchain user, not a library developer.) Or, is it something in scope for wasmtime-py? Do I have to implement every single function WASI provides, or just the ones my application calls?

Performance questions

Incredible work! I was trying to see what kind of overhead there is to call a wasm function from Python.
I am using WSL2 on Windows, with a recent Fedora, CPython3.10.
Re-using the exact gcd.wat from the examples, it looks like any call has a "cost" of about 30μs.

The code I used is a simple timer to compare performance:

import wasmtime.loader
import time
from math import gcd as math_gcd
from gcd import gcd as wasm_gcd

def python_gcd(x, y):
    while y:
        x, y = y, x % y
    return abs(x)

N = 1_000
for gcdf in math_gcd, python_gcd, wasm_gcd:
    start_time = time.perf_counter()
    for _ in range(N):
        g = gcdf(16516842, 154654684)
    total_time = time.perf_counter() - start_time
    print(f"{total_time / N * 1_000_000:8.3f}μs")

This returns about:

   0.152μs  # C code from math.gcd
   0.752μs  # Python code from python_gcd
  31.389μs  # gcd.wat

Note that I tested this with an empty "hello world", and the 30μs are still there.
I am wondering about 2 things:

  • is this overhead inevitable and linked to the design, or are there ways to reduce it?
  • I noticed in the gcd.wat code that input parameters are expected to be i32, but the code does not fail when parameters exceed 2**32, so I was wondering how this works

Env variables are not being passed to WASM

I'm not having any success getting environment variables in the WASM environment. My code is below. There is a function written in C compiled to WASM that does a getenv("FOO") which is NULL. I've also tried using wasi.inherit_env(), but that doesn't work for me either.

import os
import struct
import sys
import wasmtime
from bindings import Udf

store = wasmtime.Store()

module = wasmtime.Module(store.engine, open('funcs.wasm', 'rb').read())

linker = wasmtime.Linker(store.engine)
linker.define_wasi()

wasi = wasmtime.WasiConfig()

# Set mounts
wasi.preopen_dir(os.path.join(os.getcwd()), '/')

# Set stdin / stdout / stderr
wasi.inherit_stdin()
wasi.inherit_stdout()
wasi.inherit_stderr()

# Set command-line args
wasi.argv = ['python']

# Set environment variables
wasi.env = dict(FOO='bar').items()

store.set_wasi(wasi)

udf = Udf(store, linker, module)
udf.call_function(store)

MinGW/MSYS compatible binary wheels

Unexpectedly, the existing wasmtime binary wheels don't work on MinGW/MSYS 64-bit Python. It seems to me that, since wasmtime-py doesn't depend on the Python ABI, they ought to work, but I have no deep understanding of the machinery involved.

Type annotations for method signatures

I'd like to see type annotations throughout the code, especially in function and method signatures, so I can see what's needed where, and so users and other developers will be able to see what's needed where. Sphinx (for generating documentation) can, if we're lucky, use these when it's showing the signature of methods.

I've gone through and done this in the code, but the commit isn't ready yet and I wanted to to see if #17 would be merged, if at all, as I might need to make a case for this ticket just as I do for that one.

I also drew a big diagram of all the classes, using PlantUML, as I went. Doing these at the same time is, I find, usually a very sensible idea. It took quite a bit a work to figure out what was needed where, including much use of the debugger. I think there's a new Python tool to help do this that Guido worked on, but I find that doing it manually makes for a good way of getting to know the code.

Using file objects (or equivalent) for stdio streams

Now that #33 is here, I understand that it's only possible to open a file on the host filesystem as an stdio stream. But that's (IMO) rarely useful: most WASI programs probably can accept data via files on the command line just as well as via stdio.

What I'd like to do is to capture stdout/stderr of a WASI application. Sure, I can do this by running a new host subprocess (with some caveats) but it would be a lot nicer if wasmtime-py provided this capability directly.

Handling the exit trap

Since wasmtime-py 0.18.0, it seems that when the main() function returns, the call to _start() also simply returns, which is good. However, it looks like if exit() is called, then the result is a wasmtime._trap.Trap: Exited with i32 exit status 1. It is not clear how this trap can be handled besides matching the exception message with a regex.

Update wasm tools to 1.0.20

On the latest versions of wit-bindgen, wasm-tools version of 1.0.20 is used.

Can wasmtime-py be updated to that version as well?

This mis-versions breaks our setup, where we use wasm components for our js and python part of stack.

[question?] Is it possible to pass a borrowed list from Python?

I wrote a lib with Rust, who has a function as below:

#[no_mangle]
pub fn cpd(signal: &[f64], jump: usize, p_value: f64, permutations: usize) -> Vec<usize> {
    //...
}

I compiled it to wasm32-wasi, and call cpd function from Python:

  1. I tried as below and got an error, which said wasmtime._error.WasmtimeError: too few parameters provided: given 4, expected 6
import wasmtime.loader
import mylib

a = [1.,2.,3.,4.,5.]

print(mylib.cpd(a, 5, 0.05, 1000))
  1. Then I found this information Exports: Functions: "cpd": [I32, I32, I32, I32, F64, I32] -> []. I guess &[f64] may be converted to a fat pointer. But the code below didn't work too and got en error as wasmtime._trap.Trap: wasm trap: out of bounds memory access
import wasmtime.loader
import mylib

a = [1.,2.,3.,4.,5.]

address = id(a)
length = len(a)
capacity = length

print(mylib.cpd(address, length, capacity, 5, 0.05, 1000))

Question

So are there any ideas to solve this problem? TIA

wasmtime arm64 wheel for mac m1

Even though arm support has landed a year ago (bytecodealliance/wasmtime#339), it seems it has not found its way into a macOS AArch64 pip wheel:

Successfully installed wasmtime-0.33.0

from wasmtime import *

"unsupported architecture for wasmtime: arm64"

(macOS Monterey 12.1 M1, Python 3.8.9)

use pytest or not use pytest?

In the README it suggests to use pytest for test discovery. pytest has a bunch of other features, but some of those features don't work when test classes subclass unittest.TestCase.

If wasmtime-py wanted to use pytest, then I would throw together a PR that makes the following changes:

  1. remove instances of subclassing unittest.TestCase
  2. switch from unittest assert methods to assert x == y statements
  3. switch to pytest.raises

If wasmtime-py wanted to use unittest, I could throw together a PR that fixes setup.py so that python setup.py test works.

Any preference?

qjs.wasm loops forever

Hello,

I'm trying to run qjs.wasm with wasmtime-py. The compilation step works great, but the execution loops forever, and I don't know why. We quickly tried to debug that with @bnjbvr with no obvious solution so far.

You can get qjs.wasm from https://wapm.io/package/quickjs#explore. Here is the test script.

import wasmtime

TEST_BYTES = open('qjs.wasm', 'rb').read()

config = wasmtime.Config()
config.strategy = 'auto'
engine = wasmtime.Engine(config)
store = wasmtime.Store(engine)

wasi_config = wasmtime.WasiConfig()
wasi_config.argv = ['--help']
wasi = wasmtime.WasiInstance(store, 'wasi_unstable', wasi_config)

module = wasmtime.Module(wasi.store.engine, TEST_BYTES)

imports = []

for import_ in module.imports:
    imports.append(wasi.bind(import_))

instance = wasmtime.Instance(store, module, imports)

a = instance.exports["_start"]
a()

Ability to set a file-like object (instead of a file path) for stdin and stderr

I want to use wasmtime-py to run untrusted Python code safely inside a WASM sandbox. Here's the recipe I have so far, thanks to help from @pims https://til.simonwillison.net/webassembly/python-in-a-wasm-sandbox

Part of that recipe is this:

config = WasiConfig()

config.argv = ("python", "-c", code)
config.preopen_dir(".", "/")

with tempfile.TemporaryDirectory() as chroot:
    out_log = os.path.join(chroot, "out.log")
    err_log = os.path.join(chroot, "err.log")
    config.stdout_file = out_log
    config.stderr_file = err_log

    store = Store(linker.engine)

    # Limits how many instructions can be executed:
    store.add_fuel(fuel)
    store.set_wasi(config)

Then the code reads from that out.log temporary file after executing the code.

It would be much neater if it was possible to pass a file-like object here instead - a io.BytesIO or io.StringIO for example.

Capturing output from WASI binary (produced by TinyGo 0.26.0)

I'd like to use a WASI binary produced by TinyGo (0.26.0) in Python capturing the output from an exported function to a variable.

Specifically the parse method defined here:

https://github.com/sfomuseum/go-edtf/blob/wasi/cmd/parse-wasi/main.go

Which I can build and test like this:

$> tinygo build -o parse.wasm -target wasi ./cmd/parse-wasi/main.go

$> wasmtime parse.wasm 2022-01-01
2022-01-01

At this point the Go/WASM binary just parrots back input pending some decisions about how to work around parts of the Go stdlib that aren't available in TinyGo yet.

If I run the following Python code (in a file called test.py):

#!/usr/local/bin/python3

from wasmtime import Store, Module, Instance, Func, FuncType

if __name__ == "__main__":

    store = Store()
    module = Module.from_file(store.engine, "parse.wasm")

    instance = Instance(store, module, [])
    parse = instance.exports(store)["parse"]

    print(parse("2022-01-01"))

It fails with the following error:

Traceback (most recent call last):
  File "/usr/local/sfomuseum/go-edtf/./test.py", line 10, in <module>
    instance = Instance(store, module, [])
  File "/usr/local/lib/python3.10/site-packages/wasmtime/_instance.py", line 43, in __init__
    raise WasmtimeError._from_ptr(error)
wasmtime._error.WasmtimeError: expected 14 imports, found 0

I feel like I am missing something obvious but I am not sure what, exactly. Can you spot what I am doing wrong?

Compilation error: Implementation limit exceeded

Some people get this error while instantiating a WASM module, all of them on Windows, e.g. see here. Unfortunately it is quite opaque and I'm not really sure where to go from here, especially since wasmtime-py ignores RUST_LOG. How do I debug this?

Proposal: Wasmtime 1.0

I wanted to raise awareness in this repository about the proposal for Wasmtime to become 1.0. If others are watching just this repository and are interested to learn what 1.0 would mean for Wasmtime and this embedding, feel free to read over the RFC and leave comments!

Python idioms in the API

The README mentions that:

So far this extension has been written by folks who are primarily Rust programmers, so it's highly likely that there's some faux pas in terms of Python idioms. Feel free to send a PR to help make things more idiomatic if you see something!

Indeed, after using the API for a while I noticed a few aspects that aren't quite in line with Python conventions, though some of them are harder to ignore than others.

  1. Nullary getter methods like ExportType.name that always succeed and always return the same value are a pretty clear-cut case for using Python properties instead. This is, by far, the least idiomatic aspect of the library, and it's very easy to miss a () that feels like it shouldn't be there.
  2. There are lots of unary setter methods like Config.debug_info named after the property they're changing, and there is a good case for using Python setter-only properties instead. (As a bonus, if wasi-c-api ever adds getters for them, they could be made readable in a backwards compatible way.) This doesn't change much for e.g. Config, but for Linker for example allow_shadowing works like a simple setter with no side effects, but define_wasi, while similarly imperatively named and used, irreversibly changes its state.
  3. Nullary casting methods like ExternType.func_type feel very odd overall.
    1. They're named like accessors, but they work like dynamic_cast.
    2. The naming is inconsistent for conversion in different directions—FuncTypeExternType conversion is done by as_extern, but ExternTypeFuncType conversion is performed by func_type.
    3. My impression so far is that most uses of these casting methods are in contexts where a specific type is expected, in which case it would be more convenient if they raised TypeError instead of returning None. (If they did, another method would be necessary to dispatch on an unknown type.)
  4. Val.get_* introduces a third naming scheme for type conversion functions, one that clashes in meaning with e.g. Instance.get_export.
  5. *.get_export() methods seem like they would only fail in truly exceptional conditions and so also appear good candidates for raising an exception instead of returning None.

To summarize:

  1. I would change simple getters and setters to be properties.
  2. I would rename all casting methods to as_*().
  3. I would change *.get_export() to raise if the export is not found (this is particularly important for Caller.get_export(), since currently it looks like there is no way to distinguish the cases of "export does not exist" and "the caller is gone" at all, even for debugging, and the exception message would allow that).
  4. I would probably change the dynamic casting methods to raise if the type is wrong, and add a matching is_* method for the cases where dynamic dispatch is indeed necessary. I'm not completely certain about this one, but it seems that currently, dynamic dispatch would look something like:
if extern.memory():
    do_something_with_memory(extern.memory())
elif extern.table():
    do_something_with_table(extern.table())

so changing it to:

if extern.is_memory():
    do_something_with_memory(extern.as_memory())
elif extern.is_table():
    do_something_with_table(extern.as_table())

doesn't really make it any worse.

What do you think? All of these changes are likely to break most existing code, so I'd rather discuss them before working on a PR.

Also, I should mention that "do nothing" is an option. In general Python libraries are a lot looser with conventions than libraries in many other languages; I'd rather see the API improved, personally, but the current one isn't drastically more inconsistent than what one would expect in general.

No documentation on setter properties

Looks like I missed one aspect of the wrapper. This should work:

def setter_property(fset):
    prop = property(fset=fset)
    prop.__doc__ = fset.__doc__
    return prop

Perhaps also add a note about the setter-only nature of the property to the documentation in the decorator?

Change default branch name

As a policy, the Bytecode Alliance is changing the default branch names in all repositories. We would like for all projects to change the default to main by June 26. (We mention June 26th because there is some suggestion that GitHub may be adding the ability to make this process more seamless. Feel free to wait for that, but only up to June 26. We'll provide further support and documentation before that date.)

Please consider this a tracking issue. It is not intended for public debate.

  • Change branch name
  • Update CI
  • Update build scripts
  • Update documentation

demo example does not work

So I have these three files:

  1. demo.wat
  2. demo.wit
  3. demo.py

Then, wasm-tools component new demo.wat --wit demo.wit -o demo.wasm creates demo.wasm as expected but python -m wasmtime.bindgen demo.wasm --out-dir demo generates the following error:

$ python3 -m wasmtime.bindgen demo.wasm --out-dir demo
Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/<path>/.local/lib/python3.10/site-packages/wasmtime/bindgen/__main__.py", line 40, in <module>
    main()
  File "/<path>/.local/lib/python3.10/site-packages/wasmtime/bindgen/__main__.py", line 30, in main
    files = generate(name, contents)
  File "/<path>/.local/lib/python3.10/site-packages/wasmtime/bindgen/__init__.py", line 40, in generate
    raise RuntimeError(result.value)
RuntimeError: failed to extract interface information from component

Caused by:
    invalid leading byte (0x1) for component external kind (at offset 0x25)

Could you please give me a hint how to make the initial example work?

wasmtime-py 0.16.1 regression

Unfortunately since 0.16.1 it looks like every binary crashes on exit:

Traceback (most recent call last):
  File "/home/whitequark/.local/bin/yosys", line 11, in <module>
    load_entry_point('yosys', 'console_scripts', 'yosys')()
  File "/home/whitequark/Projects/yosys-pypi/yosys/__init__.py", line 57, in _run_yosys_argv
    run_yosys(sys.argv)
  File "/home/whitequark/Projects/yosys-pypi/yosys/__init__.py", line 55, in run_yosys
    _run_wasm_app("yosys.wasm", argv)
  File "/home/whitequark/Projects/yosys-pypi/yosys/__init__.py", line 51, in _run_wasm_app
    app.exports["_start"]()
  File "/home/whitequark/.local/lib/python3.7/site-packages/wasmtime/_func.py", line 120, in __call__
    raise Trap.__from_ptr__(trap)
wasmtime._trap.Trap: explicitly exited

Publish wheels to GitHub releases

Before splitting out wasmtime-py into its own repository, we also published the wheels to the Wasmtime releases. It'd be nice if we could do that again in this repo.

No module named 'wasi_snapshot_preview1'

Hi there! I'm testing out wasmtime-py using this simple C program:

#include <stdio.h>

int main() {
  printf("test\n");
  return 0;
}

I compiled it to .wasm using Emscripten (emcc hello.cpp -O3 -o hello.wasm -s STANDALONE_WASM) and can run it with wasmtime:

$ wasmtime hello.wasm
test

But when I try importing it in Python, I get this error:

In [1]: import wasmtime.loader                                                                                                                                

In [2]: import hello                                                                                                                                          
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-2-f81fb083bdeb> in <module>
----> 1 import hello

/usr/local/lib/python3.5/dist-packages/wasmtime/loader.py in exec_module(self, module)
     64             module_name = wasm_import.module
     65             field_name = wasm_import.name
---> 66             imported_module = importlib.import_module(module_name)
     67             item = imported_module.__dict__[field_name]
     68             if not isinstance(item, Func) and \

/usr/lib/python3.5/importlib/__init__.py in import_module(name, package)
    124                 break
    125             level += 1
--> 126     return _bootstrap._gcd_import(name[level:], package, level)
    127 
    128 

ImportError: No module named 'wasi_snapshot_preview1'

which I'm guessing is because of this:

  (import "wasi_snapshot_preview1" "fd_write" (func (;0;) (type 7)))
  (import "wasi_snapshot_preview1" "proc_exit" (func (;1;) (type 3)))

Since it works when I run the wasmtime CLI, am I missing some imports in Python to make this work?

(I also tried this with an add example in C that doesn't need any external libraries and import add worked without error).

Here's the WAT file in case it helps: https://gist.github.com/robertaboukhalil/173d450b9d841386ad362e05dd5bc747

Run wasm file like the wasmtime cli would

I'd like to get the same behaviour as the wasmtime foo.wasm cli, or rather i'd like to have the rust program i compiled to wasm behave as similar to native execution as possible. This is the code i currently use:

from wasmtime import Store, Module, Engine, WasiConfig, Linker

engine = Engine()
store = Store(engine)
wasi = WasiConfig()
wasi.inherit_argv()
wasi.inherit_env()
wasi.inherit_stdout()
wasi.inherit_stderr()
wasi.inherit_stdin()
store.set_wasi(wasi)
linker = Linker(engine)
linker.define_wasi()
module = Module.from_file(store.engine, 'foo.wasm')
linking1 = linker.instantiate(store, module)
start = linking1.exports(store).get("") or linking1.exports(store)["_start"]
start(store)

Is there a way with less boilerplate to get wasmtime cli like behaviour or are there any additional options i need to set?

AttributeError: unreadable attribute

Hello,

I'm getting this error (full traceback)

Traceback (most recent call last):
  File "main_wasmtime.py", line 4, in <module>
    config.wasm_multi_memory(True)
AttributeError: unreadable attribute

when I run the following code:

from wasmtime import Config

config = Config()
config.wasm_multi_memory(True)
config.wasm_threads(True)

I need the threads enabled for a wasm file I'm working with, however it doesn't look like it can be set using the wasmtime-py API correctly.

My environment is:

  • Python 3.9.7
  • Wasmtime 0.39.1
  • Ubuntu 20.04 (Linux e127538 5.13.0-44-generic 49~20.04.1-Ubuntu SMP Wed May 18 18:44:28 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux)

Am unsure how to proceed as I can't think of a workaround.

Exit traps crash on Windows with STATUS_BAD_STACK

Original report at YoWASP/yosys#1.

To reproduce the crash, do the following on Windows x64 with Python installed:

  • pip3 install yowasp-yosys
  • yowasp-yosys -V

I haven't personally verified it but I believe that any program that consists of just exit(0); will exhibit the same issue. Let me know if you'd like to have this minimized further or if this is sufficient.

Component bindings should implement `may_enter` and `may_leave` flags

Currently these are entirely unimplemented and are features of the component model that need to be added to verify the validity of components in the face of traps. The implementations of these flags would be in the generated lifting/lowerings in the same manner as dictated by CanonicalABI.md

invalid leading byte (0x3) for component external kind (at offset 0x364cf9)

Hey, I'm trying newer versions of the component model and I'm encountering an issue when generating bindings for python. Running the command python3 -m wasmtime.bindgen build/dynomite.wasm --out-dir build/tmp/py yields the error

RuntimeError: failed to extract interface information from component

Caused by:
    invalid leading byte (0x3) for component external kind (at offset 0x364cf9)

@alexcrichton said in bytecodealliance/wasm-tools#842 that this is likely due to a recent binary format change. I do however get the same error using both wasmtime-py 3.0.0 and main branch (4.0.0). The rest of the tools work, creating the component and creating js bindings with wit-bindgen, but wasmtime.bindgen with python does produce bindings.

Setting a slice of memory without having to enumerate

It's nice that the WASM memory can be inspected with a slice operator:

from wasmtime import *
engine = Engine()
store = Store(engine)
module = Module.from_file(engine, "./hello.wasm")
instance = Instance(store, module, [])
exports = instance.exports(store)
memory = exports["memory"]

Then

>>> memory.data_ptr(store)[1024:1029]
[114, 97, 116, 115, 0]

but it's not possible to set it that way which seems like a bug, or at least a missing feature:

>>> data = memory.data_ptr(store)
>>> data[1024:1028] = [114, 97, 116, 116]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: sequence index must be integer, not 'slice'

You can work around it by iterating:

for i,v in enumerate("fats".encode('utf-8')): data[i] = v

Caching compiled code

At the moment it seems that even though wasmtime itself implements a very nice caching system that makes startup latency almost disappear in my application, wasmtime-py does not use it. Is this intentional, or is this just an oversight?

For now I was able to work around that with an intrusive patch that cuts down the application size to under 2 MB, which takes about 2 seconds to compile on a modern laptop, but that's a pretty significant limitation and I'm worried about the increase in latency on low-end hardware.

Implementing an interposer

I wanted to use wasmtime-py to implement a tool similar to ltrace that intercepts WASI calls. Here's the key part of my code:

def trace(name, orig):
	def trace_fn(*args):
		rets = orig(*args)
		print("{}{!r}->{!r}\n".format(name, args, rets))
		return rets
	return trace_fn
wasi = wasmtime.WasiInstance(store, "wasi_snapshot_preview1", wasi_cfg)
imports = []
for import_ in module.imports():
	if import_.module() == "wasi_snapshot_preview1":
		export = wasi.bind(import_)
		import_type = import_.type().func_type()
		wrapper = wasmtime.Func(store, import_type, trace(import_.name(), export))
		imports.append(wrapper.as_extern())
instance = wasmtime.Instance(module, imports)
instance.get_export("_start")()

However, this crashes almost instantly with a nested trap. I think this happens because the WASI implementation is internally using something like access_caller=True in order to access the memory, since the pointer-like WASI arguments are just indexes into that. But it seems like there's no way to pass the Caller along.

Is there a good way to implement such functionality regardless?

generated vars use common names

In some cases (such as python), the generated bindings make use of local variables such as "i", "len", and "base". If you use these names in your WIT file as arguments, then you get an error. For example, consider the following WIT file:

power-of: func(base: s32, exp: s32) -> s32

Then try to generate bindings like this and notice the output:

% wit-bindgen wasmtime-py --import power.wit
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: "name `base` already defined"', crates/gen-wasmtime-py/src/lib.rs:868:34

The generated code should use more obfuscated names to reduce the potential for this kind of naming collision.

Component binding class naming issue

I was playing around the component model demo example listed in README. One issue I found with that is the python class Demo name is generated from the wasm file name. This is error-prone because I could rename the wasm name to demo.component.wasm and the generated Demo.component class name in Python is a syntax error.

How to reproduce

Go through the first demo example under the "Components" section. Change the commands to the following

$ wasm-tools component new demo.wat --wit demo.wit -o demo.component.wasm
$ python -m wasmtime.bindgen demo.component.wasm --out-dir demo

Notice how "demo.wasm" is replaced with "demo.component.wasm".

Then notice that the generated python file contains syntax error because the Demo class name is now replaced with Demo.component class name in demo/__init__.py

how to import wasm env function?

I use emcc to compile a wasm file.
the wasm file import many external functions, such as "env"."memset", env.memcpy and so on. How can I correctly import those functions. whether there are some base file or lib that I can directly import them?
I have read examples, no otherelse complex examples.
thanks for your reading. I hope get your help. thanks.

wrong function called from module when using wasmtime loader

using wasm_module.wat:

(module
  (type (;0;) (func (result i32)))
  (func $func1 (type 0) (result i32)
    i32.const 1)
  (func $func2 (type 0) (result i32)
    i32.const 2)
  (table (;0;) 1 1 funcref)
  (memory (;0;) 16)
  (global (;0;) (mut i32) (i32.const 1048576))
  (global (;1;) i32 (i32.const 1048576))
  (global (;2;) i32 (i32.const 1048576))
  (export "memory" (memory 0))
  (export "func1" (func $func1))
  (export "func2" (func $func2))
  (export "__data_end" (global 1))
  (export "__heap_base" (global 2)))

when loaded as a python module, calling func1 returns 2 instead of 1.
this is because func2 is called instead, can be verified by modifying func2

tested with this script:

from wasmtime import Store, Module, Instance, Func, FuncType, loader
import wasm_module

store = Store()
module = Module(store.engine, open("wasm_module.wat").read())

instance = Instance(store, module, [])
func1 = instance.exports(store)["func1"]
assert func1(store) == wasm_module.func1(), f"from instance {func1(store)}, from loader {wasm_module.func1()}"

the error message is

Traceback (most recent call last):
  File "/home/user/Documents/test/test.py", line 9, in <module>
    assert func1(store) == wasm_module.func1(), f"from instance {func1(store)}, from loader {wasm_module.func1()}"
AssertionError: from instance 1, from loader 2

tested with wasmtime-0.29.0.dev168

Querying the cache

Especially for larger applications, the cold startup time can be quite long. As a result people end up unsure whether the application is compiling or just hung. It would be nice if there was some way to check if I'm about to hit the cache, so that if I'm going to miss, I could print a message of some sort to stderr.

wasmtime.loader: import error when importing a WASM module which has imports

OS: Windows (10 64-bit), Python 3.9.5, Wasmtime-py 0.35.0
Steps to reproduce:
You need a Rust toolchain (make sure to "rustup target add wasm32-unknown-unknown wasm32-wasi") and the wasm-pack tool


C:\py\git>git clone https://github.com/httptoolkit/brotli-wasm
Cloning into 'brotli-wasm'...
remote: Enumerating objects: 84, done.
Receiving objects: 100% (84/84), 25.53 KiB | 2.32 MiB/s, done.(13/84)
Resolving deltas: 100% (37/37), done.)
remote: Counting objects: 100% (84/84), done.
remote: Compressing objects: 100% (53/53), done.
remote: Total 84 (delta 37), reused 63 (delta 21), pack-reused 0

C:\py\git>cd brotli-wasm

C:\py\git\brotli-wasm>wasm-pack build
[INFO]: Checking for the Wasm target...
[INFO]: Compiling to Wasm...
   Compiling proc-macro2 v1.0.36
   Compiling unicode-xid v0.2.2
   Compiling syn v1.0.86
   Compiling wasm-bindgen-shared v0.2.79
   Compiling log v0.4.14
   Compiling cfg-if v1.0.0
   Compiling serde_derive v1.0.136
   Compiling bumpalo v3.9.1
   Compiling serde v1.0.136
   Compiling lazy_static v1.4.0
   Compiling serde_json v1.0.79
   Compiling wasm-bindgen v0.2.79
   Compiling alloc-no-stdlib v2.0.3
   Compiling itoa v1.0.1
   Compiling ryu v1.0.9
   Compiling wee_alloc v0.4.5
   Compiling cfg-if v0.1.10
   Compiling memory_units v0.4.0
   Compiling alloc-stdlib v0.2.1
   Compiling brotli-decompressor v2.3.2
   Compiling quote v1.0.15
   Compiling brotli v3.3.3
   Compiling wasm-bindgen-backend v0.2.79
   Compiling wasm-bindgen-macro-support v0.2.79
   Compiling wasm-bindgen-macro v0.2.79
   Compiling console_error_panic_hook v0.1.7
   Compiling brotli-wasm v1.0.0 (C:\py\git\brotli-wasm)
    Finished release [optimized] target(s) in 55.75s
[INFO]: Installing wasm-bindgen...
[INFO]: Optimizing wasm binaries with `wasm-opt`...
[INFO]: :-) Done in 1m 11s
[INFO]: :-) Your wasm pkg is ready to publish at C:\py\git\brotli-wasm\pkg.

C:\py\git\brotli-wasm>cd pkg

C:\py\git\brotli-wasm\pkg>python
Python 3.9.5 (tags/v3.9.5:0a7dcbd, May  3 2021, 17:27:52) [MSC v.1928 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import wasmtime.loader
>>> import brotli_wasm_bg
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "C:\python\lib\site-packages\wasmtime\loader.py", line 72, in exec_module
    imported_module = importlib.import_module(module_name)
  File "C:\python\lib\importlib\__init__.py", line 122, in import_module
    raise TypeError(msg.format(name))
TypeError: the 'package' argument is required to perform a relative import for './brotli_wasm_bg.js'
>>> exit()

C:\py\git\brotli-wasm\pkg>

Is this library any good for using wasm libs in python?

I made a very minimal example:

Rust code compiled to wasm32-wasi

use wasm_bindgen::prelude::*;

#[wasm_bindgen]
pub fn hello()->String{
    String::from("Hello World!")
}

Python code to use it

>>> from wasmtime import loader
>>> import my_test
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.9/site-packages/wasmtime/loader.py", line 71, in exec_module
    imported_module = importlib.import_module(module_name)
  File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ModuleNotFoundError: No module named '__wbindgen_placeholder__'

Is it possible at all to use this wasmtime-py for calling functions out of a wasi library?

In the Python generator, create aliases for numeric types

When users are consuming generated code in Python, they are not able to tell which integer type a given int was or what size a float was. We can make this somewhat easier by introducing aliases for used numeric types that preserve that type info.

e.g.

def foo(a: int) -> int:
   ...

becomes

s32 = int
s64 = int

def foo(a: s32) -> s64:
   ...

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.