wasilibs / nottinygc Goto Github PK

View Code? Open in Web Editor NEW

60.0 4.0 5.0 708 KB

Higher-performance allocator for TinyGo WASI apps

License: MIT License

Dockerfile 3.38% Go 96.62%

bdwgc tinygo webassembly

nottinygc's Introduction

nottinygc

nottinygc requires TinyGo 0.28+

nottinygc has reached end-of-life. The Go compiler will support wasmexport soon so should compile most binaries, including proxy-wasm. As soon as gotip includes it, this repository will be archived, and until then, it is recommended to pause rather than gamble on this bandaid.

nottinygc is a replacement memory allocator for TinyGo targetting WASI. The default allocator is built for small code size which can cause performance issues in higher-scale use cases. nottinygc replaces it with bdwgc for garbage collection and mimalloc for standard malloc-based allocation. These mature libraries can dramatically improve the performance of TinyGo WASI applications at the expense of several hundred KB of additional code footprint.

Note that this library currently only works when the scheduler is disabled.

Usage

Using the library requires both importing it and modifying flags when invoking TinyGo.

Add a blank import to the library to your code's main package.

import _ "github.com/wasilibs/nottinygc"

Additionally, add -gc=custom and -tags=custommalloc to your TinyGo build flags.

tinygo build -o main.wasm -gc=custom -tags=custommalloc -target=wasi -scheduler=none main.go

Using with Envoy

This library relies on WASI to implement certain functionality, for which Envoy's implementation is incomplete, and in addition, it's ABI, proxy-wasm has design issues that prevent working with defaults appropriate for normal TinyGo applications. As this project is commonly used with Envoy, we provide a build tag, to work around these issues. If building an Envoy plugin, add -tags=nottinygc_envoy (or combine it with additional tags) to your TinyGo build flags. This will disable export of malloc/free/ and define a no-op sched_yield function.

Other hosts that implement WASI fully, such as wazero and ABIs with correct memory semantics, such as http-wasm, will not have any issue. Implementations of proxy-wasm other than Envoy may also work fine without the build tag.

Performance

Benchmarks are run against every commit in the bench workflow. GitHub action runners are highly virtualized and do not have stable performance across runs, but the relative numbers within a run should still be somewhat, though not precisely, informative.

One run looks like this

BenchmarkGC/bench.wasm-2         	      52	 220294559 ns/op
BenchmarkGC/benchref.wasm-2      	       6	2000167805 ns/op

The benchmark is very simple, allocating some large strings in a loop. We see nottinygc perform almost 10x better in this benchmark. Note that just allocation / collection time is not the only aspect of GC performance, allocation locality and fragmentation can also affect performance of real-world applications. We have found that the default allocator can cause applications to run out of memory, possibly due to fragmentation, whereas this library will continue to run indefinitely.

nottinygc's People

Contributors

Stargazers

Watchers

Forkers

codifierr tillknuesting higress-group

nottinygc's Issues

browser support?

Should this work on browser?

I have tried to use nottinygc in our WASI project, but it fails.

With the standard tinygo GC the execution runs successfully.

Tested using @wasmer/wasi npm package and also https://github.com/bjorn3/browser_wasi_shim

We are using tinygo 0.28 (current dev branch).

Any hint is welcome! Thanks.

Getting integer divide by zero error

Hello,
I was trying out this library with a WASM module written in Go and compiled with the -scheduler=asyncify TinyGo build option.

When running the WASM module with wasmtime the module exits with:

Caused by:
    0: failed to invoke command default
    1: error while executing at wasm backtrace:
           0: 0xa142 - <unknown>!GC_expand_hp_inner
           1: 0x10db3 - <unknown>!GC_init
           2: 0x1220b - <unknown>!GC_generic_malloc_inner
           3: 0x127b4 - <unknown>!GC_generic_malloc
           4: 0x12924 - <unknown>!GC_malloc
           5: 0x31050 - runtime.alloc
                           at ~/go/pkg/mod/github.com/wasilibs/[email protected]/gc.go:53:6
           6: 0x53096 - <unknown>!runtime.run$1
           7: 0x52c5a - <goroutine wrapper>
                           at /usr/local/Cellar/tinygo/0.29.0/src/runtime/scheduler_any.go:23:2
           8: 0x12b84 - <unknown>!tinygo_launch
           9: 0x52b10 - (*internal/task.Task).Resume
                           at /usr/local/Cellar/tinygo/0.29.0/src/internal/task/task_asyncify.go:109:17              - runtime.scheduler
                           at /usr/local/Cellar/tinygo/0.29.0/src/runtime/scheduler.go:236:11              - runtime.run
                           at /usr/local/Cellar/tinygo/0.29.0/src/runtime/scheduler_any.go:28:11              - _start
                           at /usr/local/Cellar/tinygo/0.29.0/src/runtime/runtime_wasm_wasi.go:21:5
    2: wasm trap: integer divide by zero

TinyGo Version: 0.29.0 darwin/amd64 (using go version go1.20.1 and LLVM version 15.0.0)
nottinygc Version: v0.4.0

Any advice is much appreciated.

nottinygc memory leak with fastcache

Description

I am developing a WASM plugin for Istio ingress gateway. In the WASM Plugin, the fastcache is used for caching and the total cache size is limit to 32MB for preventing the OOM, however, the memory usage still keeps increasing

How to reproduce

dependecy:

tinygo v0.28.1
nottinygc v0.4.0
proxy-wasm-go-sdk v0.22.0
istion v1.18.0

The code of plugin to describe the issuse :

proxy-wasm-memory-leak

I`ve modify the source code of fastcache to make it can be compiled with tinygo

The Test script:

func TestMemoryLeak(t *testing.T) {
// Ingress Host and port :192.168.58.3:31505
	req, err := http.NewRequest("GET", "http://192.168.58.3:31505", nil)
	if err != nil {
		panic(err)
	}
	for i := 0; i < 10000000; i++ {
		_, err := http.DefaultClient.Do(req)
		if err != nil {
			continue
		}
	}
	fmt.Println("done")
}

what I’ve inspect is that is memory is keep increasing and never release

There may be a memory leak when processing memory blocks containing non-ASCII characters.

When I implemented the envoy proxy-wasm plugin, I found that if the response body is processed by gzip compression, a memory leak will occur. This problem also exists in the coraza-waf plugin. I suspect it may be affected by non-ASCII characters. You can use these test server program to reproduce it:

ASCII Response:

package main

import (
        "fmt"
        "log"
        "math/rand"
        "net/http"
        "time"
)

func echoHandler(w http.ResponseWriter, r *http.Request) {
        maxSize := 100 * 1024              // 100KB
        randSize := rand.Intn(maxSize) + 1 

        characters := "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789 ,.!?;:"

        rand.Seed(time.Now().UnixNano())

        randomText := make([]byte, randSize)
        for i := range randomText {
                randomText[i] = characters[rand.Intn(len(characters))]
        }

        w.Header().Set("Content-Type", "text/plain")

        _, err := w.Write(randomText)
        if err != nil {
                fmt.Println("Error writing body")
                return
        }
}

func main() {
        http.HandleFunc("/", echoHandler)

        fmt.Println("Server is running on port 5000...")

        log.Fatal(http.ListenAndServe(":5000", nil))
}

Non ASCII Response

package main

import (
        "fmt"
        "log"
        "math/rand"
        "net/http"
)

func echoHandler(w http.ResponseWriter, r *http.Request) {
        maxSize := 100 * 1024              // 100KB
        randSize := rand.Intn(maxSize) + 1 
        randomData := make([]byte, randSize)
        _, err := rand.Read(randomData) 
        if err != nil {
                http.Error(w, "Failed to generate random data", http.StatusInternalServerError)
                return
        }

        _, err = w.Write(randomData)
        if err != nil {
                fmt.Println("Error writing body")
                return
        }

}
func main() {
        http.HandleFunc("/", echoHandler)
        fmt.Println("Server is running on port 5000...")
        log.Fatal(http.ListenAndServe(":5000", nil))
}

--export=malloc --export=free cause memory leak

TinyGo Version: 0.28.1
nottinygc Version: v0.4.0

when cgo LDFLAGS is
#cgo LDFLAGS: -Lwasm -lgc -lmimalloc -lclang_rt.builtins-wasm32 --export=malloc --export=free
the wasm plugin will cause memory leak.

but when I remove --export=malloc --export=free it will not.

What't the reason?

can nottinygc support wasm?

tinygo gc has memory overflow in wasm。
can nottinygc support wasm?

Getting integer divide by zero error when use compress/gzip

I was use this library with a wasi module written in Go to enhance the performance, but i encounter an error with
panic: runtime error: divide by zero
when i use compress/gzip stdlib to decode gzip bytes.

the full compile command:
tinygo build -o wasm.wasm -target wasi -gc=custom -tags=custommalloc -target=wasi -scheduler=none ./main.go

i run it with wasmtime:
env WASMTIME_BACKTRACE_DETAILS=1 wasmtime wasm.wasm

panic: runtime error: divide by zero
Error: failed to run main module `wasm.wasm`

Caused by:
    0: failed to invoke command default
    1: error while executing at wasm backtrace:
           0: 0x24d61 - runtime.abort
                           at /root/software/tinygo/src/runtime/runtime_tinygowasm.go:70:6              - runtime.runtimePanicAt
                           at /root/software/tinygo/src/runtime/panic.go:71:7
           1: 0x27519 - runtime.divideByZeroPanic
                           at /root/software/tinygo/src/runtime/panic.go:177:16
           2: 0x1cf57 - runtime.alloc
                           at /root/workplace/go_poject/GOPATH/pkg/mod/github.com/wasilibs/[email protected]/intmap.go:93:19
           3: 0x23408 - compress/flate.newHuffmanEncoder
                           at /root/.g/go/.versions/1.21.3/src/compress/flate/huffman_code.go:60:24
           4: 0x22609 - compress/flate.generateFixedOffsetEncoding
                           at /root/.g/go/.versions/1.21.3/src/compress/flate/huffman_code.go:95:24              - runtime.run
                           at /root/software/tinygo/src/runtime/scheduler_none.go:24:9              - _start
                           at /root/software/tinygo/src/runtime/runtime_wasm_wasi.go:21:5

when i remove the gzip lib, everything is working fine

tinygo version:
tinygo version 0.30.0 linux/amd64 (using go version go1.21.3 and LLVM version 16.0.1)

Can I use malloc?

I am using malloc with TinyGo but with nottinygc it panics.

How can I resolve this?

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x1199875]

`// These function are exported by TinyGo
malloc := mod.ExportedFunction("malloc")
free := mod.ExportedFunction("free")

// Allocate Memory
results, err := malloc.Call(ctx, nameSize)
if err != nil {
	log.Panicln(err)
}
namePosition := results[0]

// This pointer is managed by TinyGo,
// but TinyGo is unaware of external usage.
// So, we have to free it when finished
defer free.Call(ctx, namePosition)

Memory leak exists, should use GC_malloc_ignore_off_page when alloc large object.

Hi @anuraaga, thanks for your project, we are using this nottinygc in our higress project which based on Envoy's wasm extension mechanism.

We discovered a memory leak during use, and you can find demo code to reproduce the memory leak here:
https://github.com/alibaba/higress/tree/main/plugins/wasm-go/extensions/gc-test

Note here that we replace nottinygc with github.com/higress-group/nottinygc, if we comment out this replacement like this:

...
// replace github.com/wasilibs/nottinygc v0.5.1 => github.com/higress-group/nottinygc v0.0.0-20231019105920-c4d985d443e1

require (
	github.com/alibaba/higress/plugins/wasm-go v0.0.0
	github.com/tetratelabs/proxy-wasm-go-sdk v0.22.0
	github.com/tidwall/gjson v1.14.3
)
...

Then build the wasm file and start envoy with the following configuration:

node:
  cluster: test-cluster
  id: test-idn

admin:
  address:
    socket_address: { address: 127.0.0.1, port_value: 9901 }

static_resources:
  listeners:
  - name: listener_0
    address:
      socket_address: { address: 127.0.0.1, port_value: 10000 }
    per_connection_buffer_limit_bytes: 1024000000
    filter_chains:
    - filters:
      - name: envoy.http_connection_manager
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
          stat_prefix: ingress_http
          access_log:
          - name: envoy.access_loggers.stdout
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.access_loggers.stream.v3.StdoutAccessLog
          http_filters:
          - name: gctest
            typed_config:
              "@type": type.googleapis.com/udpa.type.v1.TypedStruct
              type_url: type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm
              value:
                config:
                  name: basic_auth
                  vm_config:
                    runtime: envoy.wasm.runtime.v8
                    code:
                      local:
                        filename: ./mem.wasm
                    allow_precompiled: true
                  fail_open: true
                  configuration:
                    "@type": type.googleapis.com/google.protobuf.StringValue
                    value: '{"bytes": 10485760}'
          - name: envoy.router
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
          route_config:
            name: local_route
            virtual_hosts:
            - name: local_service
              domains:
              - "*"
              routes:
              - match:
                  prefix: /
                direct_response:
                  status: 200
                  body:
                    inline_string: "hello world"

Use the following command to perform a stress test:

wrk -R 50 -t 10 -c 10 -d60s -s post.lua http://localhost:10000/

After a few seconds, we can see the following log:

[2023-10-23 15:42:30.858][1902490][info][wasm] [source/extensions/common/wasm/context.cc:1204] wasm log basic_auth: [gc-test] {"Sys": 1063452672,"HeapSys": 1058013184,"HeapIdle": 145313792,"HeapInuse": 0,"HeapReleased": 0}
[2023-10-23 15:42:30.861][1902490][error][wasm] [source/extensions/common/wasm/wasm_vm.cc:41] Function: proxy_on_request_headers failed: Uncaught RuntimeError: unreachable\nProxy-Wasm plugin in-VM backtrace:\n  0:  0x1d4fc - runtime._panic\n  1:  0x1ce84 - runtime.alloc\n  2:  0x26758 - main.onHttpRequestHeaders\n  3:  0x3761a - proxy_on_request_headers

invalid flag: --export=malloc

Hi, when I use v0.4.0, I meet this error

github.com/wasilibs/[email protected]/init.go:9:14: invalid flag: --export=malloc

How can I fix it?

Wasm VM failed Failed to initialize Wasm code

I'm trying to run the wasm plugin in envoy but getting following error while running.

[2024-04-15 10:47:48.818][1][error][wasm] [source/extensions/common/wasm/wasm_vm.cc:39] Failed to load Wasm module due to a missing import: wasi_snapshot_preview1.sched_yield
[2024-04-15 10:47:48.818][1][error][wasm] [source/extensions/common/wasm/wasm.cc:118] Wasm VM failed Failed to initialize Wasm code
[2024-04-15 10:47:48.819][1][critical][wasm] [source/extensions/common/wasm/wasm.cc:472] Plugin configured to fail closed failed to load
[2024-04-15 10:47:48.821][1][critical][main] [source/server/server.cc:113] error initializing configuration '/etc/envoy/envoy.yaml': Unable to create Wasm HTTP filter 
[2024-04-15 10:47:48.822][1][info][main] [source/server/server.cc:815] exiting
Unable to create Wasm HTTP filter 
make: *** [run] Error 1

Build Command:

tinygo build -o istio-plugin.wasm -gc=custom -tags=custommalloc -scheduler=none -target=wasi main.go

Envoy Config:

 http_filters:
                  - name: envoy.filters.http.wasm
                    typed_config:
                      "@type": type.googleapis.com/udpa.type.v1.TypedStruct
                      type_url: type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm
                      value:
                        config:
                          configuration:
                            "@type": type.googleapis.com/google.protobuf.StringValue
                            value: |
                              {
                                "header": "appname",
                                "value": "aicore!b32291"
                              }
                          vm_config:
                            runtime: "envoy.wasm.runtime.v8"
                            code:
                              local:
                                filename: "/etc/envoy/istio-tenant-id-plugin.wasm"
                  - name: envoy.filters.http.router
                    typed_config:
                      "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router

As mentioned in the README file, I have added "github.com/wasilibs/nottinygc" in import statements.

Import Statements:

import (
	"strings"

	_ "github.com/wasilibs/nottinygc"

	b64 "encoding/base64"

	"github.com/tetratelabs/proxy-wasm-go-sdk/proxywasm"
	"github.com/tetratelabs/proxy-wasm-go-sdk/proxywasm/types"
	"github.com/tidwall/gjson"
)

error: Linking globals named 'runtime.SetFinalizer': symbol multiply defined!

vikas@ xcp (master) $ go version
go version go1.19 linux/amd64
vikas@ xcp (master) $ tinygo build -o /work/out/wasi_wasm/xcp-guard.wasm -gc=custom -tags='custommalloc nottinygc_envoy' -scheduler=none -target=wasi main.go
error: Linking globals named 'runtime.SetFinalizer': symbol multiply defined!
make[1]: *** [Makefile.core.mk:147: out/wasi_wasm/xcp-guard.wasm] Error 1
make: *** [Makefile:58: docker.xcp-guard] Error 2

I am hitting above shared error when trying to build a wasm module with nottinygo. I have imported nottinygo in main.go and used tinygo build tags as per the README. nottinygo version is 0.7.1

Failed to load Wasm module in Istio

HI @anuraaga, thank you for releasing such a great GC strategy in proxy-wasm!

I am having trouble with the default GC in TinyGo(v0.27.0) as well.
When I try to compile my wasm plugin use nottinygc and follow the instructions in the readme, the compilation works. But when I deploy it into a k8s cluster with Istio(v1.14 and v1.16), the envoy shows the error when loading the wasm module.

Here is the error message inside envoy proxy.

Failed to load Wasm module due to a missing import: wasi_snapshot_preview1.sched_yield

Do you have any insights for such an issue? Appreciate your help!

Question: Does the gc execute before the `proxywasm.DispatchHttpCall` callback executes ?

Hi @anuraaga !

I am running into a strange behaviour when my wasm plugin executes a DispatchHttpCall. It appears the call to the service (dispatch call) is never made in majority of the runs, making me think if the gc clears the context before the execution of the dispatch call.

The flow works as expected when i don't run the custom gc
Req -> parseHeaders -> dispatchCall(ServiceA) -> callback -> getResponseHeaders -> update headers -> ReqGoesUpstream

The metrics below measure the execution_count(execution of the plugin) & the success_count(completion of callback). Below are the numbers when I don't run the custom gc - execution_count = success_count.

Whereas when i run the custom gc, only a portion of requests end up making a call upstream (I verified that requests don't reach serviceA)

My main.go has these imports

import (
	"crypto/rand"
	"fmt"
	"math/big"
	"os"
	"strings"
	"time"

	"github.com/tetratelabs/proxy-wasm-go-sdk/proxywasm"
	"github.com/tetratelabs/proxy-wasm-go-sdk/proxywasm/types"
	"github.com/tidwall/gjson"
	_ "github.com/wasilibs/nottinygc"
	"golang.org/x/exp/slices"
)

And the file ends with this after all the functions

// required for nottinygc
//
//export sched_yield
func sched_yield() int32 {
	return 0
}

Appreciate your thoughts on this behaviour , and thank you for this package :)