Crash with cgo in concurrent usage about go-re2 HOT 9 CLOSED

Thanks for doublechecking @blotus, I could repro. Sorry, had this silly mistake in my test...

https://github.com/wasilibs/go-re2/blob/main/concurrency_test.go#L6

Will take a look into this.

from go-re2.

Thanks for the repro! Indeed the intention is to be able to use regex without external locking even with cgo. Will take a look.

from go-re2.

Also if you can spare the RAM, with wasm using a sync.pool to compile the same regex multiple times could be an option for improving concurrency with wasm.

from go-re2.

Hi @buixor - I am trying out the repro - on my Mac I have run it for 2min without a crash. Can you confirm how long you run before seeing a crash with your repro? And do you see no crash when not using goroutines, i.e., no concurrency?

Also, I am just using re2 installed from homebrew, it would be good if you can confirm how you are building / installing re2 in your testing.

from go-re2.

Hello @anuraaga !

• I ran those tests on linux (kernel 5.x).
• The crash arrived within seconds (less than 10) with the provided snippet.
• The crash only seems to happen under concurrency.
• We were pulling go-re2 directly from your github repository (as we saw there was some new releases, we wanted to try latest version to see if it fixed the bug)

Please let me know if you have trouble to reproduce the bug, happy to help

from go-re2.

Thanks for the details!

We were pulling go-re2 directly from your github repository (as we saw there was some new releases, we wanted to try latest version to see if it fixed the bug)

Thanks - this is how you are getting go-re2, but as you are using cgo, you also must be providing re2 itself, which is found via pkg-config. Can you share how you are installing or building re2, the C++ library? Is it apt-get for example?

from go-re2.

Hello,

I also manage to reproduce the crash on macos, running go 1.20, it is nearly instant (using a freshly installed re2 from brew):

~/test/aa ᐅ go build -tags re2_cgo
~/test/aa ᐅ ./re2_repro
panic: runtime error: slice bounds out of range [:90072254402122128] with length 163

goroutine 23 [running]:
github.com/wasilibs/go-re2/internal.matchedString(...)
	/Users/seb/go/pkg/mod/github.com/wasilibs/[email protected]/internal/re2.go:978
github.com/wasilibs/go-re2/internal.(*Regexp).FindStringSubmatch.func1({0x1400001c4a0?, 0x0?, 0xa3?})
	/Users/seb/go/pkg/mod/github.com/wasilibs/[email protected]/internal/re2.go:559 +0x128
github.com/wasilibs/go-re2/internal.readMatches(0x1004cc5a8?, {0x1003e2e7e?, 0xa3?}, 0x140003dd900, 0xc1, 0x140003a2f40)
	/Users/seb/go/pkg/mod/github.com/wasilibs/[email protected]/internal/re2_tinygowasm.go:173 +0x54
github.com/wasilibs/go-re2/internal.(*Regexp).findSubmatch(0x1400007a050, {0x0?, 0x0?}, 0x0?)
	/Users/seb/go/pkg/mod/github.com/wasilibs/[email protected]/internal/re2.go:593 +0xa8
github.com/wasilibs/go-re2/internal.(*Regexp).FindStringSubmatch(0x0?, {0x1003e2e7e?, 0x0?})
	/Users/seb/go/pkg/mod/github.com/wasilibs/[email protected]/internal/re2.go:558 +0xbc
main.parse({0x1003e2e7e, 0xa3}, 0x0?)
	/Users/seb/test/aa/main.go:9 +0x3c
created by main.main
	/Users/seb/test/aa/main.go:21 +0x4c

On linux, the test was run on ubuntu 22.04 (also with go 1.20), using re2 from the ubuntu repositories (version 20220201, quite a bit older)

from go-re2.

@blotus Not sure you'll see this reply after so long, but wanted to share that I made a change that should reduce memory usage with the wazero codepath dramatically. If you are interested, it would be cool if you could give the latest unreleased commit a try to see how it goes for you, especially whether performance is enough to not bother with cgo or not. Real world data helps us a lot so if you happen to have some bandwidth to check that it would be really appreciated.

from go-re2.

Hi @buixor - sorry I think I got confused on my previous reply about the original poster of the issue. Not sure you'll see this reply after so long, but just in case see #38 (comment)

from go-re2.