Code Monkey home page Code Monkey logo

go-re2's Introduction

go-re2

go-re2 is a drop-in replacement for the standard library regexp package which uses the C++ re2 library for improved performance with large inputs or complex expressions. By default, re2 is packaged as a WebAssembly module and accessed with the pure Go runtime, wazero. This means that it is compatible with any Go application, regardless of availability of cgo.

The library can also be used in a TinyGo application being compiled to WebAssembly. Currently, regexp when compiled with TinyGo always has very slow performance and sometimes fails to compile expressions completely.

Note that if your regular expressions or input are small, this library is slower than the standard library. You will generally "know" if your application requires high performance for complex regular expressions, for example in security filtering software. If you do not know your app has such needs and are not using TinyGo, you should turn away now.

Behavior differences

The library is almost fully compatible with the standard library regexp package, with just a few behavior differences. These are likely corner cases that don't affect typical applications. It is best to confirm them before proceeding.

  • Invalid utf-8 strings are treated differently. The standard library silently replaces invalid utf-8 with the unicode replacement character. This library will stop consuming strings when encountering invalid utf-8.

    • experimental.CompileLatin1 can be used to match against non-utf8 strings
  • reflect.DeepEqual cannot compare Regexp objects.

Continue to use the standard library if your usage would match any of these.

Searching this codebase for // GAP will allow finding tests that have been tweaked to demonstrate behavior differences.

API differences

All APIs found in regexp are available except

  • *Reader: re2 does not support streaming input
  • *Func: re2 does not support replacement with callback functions

Note that unlike many packages that wrap C++ libraries, there is no added Close type of method. See the rationale for more details.

Experimental APIs

The experimental package contains APIs not part of standard regexp that are incubating. They may in the future be moved to stable packages. The experimental package does not provide any guarantee of API stability even across minor version updates.

Usage

go-re2 is a standard Go library package and can be added to a go.mod file. It will work fine in Go or TinyGo projects.

go get github.com/wasilibs/go-re2

Because the library is a drop-in replacement for the standard library, an import alias can make migrating code to use it simple.

import "regexp"

can be changed to

import regexp "github.com/wasilibs/go-re2"

cgo

This library also supports opting into using cgo to wrap re2 instead of using WebAssembly. This requires having re2 installed and available via pkg-config on the system. The build tag re2_cgo can be used to enable cgo support.

Ubuntu

On Ubuntu install the gcc tool chain and the re2 library as follows:

sudo apt install build-essential
sudo apt-get install -y libre2-dev

Windows

On Windows start by installing MSYS2. Then open the MINGW64 terminal and install the gcc toolchain and re2 via pacman:

pacman -S mingw-w64-x86_64-gcc
pacman -S mingw-w64-x86_64-re2

If you want to run the resulting exe program outside the MINGW64 terminal you need to add a path to the MinGW-w64 libraries to the PATH environmental variable (adjust as needed for your system):

SET PATH=C:\msys64\mingw64\bin;%PATH%

MacOS

On Mac start by installing homebrew including installation of the command line tools. Then install re2 via brew:

brew install re2

Performance

Benchmarks are run against every commit in the bench workflow. GitHub action runners are highly virtualized and do not have stable performance across runs, but the relative numbers within a run should still be somewhat, though not precisely, informative.

wafbench

wafbench tests the performance of replacing the regex operator of the OWASP CoreRuleSet and Coraza implementation with this library. This benchmark is considered a real world performance test, with the regular expressions being real ones used in production. Security filtering rules often have highly complex expressions.

One run looks like this

name \ time/op     build/wafbench_stdlib.txt  build/wafbench.txt  build/wafbench_cgo.txt
WAF/FTW-4                         21.72 ± ∞ ¹    21.17 ± ∞ ¹   -2.55% (p=0.008 n=5)    19.51 ± ∞ ¹  -10.19% (p=0.008 n=5)
WAF/POST/1-4                     2.308m ± ∞ ¹   2.727m ± ∞ ¹  +18.18% (p=0.008 n=5)   2.479m ± ∞ ¹   +7.44% (p=0.008 n=5)
WAF/POST/1000-4                 15.399m ± ∞ ¹   4.734m ± ∞ ¹  -69.26% (p=0.008 n=5)   3.742m ± ∞ ¹  -75.70% (p=0.008 n=5)
WAF/POST/10000-4                150.65m ± ∞ ¹   19.90m ± ∞ ¹  -86.79% (p=0.008 n=5)   13.47m ± ∞ ¹  -91.06% (p=0.008 n=5)
WAF/POST/100000-4               1511.3m ± ∞ ¹   171.7m ± ∞ ¹  -88.64% (p=0.008 n=5)   110.4m ± ∞ ¹  -92.69% (p=0.008 n=5)

FTW is the time to run the standard CoreRuleSet test framework. The performance of this library with WebAssembly, wafbench.txt, shows a slight improvement over the standard library in this baseline test case.

The FTW test suite will issue many requests with various payloads, generally somewhat small. The POST tests show the same ruleset applied to requests with payload sizes as shown, in bytes. We see that only with the absolute smallest payload of 1 byte does the standard library perform a bit better than this library. For any larger size, even a fairly typical 1KB, go-re2 greatly outperforms.

cgo seems to offer about a 30% improvement on WebAssembly in this library. Many apps may accept the somewhat slower performance in exchange for the build and deployment flexibility of WebAssembly but either option will work with no changes to the codebase.

Microbenchmarks

Microbenchmarks are the same as included in the Go standard library. Full results can be viewed in the workflow, a sample of results for one run looks like this

name \ time/op                  build/bench_stdlib.txt  build/bench.txt   build/bench_cgo.txt
Find-4                                     162.3n ± ∞ ¹       1057.0n ± ∞ ¹    +551.26% (p=0.008 n=5)     397.5n ± ∞ ¹   +144.92% (p=0.008 n=5)
Compile/Onepass-4                          4.017µ ± ∞ ¹       23.585µ ± ∞ ¹    +487.13% (p=0.008 n=5)     7.803µ ± ∞ ¹    +94.25% (p=0.008 n=5)
Compile/Medium-4                           9.042µ ± ∞ ¹       45.105µ ± ∞ ¹    +398.84% (p=0.008 n=5)    13.346µ ± ∞ ¹    +47.60% (p=0.008 n=5)
Compile/Hard-4                             66.52µ ± ∞ ¹       272.58µ ± ∞ ¹    +309.80% (p=0.008 n=5)     94.49µ ± ∞ ¹    +42.06% (p=0.008 n=5)
Match/Easy0/16-4                           3.721n ± ∞ ¹      615.300n ± ∞ ¹  +16435.88% (p=0.008 n=5)   155.900n ± ∞ ¹  +4089.73% (p=0.008 n=5)
Match/Easy0/32-4                           40.41n ± ∞ ¹       620.10n ± ∞ ¹   +1434.52% (p=0.008 n=5)    155.00n ± ∞ ¹   +283.57% (p=0.008 n=5)
Match/Easy0/1K-4                           242.3n ± ∞ ¹        689.7n ± ∞ ¹    +184.65% (p=0.008 n=5)     154.8n ± ∞ ¹    -36.11% (p=0.008 n=5)
Match/Easy0/32K-4                         3279.0n ± ∞ ¹       1384.0n ± ∞ ¹     -57.79% (p=0.008 n=5)     154.0n ± ∞ ¹    -95.30% (p=0.008 n=5)
Match/Easy0/1M-4                        240196.0n ± ∞ ¹      42954.0n ± ∞ ¹     -82.12% (p=0.008 n=5)     155.0n ± ∞ ¹    -99.94% (p=0.008 n=5)
Match/Easy0/32M-4                      7942195.0n ± ∞ ¹    1359859.0n ± ∞ ¹     -82.88% (p=0.008 n=5)     155.4n ± ∞ ¹   -100.00% (p=0.008 n=5)
Match/Easy0i/16-4                          3.723n ± ∞ ¹      589.600n ± ∞ ¹  +15736.69% (p=0.008 n=5)   155.800n ± ∞ ¹  +4084.80% (p=0.008 n=5)
Match/Easy0i/32-4                          681.2n ± ∞ ¹        593.3n ± ∞ ¹     -12.90% (p=0.008 n=5)     154.3n ± ∞ ¹    -77.35% (p=0.008 n=5)
Match/Easy0i/1K-4                        20325.0n ± ∞ ¹        667.6n ± ∞ ¹     -96.72% (p=0.008 n=5)     155.4n ± ∞ ¹    -99.24% (p=0.008 n=5)
Match/Easy0i/32K-4                      913391.0n ± ∞ ¹       1368.0n ± ∞ ¹     -99.85% (p=0.008 n=5)     155.2n ± ∞ ¹    -99.98% (p=0.008 n=5)
Match/Easy0i/1M-4                     29087522.0n ± ∞ ¹      42975.0n ± ∞ ¹     -99.85% (p=0.008 n=5)     155.5n ± ∞ ¹   -100.00% (p=0.008 n=5)
Match/Easy0i/32M-4                   929572314.0n ± ∞ ¹    1362975.0n ± ∞ ¹     -99.85% (p=0.008 n=5)     155.7n ± ∞ ¹   -100.00% (p=0.008 n=5)
Match/Easy1/16-4                           3.721n ± ∞ ¹      591.000n ± ∞ ¹  +15782.83% (p=0.008 n=5)   156.600n ± ∞ ¹  +4108.55% (p=0.008 n=5)
Match/Easy1/32-4                           37.39n ± ∞ ¹       595.30n ± ∞ ¹   +1492.14% (p=0.008 n=5)    155.40n ± ∞ ¹   +315.62% (p=0.008 n=5)
Match/Easy1/1K-4                           517.5n ± ∞ ¹        685.1n ± ∞ ¹     +32.39% (p=0.008 n=5)     156.4n ± ∞ ¹    -69.78% (p=0.008 n=5)
Match/Easy1/32K-4                        25905.0n ± ∞ ¹       1342.0n ± ∞ ¹     -94.82% (p=0.008 n=5)     158.2n ± ∞ ¹    -99.39% (p=0.008 n=5)
Match/Easy1/1M-4                        929177.0n ± ∞ ¹      42940.0n ± ∞ ¹     -95.38% (p=0.008 n=5)     155.1n ± ∞ ¹    -99.98% (p=0.008 n=5)
Match/Easy1/32M-4                     29798707.0n ± ∞ ¹    1362212.0n ± ∞ ¹     -95.43% (p=0.008 n=5)     155.6n ± ∞ ¹   -100.00% (p=0.008 n=5)
Match/Medium/16-4                          3.723n ± ∞ ¹      590.900n ± ∞ ¹  +15771.61% (p=0.008 n=5)   155.000n ± ∞ ¹  +4063.31% (p=0.008 n=5)
Match/Medium/32-4                          566.8n ± ∞ ¹        595.2n ± ∞ ¹      +5.01% (p=0.008 n=5)     154.9n ± ∞ ¹    -72.67% (p=0.008 n=5)
Match/Medium/1K-4                        20063.0n ± ∞ ¹        669.7n ± ∞ ¹     -96.66% (p=0.008 n=5)     155.3n ± ∞ ¹    -99.23% (p=0.008 n=5)
Match/Medium/32K-4                      924929.0n ± ∞ ¹       1340.0n ± ∞ ¹     -99.86% (p=0.008 n=5)     155.8n ± ∞ ¹    -99.98% (p=0.008 n=5)
Match/Medium/1M-4                     29406989.0n ± ∞ ¹      42947.0n ± ∞ ¹     -99.85% (p=0.008 n=5)     154.6n ± ∞ ¹   -100.00% (p=0.008 n=5)
Match/Medium/32M-4                   963966642.0n ± ∞ ¹    1363441.0n ± ∞ ¹     -99.86% (p=0.008 n=5)     154.6n ± ∞ ¹   -100.00% (p=0.008 n=5)
Match/Hard/16-4                            3.744n ± ∞ ¹      596.000n ± ∞ ¹  +15818.80% (p=0.008 n=5)   155.600n ± ∞ ¹  +4055.98% (p=0.008 n=5)
Match/Hard/32-4                            997.3n ± ∞ ¹        598.1n ± ∞ ¹     -40.03% (p=0.008 n=5)     155.6n ± ∞ ¹    -84.40% (p=0.008 n=5)
Match/Hard/1K-4                          30435.0n ± ∞ ¹        686.5n ± ∞ ¹     -97.74% (p=0.008 n=5)     154.7n ± ∞ ¹    -99.49% (p=0.008 n=5)
Match/Hard/32K-4                       1348825.0n ± ∞ ¹       1342.0n ± ∞ ¹     -99.90% (p=0.008 n=5)     155.7n ± ∞ ¹    -99.99% (p=0.008 n=5)
Match/Hard/1M-4                       43023861.0n ± ∞ ¹      42891.0n ± ∞ ¹     -99.90% (p=0.008 n=5)     155.9n ± ∞ ¹   -100.00% (p=0.008 n=5)
Match/Hard/32M-4                    1380076363.0n ± ∞ ¹    1362504.0n ± ∞ ¹     -99.90% (p=0.008 n=5)     155.7n ± ∞ ¹   -100.00% (p=0.008 n=5)
Match/Hard1/16-4                          2661.0n ± ∞ ¹        697.6n ± ∞ ¹     -73.78% (p=0.008 n=5)     169.0n ± ∞ ¹    -93.65% (p=0.008 n=5)
Match/Hard1/32-4                          5084.0n ± ∞ ¹        796.9n ± ∞ ¹     -84.33% (p=0.008 n=5)     201.5n ± ∞ ¹    -96.04% (p=0.008 n=5)
Match/Hard1/1K-4                         157.089µ ± ∞ ¹        6.838µ ± ∞ ¹     -95.65% (p=0.008 n=5)     2.092µ ± ∞ ¹    -98.67% (p=0.008 n=5)
Match/Hard1/32K-4                        6748.74µ ± ∞ ¹       196.99µ ± ∞ ¹     -97.08% (p=0.008 n=5)     62.52µ ± ∞ ¹    -99.07% (p=0.008 n=5)
Match/Hard1/1M-4                         214.239m ± ∞ ¹        6.290m ± ∞ ¹     -97.06% (p=0.008 n=5)     1.895m ± ∞ ¹    -99.12% (p=0.008 n=5)
Match/Hard1/32M-4                        6936.54m ± ∞ ¹       201.05m ± ∞ ¹     -97.10% (p=0.008 n=5)     59.88m ± ∞ ¹    -99.14% (p=0.008 n=5)
MatchParallel/Easy0/16-4                   1.710n ± ∞ ¹      942.200n ± ∞ ¹  +54999.42% (p=0.008 n=5)    81.700n ± ∞ ¹  +4677.78% (p=0.008 n=5)
MatchParallel/Easy0/32-4                   19.10n ± ∞ ¹       944.80n ± ∞ ¹   +4846.60% (p=0.008 n=5)     82.03n ± ∞ ¹   +329.48% (p=0.008 n=5)
MatchParallel/Easy0/1K-4                   96.72n ± ∞ ¹       978.40n ± ∞ ¹    +911.58% (p=0.008 n=5)     80.82n ± ∞ ¹    -16.44% (p=0.008 n=5)
MatchParallel/Easy0/32K-4                1285.00n ± ∞ ¹      1425.00n ± ∞ ¹     +10.89% (p=0.008 n=5)     80.82n ± ∞ ¹    -93.71% (p=0.008 n=5)
MatchParallel/Easy0/1M-4                74284.00n ± ∞ ¹     41354.00n ± ∞ ¹     -44.33% (p=0.008 n=5)     79.92n ± ∞ ¹    -99.89% (p=0.008 n=5)
MatchParallel/Easy0/32M-4             2382580.00n ± ∞ ¹   1320813.00n ± ∞ ¹     -44.56% (p=0.008 n=5)     80.95n ± ∞ ¹   -100.00% (p=0.008 n=5)
MatchParallel/Easy0i/16-4                  1.711n ± ∞ ¹      937.600n ± ∞ ¹  +54698.36% (p=0.008 n=5)    81.650n ± ∞ ¹  +4672.06% (p=0.008 n=5)
MatchParallel/Easy0i/32-4                 339.90n ± ∞ ¹       945.70n ± ∞ ¹    +178.23% (p=0.008 n=5)     81.28n ± ∞ ¹    -76.09% (p=0.008 n=5)
MatchParallel/Easy0i/1K-4               10107.00n ± ∞ ¹       992.20n ± ∞ ¹     -90.18% (p=0.008 n=5)     81.21n ± ∞ ¹    -99.20% (p=0.008 n=5)
MatchParallel/Easy0i/32K-4             451362.00n ± ∞ ¹      1422.00n ± ∞ ¹     -99.68% (p=0.008 n=5)     81.38n ± ∞ ¹    -99.98% (p=0.008 n=5)
MatchParallel/Easy0i/1M-4            14439204.00n ± ∞ ¹     41325.00n ± ∞ ¹     -99.71% (p=0.008 n=5)     80.67n ± ∞ ¹   -100.00% (p=0.008 n=5)
MatchParallel/Easy0i/32M-4          563552821.00n ± ∞ ¹   1322234.00n ± ∞ ¹     -99.77% (p=0.008 n=5)     80.94n ± ∞ ¹   -100.00% (p=0.008 n=5)
MatchParallel/Easy1/16-4                   1.712n ± ∞ ¹      940.700n ± ∞ ¹  +54847.43% (p=0.008 n=5)    82.050n ± ∞ ¹  +4692.64% (p=0.008 n=5)
MatchParallel/Easy1/32-4                   18.04n ± ∞ ¹       944.10n ± ∞ ¹   +5133.37% (p=0.008 n=5)     81.89n ± ∞ ¹   +353.94% (p=0.008 n=5)
MatchParallel/Easy1/1K-4                  256.50n ± ∞ ¹      1007.00n ± ∞ ¹    +292.59% (p=0.008 n=5)     81.64n ± ∞ ¹    -68.17% (p=0.008 n=5)
MatchParallel/Easy1/32K-4               11781.00n ± ∞ ¹      1424.00n ± ∞ ¹     -87.91% (p=0.008 n=5)     81.45n ± ∞ ¹    -99.31% (p=0.008 n=5)
MatchParallel/Easy1/1M-4               407922.00n ± ∞ ¹     41413.00n ± ∞ ¹     -89.85% (p=0.008 n=5)     81.57n ± ∞ ¹    -99.98% (p=0.008 n=5)
MatchParallel/Easy1/32M-4            13077618.00n ± ∞ ¹   1326004.00n ± ∞ ¹     -89.86% (p=0.008 n=5)     81.48n ± ∞ ¹   -100.00% (p=0.008 n=5)
MatchParallel/Medium/16-4                  1.719n ± ∞ ¹      937.100n ± ∞ ¹  +54414.25% (p=0.008 n=5)    79.930n ± ∞ ¹  +4549.80% (p=0.008 n=5)
MatchParallel/Medium/32-4                 284.90n ± ∞ ¹       948.90n ± ∞ ¹    +233.06% (p=0.008 n=5)     79.97n ± ∞ ¹    -71.93% (p=0.008 n=5)
MatchParallel/Medium/1K-4                9223.00n ± ∞ ¹      1003.00n ± ∞ ¹     -89.13% (p=0.008 n=5)     80.01n ± ∞ ¹    -99.13% (p=0.008 n=5)
MatchParallel/Medium/32K-4             469468.00n ± ∞ ¹      1418.00n ± ∞ ¹     -99.70% (p=0.008 n=5)     79.45n ± ∞ ¹    -99.98% (p=0.008 n=5)
MatchParallel/Medium/1M-4            14414208.00n ± ∞ ¹     41372.00n ± ∞ ¹     -99.71% (p=0.008 n=5)     80.16n ± ∞ ¹   -100.00% (p=0.008 n=5)
MatchParallel/Medium/32M-4          571771553.00n ± ∞ ¹   1320563.00n ± ∞ ¹     -99.77% (p=0.008 n=5)     79.68n ± ∞ ¹   -100.00% (p=0.008 n=5)
MatchParallel/Hard/16-4                    1.717n ± ∞ ¹      940.800n ± ∞ ¹  +54693.24% (p=0.008 n=5)    81.440n ± ∞ ¹  +4643.16% (p=0.008 n=5)
MatchParallel/Hard/32-4                   487.00n ± ∞ ¹       949.10n ± ∞ ¹     +94.89% (p=0.008 n=5)     81.31n ± ∞ ¹    -83.30% (p=0.008 n=5)
MatchParallel/Hard/1K-4                 14814.00n ± ∞ ¹      1002.00n ± ∞ ¹     -93.24% (p=0.008 n=5)     81.13n ± ∞ ¹    -99.45% (p=0.008 n=5)
MatchParallel/Hard/32K-4               685271.00n ± ∞ ¹      1423.00n ± ∞ ¹     -99.79% (p=0.008 n=5)     81.37n ± ∞ ¹    -99.99% (p=0.008 n=5)
MatchParallel/Hard/1M-4              21979040.00n ± ∞ ¹     41376.00n ± ∞ ¹     -99.81% (p=0.008 n=5)     81.13n ± ∞ ¹   -100.00% (p=0.008 n=5)
MatchParallel/Hard/32M-4           1382894393.00n ± ∞ ¹   1321072.00n ± ∞ ¹     -99.90% (p=0.008 n=5)     81.32n ± ∞ ¹   -100.00% (p=0.008 n=5)
MatchParallel/Hard1/16-4                 1303.00n ± ∞ ¹      1020.00n ± ∞ ¹     -21.72% (p=0.008 n=5)     86.51n ± ∞ ¹    -93.36% (p=0.008 n=5)
MatchParallel/Hard1/32-4                 2526.00n ± ∞ ¹      1094.00n ± ∞ ¹     -56.69% (p=0.008 n=5)     97.66n ± ∞ ¹    -96.13% (p=0.008 n=5)
MatchParallel/Hard1/1K-4                 77806.0n ± ∞ ¹       3413.0n ± ∞ ¹     -95.61% (p=0.008 n=5)     633.1n ± ∞ ¹    -99.19% (p=0.008 n=5)
MatchParallel/Hard1/32K-4                3519.23µ ± ∞ ¹        97.97µ ± ∞ ¹     -97.22% (p=0.008 n=5)     17.12µ ± ∞ ¹    -99.51% (p=0.008 n=5)
MatchParallel/Hard1/1M-4                121966.5µ ± ∞ ¹       3122.6µ ± ∞ ¹     -97.44% (p=0.008 n=5)     517.1µ ± ∞ ¹    -99.58% (p=0.008 n=5)
MatchParallel/Hard1/32M-4                6725.29m ± ∞ ¹       100.80m ± ∞ ¹     -98.50% (p=0.008 n=5)     16.86m ± ∞ ¹    -99.75% (p=0.008 n=5)

Most benchmarks from the standard library are similar to Find, testing simple expressions with small input. In all of these, the standard library performs much better. To reiterate the guidance at the top of this README, if you only use simple expressions with small input, you should not use this library.

The compilation benchmarks show that re2 is much slower to compile expressions than the standard library - this is more than just the overhead of foreign function invocation. This likely results in the improved performance at runtime in other cases. Because memory allocated in Wasm or cgo is essentially "off-heap", it is difficult to measure allocation overhead compared to standard library. More investigation needs to be done, but we expect the memory overhead for all approaches to be similar.

The match benchmarks show the performance tradeoffs for complexity vs input size. We see the standard library perform the best with low complexity and size, but for high complexity or high input size, go-re2 with WebAssembly outperforms, often significantly. Notable is Hard1, where even on the smallest size this library outperforms. The expression is ABCD|CDEF|EFGH|GHIJ|IJKL|KLMN|MNOP|OPQR|QRST|STUV|UVWX|WXYZ, a simple OR of literals - re2 has the concept of regex sets and likely is able to optimize this in a special way. The CoreRuleSet contains many expressions of a form like this - this possibly indicates good performance in real world use cases.

go-re2's People

Contributors

anuraaga avatar codefromthecrypt avatar evacchi avatar mathetake avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

go-re2's Issues

When compiling to WASM for js use (GOOS=js) im getting a memory allocation errors

with go builtin regexp it works, with this module not

errors:

runtime: out of memory: cannot allocate 4294967296-byte block (4030464 in use)
 fatal error: out of memory

goroutine 1 gp=0x18001c0 m=0 mp=0x3efee0 [running]:
runtime.throw({0x7dca0, 0xd})
  /opt/homebrew/opt/go/libexec/src/runtime/panic.go:1023 +0x3 fp=0x1813618 sp=0x18135f0 pc=0x12630003
runtime.(*mcache).allocLarge(0x13e0108, 0x100000000, 0x1)
   /opt/homebrew/opt/go/libexec/src/runtime/mcache.go:236 +0x1f fp=0x1813668 sp=0x1813618 pc=0x1114001f
runtime.mallocgc(0x100000000, 0x2cbc0, 0x1)
  /opt/homebrew/opt/go/libexec/src/runtime/malloc.go:1165 +0xab fp=0x18136f0 sp=0x1813668 pc=0x10c700ab
runtime.makeslice(0x2cbc0, 0x100000000, 0x100000000)
    /opt/homebrew/opt/go/libexec/src/runtime/slice.go:107 +0x12 fp=0x1813718 sp=0x18136f0 pc=0x13440012
 github.com/tetratelabs/wazero/internal/wasm.NewMemoryInstance(0x180c9b0)
   /Users/nitzan/go/pkg/mod/github.com/tetratelabs/[email protected]/internal/wasm/memory.go:92 +0xc fp=0x1813780 sp=0x1813718 pc=0x1fad000c
github.com/tetratelabs/wazero/internal/wasm.(*ModuleInstance).buildMemory(0x18b2400, 0x1800e00)
 /Users/nitzan/go/pkg/mod/github.com/tetratelabs/[email protected]/internal/wasm/module.go:658 +0x5 fp=0x18137a8 sp=0x1813780 pc=0x1fd20005
github.com/tetratelabs/wazero/internal/wasm.(*Store).instantiate(0x1836180, {0xcf440, 0x43d7e0}, 0x1800e00, {0x7a473, 0x3}, 0x18bc9c0, {0x43d7e0, 0x0, 0x0})
  /Users/nitzan/go/pkg/mod/github.com/tetratelabs/[email protected]/internal/wasm/store.go:366 +0x1e fp=0x1813910 sp=0x18137a8 pc=0x1fee001e
github.com/tetratelabs/wazero/internal/wasm.(*Store).Instantiate(0x1836180, {0xcf440, 0x43d7e0}, 0x1800e00, {0x7a473, 0x3}, 0x18bc9c0, {0x43d7e0, 0x0, 0x0})
  /Users/nitzan/go/pkg/mod/github.com/tetratelabs/[email protected]/internal/wasm/store.go:325 +0x2 fp=0x1813998 sp=0x1813910 pc=0x1fed0002
github.com/tetratelabs/wazero.(*runtime).InstantiateModule(0x1830c30, {0xcf440, 0x43d7e0}, {0xcfc78, 0x184a540}, {0xd05b0, 0x18d6000})
   /Users/nitzan/go/pkg/mod/github.com/tetratelabs/[email protected]/runtime.go:312 +0x20 fp=0x1813aa0 sp=0x1813998 pc=0x20cb0020
 github.com/tetratelabs/wazero.(*runtime).InstantiateWithConfig(0x1830c30, {0xcf440, 0x43d7e0}, {0x341920, 0x1c, 0x1c}, {0xd05b0, 0x18d6000})
   /Users/nitzan/go/pkg/mod/github.com/tetratelabs/[email protected]/runtime.go:277 +0x7 fp=0x1813b00 sp=0x1813aa0 pc=0x20ca0007
github.com/wasilibs/go-re2/internal.init.0()
   /Users/nitzan/go/pkg/mod/github.com/wasilibs/[email protected]/internal/re2_wazero.go:170 +0x3e fp=0x1813e28 sp=0x1813b00 pc=0x213a003e
 runtime.doInit1(0x340dd0)
   /opt/homebrew/opt/go/libexec/src/runtime/proc.go:7176 +0x19 fp=0x1813f68 sp=0x1813e28 pc=0x131a0019
runtime.doInit(...)
  /opt/homebrew/opt/go/libexec/src/runtime/proc.go:7143
runtime.main()
  /opt/homebrew/opt/go/libexec/src/runtime/proc.go:253 +0x62 fp=0x1813fe0 sp=0x1813f68 pc=0x12850062
runtime.goexit({})
   /opt/homebrew/opt/go/libexec/src/runtime/asm_wasm.s:434 +0x1 fp=0x1813fe8 sp=0x1813fe0 pc=0x14d20001

random long time findstring

hello, i have almost 500 regex to findstring to 20000 various length text,as i test some times , i found this will occur a strange circumstance:
some time is a regex will cost a long time (servel s and more),and others are ms or us,but the next time will be b regex has this problem.in my tests,every time is a different regex to has this problem,so i think may it is not regex's problem, maybe this lib has some places need to improved or changed

Unbounded memory growth when no match is found

I found this library and was excited because I heard good things about google's regex library. The benchmarks I ran showed a significant improvement over the standard library, from 2x to 10x faster. But I found unbounded memory growth that should be investigated.

I am using ubuntu 22.04 and it happens with wasm and cgo.

github.com/wasilibs/go-re2 v1.3.0 

run this, you'll see constant memory usage

var a string

func main() {
	animalRegex := regexp.MustCompile(`dog`)
	for i := 0; i < 1000000000; i++ {
		a = animalRegex.ReplaceAllLiteralString(`The quick brown fox jumps over the lazy dog`, "animal")
	}

}

Modify it like so, so there are no matches and watch how your memory grows. I ran it with GOMAXPROCS=1

var a string

func main() {
	animalRegex := regexp.MustCompile(`cat`)
	for i := 0; i < 1000000000; i++ {
		a = animalRegex.ReplaceAllLiteralString(`The quick brown fox jumps over the lazy dog`, "animal")
	}

}

Notably, this behavior is not seen with the FindString,match or split variants

v1.5.0 panic: unable to mmap memory: cannot allocate memory

my vps: CPU: 2 cores Memory: 4GiB, This problem will not occur when I fall back to [email protected][email protected]

panic: unable to mmap memory: cannot allocate memory

goroutine 1 [running]:
github.com/tetratelabs/wazero/internal/wasm.NewMemoryInstance(0xc001190280)
        /root/go/pkg/mod/github.com/tetratelabs/[email protected]/internal/wasm/memory.go:85 +0x1d7
github.com/tetratelabs/wazero/internal/wasm.(*ModuleInstance).buildMemory(0xc001070d00, 0xc001102a80)
        /root/go/pkg/mod/github.com/tetratelabs/[email protected]/internal/wasm/module.go:658 +0x2d
github.com/tetratelabs/wazero/internal/wasm.(*Store).instantiate(0xc000074b40, {0x500c848, 0x8794120}, 0xc001102a80, {0x3e6ad13, 0x3}, 0xc001178840, {0x8794120, 0x0, 0x0})
        /root/go/pkg/mod/github.com/tetratelabs/[email protected]/internal/wasm/store.go:366 +0x2c5
github.com/tetratelabs/wazero/internal/wasm.(*Store).Instantiate(0xc000074b40, {0x500c848, 0x8794120}, 0x0?, {0x3e6ad13?, 0x0?}, 0x0?, {0x8794120, 0x0, 0x0})
        /root/go/pkg/mod/github.com/tetratelabs/[email protected]/internal/wasm/store.go:325 +0x5a
github.com/tetratelabs/wazero.(*runtime).InstantiateModule(0xc0011666f0, {0x500c848, 0x8794120}, {0x501d160, 0xc001168580}, {0x5034ec0, 0xc000b73950})
        /root/go/pkg/mod/github.com/tetratelabs/[email protected]/runtime.go:312 +0x1e5
github.com/tetratelabs/wazero.(*runtime).InstantiateWithConfig(0xc0011666f0, {0x500c848, 0x8794120}, {0x7d71d70?, 0x553440?, 0xc000e9ff80?}, {0x5034ec0, 0xc000b73950})
        /root/go/pkg/mod/github.com/tetratelabs/[email protected]/runtime.go:277 +0x74
github.com/wasilibs/go-re2/internal.init.0()
        /root/go/pkg/mod/github.com/wasilibs/[email protected]/internal/re2_wazero.go:157 +0x34d

Case Insensitive option on compilation

It could be possible to add the CompileOptions parameter to the re2.go exposed functions?

type CompileOptions struct {
	Posix           bool
	Longest         bool
	CaseInsensitive bool
	Latin1          bool
}

I know that the options are not so much but, the CaseInsensitive compilation could be very useful and, currently cannot be set during compilation.

internal.Compile provides some Options, why not open them up?

re2/re2.cc:798: DFA out of memory: pattern length 309, program size 17366, list count 7888, bytemap range 49

I found solution for re2:
use re::engine::RE2 -max_mem => 8<<23; # 64MiB

And then i found cre2_opt_set_max_mem is provided inside cre2.cpp.

If Add cre2OptSetMaxMem to libre2ABI and compile options, user could input max mem args to compile expr.

Compile with -tags re_cgo under Windows

Thanks for the library. I created a fork with very minor changes that allows compilation with cgo under Windows (MSYS2 - instructions are added to the README as well). Maybe it's possible to adjust your main library - then I can get rid of my fork.

The following three files are changed:

https://github.com/markusmobius/go-re2/blob/main/internal/cre2/cre2.go

line 7: include stdint.h library
#include <stdint.h>

line 29: change of malloc signatures
void* malloc(size_t size);

line 110: change from ulong to uint64:
func Malloc(size int) unsafe.Pointer {
return C.malloc(C.uint64_t(size))
}

https://github.com/markusmobius/go-re2/blob/main/internal/cre2/cre2_re2_cgo.go
Change of CXXFLAGS:
#cgo CXXFLAGS: -std=c++17

https://github.com/markusmobius/go-re2/blob/main/README.md
A short section on how to compile with re2_cgo under Ubuntu and Windows have been added (sorry for being verbose but I am not a cgo expert and it took me a while to figure out how to even install re2 on Ubuntu).

Everything still compiles under Ubuntu fine and the tests pass on both platforms.

Crash with cgo in concurrent usage

Hello,

First, thanks for the cool project, we're experimenting with it in the context of crowdsec, and the improvements are significant (up to +100%).

However, when switching from WASM to cgo to avoid the mutex and have better concurrency, we found a consistent crash:

panic: runtime error: slice bounds out of range [:54043649582105225] with length 163

goroutine 19 [running]:
github.com/wasilibs/go-re2/internal.matchedString(...)
	/home/seb/go/pkg/mod/github.com/wasilibs/[email protected]/internal/re2.go:978
github.com/wasilibs/go-re2/internal.(*Regexp).FindStringSubmatch.func1({0xc000520010?, 0x0?, 0xa3?})
	/home/seb/go/pkg/mod/github.com/wasilibs/[email protected]/internal/re2.go:559 +0x125
github.com/wasilibs/go-re2/internal.readMatches(0xe094d0?, {0x4afd7f?, 0xa3?}, 0xc00069ac80, 0xc1, 0xc0006c6f50)
	/home/seb/go/pkg/mod/github.com/wasilibs/[email protected]/internal/re2_tinygowasm.go:173 +0x54
github.com/wasilibs/go-re2/internal.(*Regexp).findSubmatch(0xc00008e050, {0x0?, 0x0?}, 0x0?)
	/home/seb/go/pkg/mod/github.com/wasilibs/[email protected]/internal/re2.go:593 +0xab
github.com/wasilibs/go-re2/internal.(*Regexp).FindStringSubmatch(0x0?, {0x4afd7f?, 0x0?})
	/home/seb/go/pkg/mod/github.com/wasilibs/[email protected]/internal/re2.go:558 +0x105
main.parse({0x4afd7f, 0xa3}, 0x0?)
	/tmp/cc/main.go:9 +0x37
created by main.main
	/tmp/cc/main.go:21 +0x4a

You can reproduce this with the code below:

package main

import (
	regexp "github.com/wasilibs/go-re2"
)

func parse(logLine string, rx *regexp.Regexp) {
	for {
		rx.FindStringSubmatch(logLine)
	}
}

func main() {
	done := make(chan bool)
	logLine := `139.162.189.33 - - [18/Apr/2018:07:52:34 +0200] "GET /420191f0.css HTTP/1.1" 200 136 "-" "Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; rv:11.0) like Gecko" "-"`
	rx := `((((((([0-9A-Fa-f]{1,4}:){7}([0-9A-Fa-f]{1,4}|:))|(([0-9A-Fa-f]{1,4}:){6}(:[0-9A-Fa-f]{1,4}|((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3})|:))|(([0-9A-Fa-f]{1,4}:){5}(((:[0-9A-Fa-f]{1,4}){1,2})|:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3})|:))|(([0-9A-Fa-f]{1,4}:){4}(((:[0-9A-Fa-f]{1,4}){1,3})|((:[0-9A-Fa-f]{1,4})?:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){3}(((:[0-9A-Fa-f]{1,4}){1,4})|((:[0-9A-Fa-f]{1,4}){0,2}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){2}(((:[0-9A-Fa-f]{1,4}){1,5})|((:[0-9A-Fa-f]{1,4}){0,3}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){1}(((:[0-9A-Fa-f]{1,4}){1,6})|((:[0-9A-Fa-f]{1,4}){0,4}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(:(((:[0-9A-Fa-f]{1,4}){1,7})|((:[0-9A-Fa-f]{1,4}){0,5}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:)))(%.+)?)|((?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)))|(\b[0-9A-Za-z][0-9A-Za-z-]{0,62}(?:\.[0-9A-Za-z][0-9A-Za-z-]{0,62})*(\.?|\b))) )?(((((([0-9A-Fa-f]{1,4}:){7}([0-9A-Fa-f]{1,4}|:))|(([0-9A-Fa-f]{1,4}:){6}(:[0-9A-Fa-f]{1,4}|((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3})|:))|(([0-9A-Fa-f]{1,4}:){5}(((:[0-9A-Fa-f]{1,4}){1,2})|:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3})|:))|(([0-9A-Fa-f]{1,4}:){4}(((:[0-9A-Fa-f]{1,4}){1,3})|((:[0-9A-Fa-f]{1,4})?:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){3}(((:[0-9A-Fa-f]{1,4}){1,4})|((:[0-9A-Fa-f]{1,4}){0,2}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){2}(((:[0-9A-Fa-f]{1,4}){1,5})|((:[0-9A-Fa-f]{1,4}){0,3}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){1}(((:[0-9A-Fa-f]{1,4}){1,6})|((:[0-9A-Fa-f]{1,4}){0,4}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(:(((:[0-9A-Fa-f]{1,4}){1,7})|((:[0-9A-Fa-f]{1,4}){0,5}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:)))(%.+)?)|((?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)))|(\b[0-9A-Za-z][0-9A-Za-z-]{0,62}(?:\.[0-9A-Za-z][0-9A-Za-z-]{0,62})*(\.?|\b))) - ((([a-zA-Z\.\@\-\+_%]+)))? \[(((?:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9])/(\bJan(?:uary|uar)?|Feb(?:ruary|ruar)?|M(?:a|ä)?r(?:ch|z)?|Apr(?:il)?|Ma(?:y|i)?|Jun(?:e|i)?|Jul(?:y)?|Aug(?:ust)?|Sep(?:tember)?|O(?:c|k)?t(?:ober)?|Nov(?:ember)?|De(?:c|z)(?:ember)?\b)/((?:\d\d){1,2}):((2[0123]|[01]?[0-9]):([0-5][0-9]):((?:[0-5]?[0-9]|60)(?:[:.,][0-9]+)?)) ([+-]?(?:[0-9]+)))\] "(\b\w+\b) (.*?) HTTP/(([+-]?(?:(?:[0-9]+(?:\.[0-9]+)?)|(?:\.[0-9]+))))" (([+-]?(?:(?:[0-9]+(?:\.[0-9]+)?)|(?:\.[0-9]+)))) (([+-]?(?:(?:[0-9]+(?:\.[0-9]+)?)|(?:\.[0-9]+)))) "([^"]*)" "([^"]*)"( (([+-]?(?:(?:[0-9]+(?:\.[0-9]+)?)|(?:\.[0-9]+)))) (([+-]?(?:(?:[0-9]+(?:\.[0-9]+)?)|(?:\.[0-9]+)))) \[(.*?)\] \[(.*?)\])?`

	r := regexp.MustCompile(rx)

	for i := 0; i < 20; i++ {
		go parse(logLine, r)
	}

	<-done

}

Are we correct in assuming that it is intended that we can safely use the same compiled regexp concurrently? For now we're going to stick to WASM, but really looking forward to be able to use cgo as we have a hard time to get linear gains with wasm (because of the mutex ?).

Runtime cgo crash when building with -race flag and running FindAllSubmatchIndex

First of all thanks for the amazing work you did here, this is a great improvement in performance 💪
The issue that I found is the following:
It looks like there is some kind of issue with using cgo while using data race detector of go.
The output that I got is the following:

fatal error: checkptr: pointer arithmetic result points to invalid allocation

goroutine 242 [running]:
runtime.throw({0x205ba86?, 0x1e06a80?})
	/usr/local/go/src/runtime/panic.go:1047 +0x40 fp=0xc000f5f770 sp=0xc000f5f740 pc=0x44e470
runtime.checkptrArithmetic(0xc000f5f848?, {0x0, 0x0, 0x0?})
	/usr/local/go/src/runtime/checkptr.go:69 +0xac fp=0xc000f5f7a0 sp=0xc000f5f770 pc=0x41b11c
github.com/wasilibs/go-re2/internal.matchFrom(...)
	/root/go/pkg/mod/github.com/wasilibs/[email protected]/internal/re2_tinygowasm.go:73
github.com/wasilibs/go-re2/internal.(*Regexp).findAllSubmatch(0xc00072bd10, {0x556f70?, 0x10001f8f8?}, 0xffffffffffffffff, 0xc000f5f970)
	/root/go/pkg/mod/github.com/wasilibs/[email protected]/internal/re2.go:485 +0x124 fp=0xc000f5f8d0 sp=0xc000f5f7a0 pc=0x15828e4
github.com/wasilibs/go-re2/internal.(*Regexp).FindAllSubmatchIndex(0xc00072bd10, {0xc0009e5400, 0x4e8, 0x500}, 0xc0009a8990?)
	/root/go/pkg/mod/github.com/wasilibs/[email protected]/internal/re2.go:415 +0x10c fp=0xc000f5f9d0 sp=0xc000f5f8d0 pc=0x1581a8c

then every other process on this pod started crashing as well...

this only happen when I run go build using both -tags re2_cgo and -race

Expected behaviour is the same as with out the race build flag, meaning running properly without crashing.

Please let me know if any other information is needed to investigate this issue, I will also do some investigating when I have the time, but wanted to let you know.

Thanks!

Case sensitivity

Hi there,

When trying to drop-in-place replace our stdlib regexp usage, it seems that case sensitivity behaves differently, and the setting to toggle this during compilation is buried in the internal package. Is there a way to access the setting?

Thank you for your help.

wasm error: unaligned atomic

Hi, im using the regexp with 20 goroutines and getting this panic, with v1.5.2

panic: wasm error: unaligned atomic
        wasm stack trace:
        	.$189(i32)
        	.$403(i32)
        	.$534(i32,i32)
        	.$795(i32,i32,i32,i32,i32,i32,i32) i32
        	.$813(i32,i32,i32,i32,i32,i32,i32,i32) i32 [runtime.gopanic() panic.go:770]

memory consumption

hello,

When I switched my project from the standard library to go-re2 memory consumption increased a few times.

This is memory usage on standard library after 30min
701.69MB 95.28% 95.28% 701.69MB 95.28% bufio.(*Scanner).Text (inline)
10.09MB 1.37% 96.65% 16.59MB 2.25% websitedata/internal/controllers.(*App).LoadHostData
6.50MB 0.88% 97.54% 6.50MB 0.88% go.mongodb.org/mongo-driver/bson/bsonrw.(*valueReader).readString
4.51MB 0.61% 98.15% 4.51MB 0.61% regexp/syntax.(*compiler).inst (inline)

This is memory usage on go-re2 after 5 minutes

2678.14MB 83.98% 83.98% 2678.14MB 83.98% github.com/tetratelabs/wazero/internal/wasm.(*MemoryInstance).Grow
169.09MB 5.30% 89.28% 169.09MB 5.30% github.com/tetratelabs/wazero/internal/wasm.NewMemoryInstance (inline)
104.70MB 3.28% 92.56% 104.70MB 3.28% github.com/tetratelabs/wazero/internal/engine/compiler.(*engine).NewModuleEngine
94.54MB 2.96% 95.53% 94.54MB 2.96% bufio.(*Scanner).Text (inline)
85.33MB 2.68% 98.20% 85.33MB 2.68% github.com/tetratelabs/wazero/internal/engine/compiler.(*moduleEngine).newCallEngine
3.01MB 0.094% 98.30% 90.54MB 2.84% websitedata/internal/controllers.(*App).ParseWatByLine
3MB 0.094% 98.39% 342.04MB 10.73% github.com/wasilibs/go-re2/internal.newABI

Racey named groups

I get a stack trace when i use named groups and regex in a number of go routines:

wasm stack trace:
	.$170(i32) i32
	.$189(i32,i32,i32) i32 [recovered]
	panic: wasm error: out of bounds memory access
wasm stack trace:
	.$170(i32) i32
	.$189(i32,i32,i32) i32

goroutine 262 [running]:
gitlab.com/highstead/hightest/replacer.replace.func1()
	/Users/highstead/highstead/hightest/go/replacer/replacer.go:150 +0x1fc
panic({0x103924c60?, 0x14000461760?})
	/opt/homebrew/opt/go/libexec/src/runtime/panic.go:914 +0x218
github.com/wasilibs/go-re2/internal.namedGroupsIterNext(0x14000e3d400, 0x20ef0)
	/Users/highstead/pkg/mod/github.com/wasilibs/[email protected]/internal/re2_wazero.go:288 +0x2d4
github.com/wasilibs/go-re2/internal.subexpNames(0x14000e3d400, 0x2e72884?, 0x1)
	/Users/highstead/pkg/mod/github.com/wasilibs/[email protected]/internal/re2.go:864 +0xa4
github.com/wasilibs/go-re2/internal.(*Regexp).SubexpNames(...)
	/Users/highstead/pkg/mod/github.com/wasilibs/[email protected]/internal/re2.go:702
github.com/wasilibs/go-re2/internal.(*Regexp).ReplaceAllString(0x1400007e640, {0x14014ec6000, 0x7a5}, {0x1036cb1e3, 0xf})
	/Users/highstead/pkg/mod/github.com/wasilibs/[email protected]/internal/re2.go:825 +0x60

I can work around this by calling regex.SubexpNames() ahead of time, but the prefered method would probably be to put a mutex around:
https://github.com/wasilibs/go-re2/blob/main/internal/re2.go#L702C25-L702C25

Or just initialize the groupnames during the mustcompile function.

If you're open to either give me a shout and i'll pop a pull request.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.