Will hyperscan full support pcre library and capture feature. about hyperscan HOT 4 CLOSED

Hi Tidy,

I believe that Hyperscan's existing prefiltering support should allow you to do what you're describing here. The implementation would likely look quite similar to the Hyperscan integration used by the rspamd project.

In prefiltering mode, Hyperscan accepts many pattern forms that it cannot implement in normal operation, such as lookahead/lookbehind assertions, and replaces them internally with broader constructs that may match more often.

For your application, in broad strokes:

At pattern compile time, take your list of PCRE patterns and compile them all into a Hyperscan database, with each pattern marked with the HS_FLAG_PREFILTER and HS_FLAG_SINGLEMATCH flags. Each pattern would also need to be compiled with PCRE.

At scan time, for a given block, you would first scan it with the Hyperscan prefiltering database and collect the list of patterns that match. For each pattern that produced a match with Hyperscan, call pcre_exec on the block to confirm the match with PCRE.

Assuming that your patterns are fairly specific (and unlikely to all match on every block), this will allow you to considerably reduce the number of PCRE scans you need to perform.

from hyperscan.

Chimera was a separate project to Hyperscan, and never shipped as a product. Chimera was a hybrid of Hyperscan and libpcre - it performed first stage matching with Hyperscan and then used libpcre to calculate the full match and capturing groups.

These features will not be introduced to Hyperscan as they aren't compatible with Hyperscan's streaming support and match semantics. We are considering introducing a project that provides these features, but it is still in the design phase.

Is there anything you can tell us about your use case for these features?

from hyperscan.

Thanks Matt for your kind response!

What's the name of the new project, will we be allowed to participate in the new project?

Here is my use case:
I need search certain pattern using pcre signature from network-based binary traffic and capture these information from it using block mode.
The issues I am facing:
Lookahead and Lookbehind Zero-Length Assertions not supported well , so we can't capture the exact matching string. Not sure if this issue can be resolved by Prefiltering Mode or not.

Thanks,
Tidy

from hyperscan.

@jviiret , Got it. Thanks you very much for your detail explanation!
So, we can close the thread.

from hyperscan.