Problem with musl and fat runtime? about python-hyperscan HOT 5 OPEN

either way i think you're right, i'll set march but only on musl

• update manylinux-hyperscan build to conditionally set march=core2 on alpine
• might be able to remove FAT_RUNTIME completely from build because it's default on linux
• update docs for build process, add support for newer AVX512* that require manually enabling support when building HS: https://intel.github.io/hyperscan/dev-reference/getting_started.html#fat-runtime
• update this project to use newer container and release new version (might be able to get away with keeping existing release and bumping build number for musl wheels, i'll try)

from python-hyperscan.

i can disable fat runtime and set march to core2 for just the musl wheels, but that would mean applications running on alpine on hosts that support AVX2 would still use SSE3, right?

is this still a pain point for you? i'm trying to think of the best universal way to support this, and other instruction sets that are newer (like AVX512, which requires manual config at buildtime and isn't turned on with fat runtime) that might require changing the default build configuration.

my first thought is before changing the build configuration across the board for all musl-based wheels, we can avoid the crash on processors with the minimum required march by testing for libc/ifunc support and turning off fat runtime, and setting march. that way the source dist still builds for the minimum requirements. would that be a good compromise?

from python-hyperscan.

I have a workaround locally. It's not pretty, but it works.

But, I still wonder if this wheel is technically not valid - aren't cpu features part of the ABI? So if you are following the PEPs you'd have to build for the same minimum cpu that eg alpine uses?

In which case the "right" thing to do by the spec is to make the wheel work everywhere, and make the sdist for people with beefier processors.

from python-hyperscan.

ok this will require a change in the underlying docker image i maintain with hyperscan and deps needed for static linking, but correct me if i'm wrong, based on your logs it looks like HS properly turned fat runtime off, so i'm guessing -march=native breaks on musl?

from python-hyperscan.

Yes, afaict you are right. fat runtime is turned off so -march=native is a no go, because the GitHub runners have AVX.

from python-hyperscan.