Benchmarks & examples about flexible-vectors HOT 3 OPEN

Interesting discussion; thanks for the ping! This is a good list from @lemaitre. To break things down a bit, I sort of think there are two competing goals here:

• Apples-to-apples comparisons with native (and fixed-width WASM) SIMD, for which "standard" benchmark suites are the right thing because of their availability and portability.
• More realistic niche use cases, which can be oddballs that "stretch" the ISA a bit more to reveal interesting/challenging cases that are likely to arise in the wild.

To make things more complicated, there is also a separate dimension to distinguish "algorithms" (kernels, such as GEMM) from "applications" (such as an H.264 encoder). The former is probably the better focus, at least early on.

I think the thing that makes benchmarking SIMD ISAs/algorithms especially hard (compared to "normal" benchmarks) is that there is a huge range in implementation qualities. A naively auto-vectorized version of a simple 3x3 matrix product, for example, will be far worse than an expert-tuned implementation—and the expert-tuned implementation will differ radically for different ISA targets! Which means that, if you invent a new SIMD ISA (or WASM extension), it can be really hard to say whether it's "better" without doing a bunch of incredibly painful tuning for that architecture.

The problem is that the straightforward/obvious implementations of a given benchmark could be misleading w/r/t what the "peak" performance could be, given infinite programmer effort. "Infinite programmer effort" may seem like a fantasy, but in practice, the world has shown that it is willing to expend seemingly-infinite effort for high-value libraries like BLAS. That's not feasible to do for early-stage designs, so there will inevitably be some artistic guesswork involved about how much a given ISA feature will matter for industrial-strength vectorized implementations.

from flexible-vectors.

Also, just because it came up on HN today, here's a nice explanation of a somewhat bonkers optimized fixed-length reduction that's worth pondering.

from flexible-vectors.

@sampsyo there is a conversation about code to compile in #27 (comment). There a few interesting things in that PR, I am trying to separate them out a bit. From @lemaitre:

I know that's the tricky bit... Here is a small list of algorithms that definitely need to be implementable:

• gemm (matrix multiplication)
• reductions (not necessarily additions)
• Stencils
• prefix sums (1D)
• Integral images (like prefix sums but in 2D)
• interpolations
• On-The-Fly narrowing or widening (ie: chaining conversions with computation instead of having a separate conversion step)

I am also personally interested in more algorithms:

• Run-Length Encoding
• Union-Find (graph processing)
• Background subtraction (like Sigma-Delta)

from flexible-vectors.