Feature detection and feature requirement fallback about design HOT 6 CLOSED

For the first two issues, did you read/consider my response on issue #84?

For the last issue (polyfilling when a wasm feature is missing), there is a proposal for this in the backwards compatibility section of BinaryEncoding.md. The basic idea is to provide, for each op, an optional polyfill function that should be called instead if the op is unknown. Obviously, this can't be done for all features; some will be so fundamental (say, threads) that it's not sufficient just to polyfill some ops; in those cases, the hasWebAssemblyFeature test would be used before loading the module (to, e.g., load some v.1-compiled module that didn't use the feature). Also, more ad hoc and raw polyfilling could be done by hooking into the ES6 module loader pipeline which can arbitrarily manipulate the bits after they are fetched but before they are decoded by the browser. I'm pretty wary about trying to expose the bytes of an already loaded module and allowing mutation (it'd be like toSource but worse).

As for multiple wasm modules, each one would be independent, there would be no global state necessarily shared by all modules, although an application could choose to do this.

from design.

I'm really uncomfortable with shackling this to ES6 modules and a module polyfill. Do we have any examples in the wild of how this would work that shows it has reasonable performance characteristics, is straightforward to debug, and actually satisfies the requirements we care about? If it's the Right Way To Do It, we should do it, but it seems like it leaves obvious oversights. For example, you mention passing URLs to <script> or workers but there's absolutely no explanation of how this would work and how it would interact with the polyfill. Synchronous loads in <script>, loading in a worker, etc are important use cases, so if we're going to rule them out and require ES6 module imports as the only way to load wasm... well, maybe that's unavoidable, but it's a significant decision to make.

I don't understand how an optional polyfill function would solve things here. The point isn't to call out to JS to implement some sort of SIMD add, but to handle scenarios where you need to use a polyfill to implement an entire function or entire application because of significant feature differences, instead of just importing a single JS function across the FFI. Significant changes to the type system, broad feature additions, etc could all make this sort of feature detection important.

What does 'hooking into the module loader pipeline' mean here? How does that grant you access to the raw bits in a way that doesn't have the downsides of exposing module sections? I'm not sure where mutating the module bits comes into the picture, I guess it's implied by exposing a Uint8array?

Fully independent, isolated modules make sense now but it seems like that answer would fail to be sufficient as soon as dynamic linking enters the picture. Any future solution for dynamic linking would be hopelessly incompatible with an environment where two polyfills are running in the same page, or a polyfill is running alongside a native webasm implementation, wouldn't it? Are we just okay with scoping that out entirely? (I don't think that's something we have to support now, but we should think about it.)

from design.

@kg People are using the module polyfill right now to write ES6 module code that runs on current web browsers and it apparently works well enough to be used in production. (I think Ember is a popular example?) The polyfills work with <script> (leveraging the fact that unrecognized type attributes cause a browser to neither render nor error; the polyfill can then come in and grab the innerText of the <script type='module'>s and do what the spec says.

If you hook into the module loader pipeline, you have the ability to transform (and completely determine) the bytes that are fed into the decoder, allowing you to do whatever sophisticating polyfilling. I don't expect this would be the normal thing to do though: for small features, polyfilling the missing op would be sufficient; for big features, as people do w/ JS, you'd probably want to feature test first and then load different code.

The point I've tried to explain in the modules section of V1.md is that module importing != linking. Dynamic linking will necessarily be a separate and WebAssembly-specific thing that will logically be injecting new code into an existing module; module importing is about loading logically separate things where you don't even know the language on the other side of the import (modularity).

from design.

Notes on this subject from our discussions the other day:

We discussed having some fallback/feature detection support built into the executable format. The instruction table (list of named instructions assigned to indices, essentially declaring which instructions an executable uses) provides a way to identify whether an executable uses an unsupported feature. The runtime can then reject an attempt to run the executable early, giving an easy opportunity to drop to a polyfill runtime or show an error nag.

Proposed was an extension on top of this: Entries in the instruction table can point to a webasm polyfill for the instruction. This allows introducing a subset of new features or builtins such that if a native implementation is available in the VM, it is used, but if one isn't available the VM will directly call out to a blob of webasm that implements the feature. This provides a nice optional fallback strategy for small improvements.

What the loader pipeline and decode hooks actually mean is still unresolved, I think.

from design.

FWIW, the backwards compatibility section describes very briefly the polyfill idea ("However, a name may include a corresponding polyfill function (identified by index into the function array) to be called if the name isn't natively implemented."). Definitely open to PRs to further flesh this out.

from design.

We have validate. Not sure we addressed this fully, but it's been a while... Closing.

from design.