Comments (29)
Hey @guybedford, everybody,
as a hackathon project last week we did PoC of browser host in just JavaScript
https://github.com/pavelsavara/jsco
Good part: it has just 35KB!
Bad part: I cut many corners, it's a demo-ware. It can't run your component. Yet!
Live demo here https://pavelsavara.github.io/jsco/
If anyone is interested to help, please reach out directly to me or open an issue on jsco project.
There is also detailed to-do list https://github.com/pavelsavara/jsco/blob/main/TODO.md
from jco.
@DougAnderson444 You can't load it directly, but you can load the transpiled component with js-sys
from Rust. I have tried to illustrate it here:
Your Rust code will load a wasm component, and then wasm-bridge will transpile it using the Rust crate that jco uses, and then gives the result wasm module to the browser via js-sys.
For the end user, it works exactly the same way as on desktop, they can have one singe source core in Rust that loads and executes wasm components the same way you would do it in wasmtime, but it works on desktop as well as on the web.
from jco.
A browser build would be straightforward to add, and we could support all API methods except for optimize
. It would just require careful handling of the imports in the API entry point to not throw in browsers. Marking this issue to track the feature.
from jco.
Hello, I am interested in running jco transpile
directly in the browser ... well, ideally from Rust using js-sys
. I am working on a project to "run wasmtime on the web".
Current solution
The relevant part is that when it comes to wasm components, I have to tell the end user that they need to install jco, transpile the components with jco transpile
, and then convert the result again using a custom wasm-bridge-cli
command.
Ideal solution
If jco
could run directly in the browser, then the users of my crate wouldn't need to do that and the component could be transpiled on the web directly.
Making it synchronous?
I do not know how realistic would be to make it synchronous ... the entire source of jco
could (in theory) be put into a singe .js file, and that could be fed into js-sys::eval
, which is synchronous. But I assume that might be unrealistic.
A pure Rust solution
Making an async version that would work from Rust using js-sys
without touching any JS directly would be great too. The important part is that the user doesn't have to "deploy" the jco js module anywhere, it would just work out-of-the-box without any configuration.
from jco.
I couldn't find any such code in Blink, Gecko and WebKit though.
Good digging. I guess exporting both options (sync for convenience and async for non-blocking performance) and letting the user decide which they want to use would be best.
from jco.
Yes, I understand, I I think the names are a bit confusing too. There is an adapter that converts from preview1
to preview2
and it has preview1
in the name, but the output is set for preview2
. You can actually skip this step that I did by using bytecodealliance/cargo-component which has that adapter built into the build step.
Then it is linked using preview2 in smoke.rs
As far as putting it together, yes also a confusing complicated step. We really ought to write a Blog post, but this stuff was changing so often I wasn't sure if the Blog would be obsolete very soon.
from jco.
+1 as I'm running into this now when I try to import transpile
in a browser context:
// in the browser, doesn't work
import { transpile } from '@bytecodealliance/jco';
It can't be done because top level await and a module called "module" that Webpack inserts.
Even if I were to import it, could I call transpile
in the browser?
from jco.
Dug a bit deeper, I noticed that js-component-bindgen-component
exports { generate }
which output the files without any NodeJS deps:
let { files, imports, exports } = generate(component, opts);
...we could use rollup to bundle those deps into a script directly usable by the browser.
from jco.
That only works if you work on the assumption that the artifact can/should be changed for different targets. If you want to be able to deploy a single wasm component and have it run in multiple environments, having to then do a separate step for just the browser defeats a decent chunk of the purpose. Especially for use-provided logic.
from jco.
I'm confident we can streamline it all in JavaScript, and all in the browser pretty efficiently, we just need to chip away at it piece by piece.
I've already written the rollup-plugin
for direct js usage in the browser. which I suspect will be integrated into bjorn3/browser_wasi_shim#33 (comment) shortly
from jco.
@guybedford I finished writing the Rollup Plugin 🚀
https://github.com/DougAnderson444/rollup-plugin-wit-component
This plugin essentially bundles all the dependencies from
// import { generate } from 'obj/js-component-bindgen-component.js'
let { files, imports, exports } = generate(wasmBytes, opts);
into a single file that browsers can use to import
the javascript functions.
This bundling step can even occur in the browser itself! That is what the demo does.
The plugin can be used on it's own, but in my mind it would make more sense to somehow make it available from jco
as it's the go-to place for wit javascript stuff.
from jco.
@DougAnderson444 this is awesome!! import { generate } from '@bytecodealliance/jco'
should work I believe as a dependency of the plugin?
For the browser support, that definitely should be upstreamed I think, would you be interested in upstreaming this? Or is it only working because you managed to work around the FS by avoiding WIT deps and stubbing out the WASI dependencies in the build?
The online demo looks great - would be amazing to have an interactive demo.
from jco.
When I try to use import { generate } from '@bytecodealliance/jco'
I run into the Top Level Await issue because the webpack in the dist exports await
top level, but the browser can't handle it, so I had to drill down in the code until I didn't hit any node deps.
I was able to re-use most of the code from jco, including the WIT deps I think you are referring to.
Glad you agree it should be upstreamed for browsers, I am happy to help where I can. I do think jco should be exporting ES modules in the dist as well, perhaps we can incorporate that at the same time. I'm surprised you didn't use rollup for jco?
I was thinking of an interactive demo too, I really like the browser stuff over at wasmbuilder.app and I've been thinking of ways to either incorporate it there or do something similar. The demo would have to pull out the function names and input types and make a best-guess form to call the functions. I'll figure something out.
from jco.
When I try to use import { generate } from '@bytecodealliance/jco' I run into the Top Level Await issue because the webpack in the dist exports await top level, but the browser can't handle it, so I had to drill down in the code until I didn't hit any node deps.
Good to know - we can update the JCO internal build to use the --tla-compat
build option to resolve this. I wasn't sure if we needed that for downstream users, but it sounds like we do.
If we got that resolved what would be remaining to support it being a direct import of the plugin?
I do think jco should be exporting ES modules in the dist as well, perhaps we can incorporate that at the same time. I'm surprised you didn't use rollup for jco?
I'm open to changing the build to move away from ncc, not tied to it by any means.
For interactive demos, we used to have a really nice live WIT bindgen demo here - https://bytecodealliance.github.io/wit-bindgen/. It is no longer maintained though but may help for inspiration.
from jco.
We're getting there, but there still seems to be a bit of a nodejs dependency that I can't seem to get Vite to work around:
jco
> ora
> restore-cursor
> signal-exit
(then process
)
Looks like it's coming from here:
Lines 134 to 138 in dd8822c
from jco.
ora is a terminal-specific library - so I'd suggest abstracting the spinner out to an interface/proxy object, and then using ora only in the command line/node version of the library. You can potentially use https://www.npmjs.com/package/@rollup/plugin-alias or https://github.com/KeJunMao/vite-plugin-conditional-compile to provide a different version of the spinner in node and on browser, or just have a stub browser implementation that does nothing...
from jco.
A browser shim would be great, I've started some work on that in #97. If someone wants to pick this up further that would be a huge help!
from jco.
I do not know how realistic would be to make it synchronous ... the entire source of jco could (in theory) be put into a singe .js file, and that could be fed into js-sys::eval, which is synchronous. But I assume that might be unrealistic.
Compiling WebAssembly modules is always asynchronously.
from jco.
Compiling WebAssembly modules is always asynchronously.
Maybe from JS, but not from Rust. The WebAssemnly::Module::new function is js-sys
compiles bytes to a webassembly module, and does so synchonously.
You can then put this method into a JS closure and pass it to JS, and now you have a JS function that compiles bytes to wasm synchronously.
I am already using this approach in wasm-bridge; for example, here I "massage" the "instantiate" function so that it is synchronous and a "normal js function" instead of an module with exports, and here I pass the synchronous "load wasm" function to JS.
from jco.
Maybe from JS, but not from Rust. The WebAssemnly::Module::new function is js-sys compiles bytes to a webassembly module, and does so synchonously.
Looks like the new Webassembly.Module()
constructor in Javascript allows synchronous compilation, but the docs warn that it can be expensive and that you should only use it when absolutely required. It also says that a browser is allowed to throw RangeError
when trying to synchronously compile a large enough wasm module on the ui thread. I couldn't find any such code in Blink, Gecko and WebKit though.
from jco.
Keep in mind that new Webassembly.Module()
is for wasm Modules, whereas jco
is for wasm Components (the Component Model). The browser does not currently support the Component Model, hence all the shims.
What I would do for your project is use jco
to generate the JS bindings for the browser (like I do in DougAnderson444/rollup-plugin-wit-component) then call those JS bindings from Rust using js-sys
.
from jco.
What I would do for your project is use
jco
to generate the JS bindings for the browser
Yes, that's what I am doing. In fact, here is the jco transpile
command.
But this requires the user to manually convert the components using jco. If jco would run on the web, they could use the original wasm component and my crate would transpile it during runtime. This would be slower at runtime, but more convenient.
Although, I have noticed your example is using wasi, so I will definitely look into that. EDIT: Oh wait, you use the JS runtime. That definitely helps too, but I want to see an example of wasmtime rust runtime running a wit component with wasi (not really related to this issue that much, but doesn't hurt to ask).
from jco.
If jco would run on the web, they could use the original wasm component and my crate would transpile it during runtime.
That is exactly what my rollup plugin does :)
There is a bit of work to be done to incorporate it directly into jco
just looking for some time to do that.
Although, I have noticed your example is using wasi
I think you can also use wasm-unknown-unkwown
too
I want to see an example of wasmtime rust runtime running a wit component with wasi
You can check my other repo https://github.com/DougAnderson444/wit-wasm I think is what you are looking for?
from jco.
I think is what you are looking for?
Yes, many thanks. But I don't really understand how to put it together. The example at smoke.rs
uses wasi 2 preview, but the xtask
is using preview 1, so I'm not sure if I'm looking at the right files.
from jco.
Hello @DougAnderson444 , the jco rollup plugin is working well. I was able to remove the dependency on rollup
and even the plguin
from rollup-plugin-wit-component
, and only use the bindgen
"file". Here is the "js code", build and run it with this script.
If I understand correctly, the bindgen "module" is jco componentized to a wasm component and then transpiled by (essentially) itself to run on the web? That is getting pretty meta, but it works.
Future collaboration
What I wanted to ask is if I can add this folder that contains your "generate" plugin to wasm-bridge? This would allow the end user to use the same component bytes on desktop and on the web, which would be awesome.
Different imports?
Also, I noticed that the import functions are specified differently to when using jco "normally". I need to add another "layer" into the import object like this:
// from your plugin
"import-point": {
default: {
importPoint(point) {
point.x = point.x + 100;
return point;
},
},
},
instead of this:
// from using jco in command line
"import-point": {
default(point) {
point.x = point.x + 100;
return point;
},
},
This isn't anything that cannot be worked around, but it is slightly annoying. What version of jco did you use? I am using 0.9.4.
This is my wit file:
package example:protocol
world my-world {
record point {
x: s32,
y: s32,
}
import import-point: func(pnt: point) -> point
export move-point: func(pnt: point) -> point
import print: func(msg: string)
// Say "Hello" to stdout, use with wasi
export say-hello: func()
}
TS bindings also wrong
The TS bindings generated by the plugin are even worse. Not only do they not match what the code is doing, they flat out can never work, because they define two "default" functions with different signatures:
// from your plugin
export interface ImportObject {
default(pnt: Point): Point,
default(msg: string): void,
// wasi stuff...
}
Jco version 0.9.4 generates the types correctly (with --instantiation, which I am using too):
export interface ImportObject {
'import-point': {
default(pnt: Point): Point,
},
print: {
default(msg: string): void,
},
// wasi stuff ...
}
from jco.
. It also says that a browser is allowed to throw
RangeError
when trying to synchronously compile a large enough wasm module on the ui thread.
Ok, Google Chrome on mobile doesn't seem to like it:
liquislime-bevy.js:530 panicked at 'called `Result::unwrap()` on an `Err` value: JsValue(RangeError: WebAssembly.Compile is disallowed on the main thread, if the buffer size is larger than 4KB. Use WebAssembly.compile, or compile on a worker thread.
RangeError: WebAssembly.Compile is disallowed on the main thread, if the buffer size is larger than 4KB. Use WebAssembly.compile, or compile on a worker thread.
and neither does Samsung Internet, but I couldn't get the error trace. Looks like I will have to not be laze and do the asynchronous compilation after all. Anyway, that is kind of off topic.
from jco.
Hello @DougAnderson444 , the jco rollup plugin is working well. I was able to remove the dependency on
rollup
and even theplguin
fromrollup-plugin-wit-component
, and only use thebindgen
"file". Here is the "js code", build and run it with this script.
That's awesome that you were able to streamline the code! Amazing that this works in the browser =D
If I understand correctly, the bindgen "module" is jco componentized to a wasm component and then transpiled by (essentially) itself to run on the web? That is getting pretty meta, but it works.
I am a big fan of having everything compile in the browser if possible. Have you checked out https://wasmbuilder.app/?
Future collaboration
What I wanted to ask is if I can add this folder that contains your "generate" plugin to wasm-bridge? This would allow the end user to use the same component bytes on desktop and on the web, which would be awesome.
Yes that would be amazing too!
What version of jco did you use? I am using 0.9.4.
I looked back at the code, it was using 0.8.0
, I haven't had time to try with the latest version yet.
Version 0.8.0
is when Guy bumped the WIT model version, there must have been these other bugs stuck in there too.
But your method without Rollup seems to be more streamlined?
from jco.
Have you checked out https://wasmbuilder.app/?
That looks really cool. When I hover over the exports/imports, it doesn't tell me what it is (for example, signature for functions, etc), but otherwise, it is pretty interesting.
Also, I am no longer using that solution. I have code in Rust that is compiled into wasm32 and needs to load and execute wit components, and it turns out I can just use js-component-bindgen directly, instead of transpiling it by jco.
But you are welcome to use the tricks (or hacks, to be precise), from the repo I linked before to streamline the use of your rollup plugin.
from jco.
@kajacx are you trying to run that wasm32
component in the browser though? Because I was under the impression that you couldn't run a wasm32-unknown-unknown
WIT component in the browser via wasm-bindgen
(in other words, you can't run a WIT component in wasm-bindgen). Did you get this to work?
from jco.
Related Issues (20)
- How do I use `js-component-bindgen`? HOT 1
- Incorrect codegen for `wasi:http` resource `RequestOptions` HOT 2
- JCO serve error on composed component HOT 1
- Are TypeScript definitions generated by transpile supposed to include comments from the WIT file? HOT 2
- Validity of file descriptors in the context of the `SyncKit` worker HOT 3
- Unsupported URL wasi when running in Node.js HOT 2
- Transpile bindings for `u32` map to `number` in *.d.ts HOT 3
- Non-conformant exceptions should unwrap as traps HOT 2
- Jco transpile: implementation of wasi-nn for Web HOT 2
- Source phase imports output
- Performance
- JS/WIT Reference
- Lifting `list<borrow<res>>` during transpilation emits invalid code HOT 3
- Resource transferring seems not working with wasi-virt HOT 1
- Crash on startup with an out-of-memory error when environment variables are large HOT 3
- Provide wasm-tools APIs in the browser HOT 2
- `definedResourceTables` codegen generates invalid JS with `--instantiation async` HOT 1
- plain enums are generating complex typescript HOT 8
- Question: `jco types` generating namespaces instead of modules? HOT 3
- Better instantiation workflows HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from jco.