`dart run wasm:setup` fails (on windows? latest dev? not sure) about wasm HOT 24 CLOSED

Sounds like that's not quite what we want, since we're using the clang linker to bundle it all together.

The option of wrapping and re-exporting all the functions wouldn't be too bad, since we already have a python script that generates FFI boilerplate. It would probably be easy to adapt it to generate rust boilerplate too.

from wasm.

Ah. #6 is probably the fix.

from wasm.

Hmm. This is a different file not found error, so that PR won't help.

I haven't tried to get it working on windows yet, but it should be pretty straightforward. Just need to teach setup.dart where the directories are.

I'll keep this bug open to track windows support.

from wasm.

We'll have to use pkg:path or similar I think to enable windows. Thanks, @eseidel

FYI @mit-mit who was going to try out windows.

from wasm.

It looks like the problem is just that all calls to Uri.path need to be replaced by Uri.toFilePath(). I'm not entirely sure the difference, but .path seems to produce paths like `/C:/foo/bar/baz" instead of "c://foo/bar/baz" and things get mad.

from wasm.

I'm close, but haven't gotten past:

   Creating library C:\Users\micro\Documents\GitHub\wasm\.dart_tool\wasm\libwasmer.lib and object C:\Users\micro\Documents\GitHub\wasm\.dart_tool\wasm\libwasmer.exp
LINK : warning LNK4098: defaultlib 'MSVCRT' conflicts with use of other libs; use /NODEFAULTLIB:library
wasmer.lib(getrandom-3006dbff49456f8a.getrandom.bdwb2tw9-cgu.0.rcgu.o) : error LNK2019: unresolved external symbol __imp_BCryptGenRandom referenced in function _ZN9getrandom9getrandom17h649740f31a8f747eE
C:\Users\micro\Documents\GitHub\wasm\.dart_tool\wasm\libwasmer.so : fatal error LNK1120: 1 unresolved externals
clang++: error: linker command failed with exit code 1120 (use -v to see invocation)
FAILED with exit code 1120 `clang++ -shared -lws2_32 -lwsock32 -ladvapi32 -lcrypt32 -luserenv -target x86_64-pc-windows-msvc C:\Users\micro\Documents\GitHub\wasm\.dart_tool\wasm\dart_api_dl.o C:\Users\micro\Documents\GitHub\wasm\.dart_tool\wasm\finalizers.o C:\Users\micro\Documents\GitHub\wasm\.dart_tool\wasm\x86_64-pc-windows-msvc\release\wasmer.lib -o C:\Users\micro\Documents\GitHub\wasm\.dart_tool\wasm\libwasmer.so`

eseidel@71256ed

from wasm.

OK, I have it linking and finding the wasmer.dll, just not yet able to find symbols.

https://github.com/eseidel/wasm

from wasm.

OK, I have it linking and finding the wasmer.dll, just not yet able to find symbols.

https://github.com/eseidel/wasm

Woo hoo!

I'll let you run w/ this. Please submit a PR. I'm super slow since I don't have a windows machine locally.

from wasm.

Hmm. Linking, but no symbols exported:

PS C:\Users\micro\Documents\GitHub\wasm> dumpbin.exe /exports .\.dart_tool\wasm\wasmer.dll
Microsoft (R) COFF/PE Dumper Version 14.28.29914.0
Copyright (C) Microsoft Corporation.  All rights reserved.


Dump of file .\.dart_tool\wasm\wasmer.dll

File Type: DLL

  Section contains the following exports for wasmer.dll

    00000000 characteristics
    FFFFFFFF time date stamp
        0.00 version
           1 ordinal base
           8 number of functions
           8 number of names

    ordinal hint RVA      name

          1    0 00001AE0 set_finalizer_for_engine
          2    1 00001C30 set_finalizer_for_func
          3    2 00001B70 set_finalizer_for_instance
          4    3 00001C00 set_finalizer_for_memory
          5    4 00001BD0 set_finalizer_for_memorytype
          6    5 00001B40 set_finalizer_for_module
          7    6 00001B10 set_finalizer_for_store
          8    7 00001BA0 set_finalizer_for_trap

  Summary

        3000 .data
        6000 .pdata
       1C000 .rdata
        2000 .reloc
       56000 .text
        1000 _RDATA

I haven't done (professional) windows development in at least 10 years... this may be a while paging this all back in.

from wasm.

I think the problem all stems from how we're holding clang here.

The dart_api_dl.o file (not even sure that's the right extension on windows?) has the symbols in question. However:

072 00000050 SECT1  notype ()    External     | Dart_InitializeApiDL

And as far as I can tell, when it's built, it's built with the proper DLL exports per https://github.com/dart-lang/sdk/blob/master/runtime/include/dart_api.h

But this is where we're getting to the end of my windows knowledge. :) I should probably spend my hobby hacking elsewhere for a bit.

However that doesn't end up exported in linked wasmer.dll.

from wasm.

I have a windows machine, so I can try to debug the linker issues when I get a chance, but I'm a bit busy with other things atm. Is windows support considered a blocker for release?

from wasm.

from wasm.

If you need any help, feel free to ping me :-). I work at @wasmerio.

from wasm.

Thanks @Hywan – any suggestions for building on windows welcome!

from wasm.

I suspect if someone with windows knowledge (or anyone with enough time) and wrote out the cl.exe and link.exe versions of those clank lines it would work fine. The symbols are clearly in the .o files (maybe they should be renamed .obj on windows?) and just need to figure out the right invocation of clank to get them to end up exported in the .dll.

https://github.com/eseidel/wasm/blob/main/bin/setup.dart#L218 is where I got to and stopped.

from wasm.

@kevmoo I don't know how Windows work exactly. I've received a new machine with Windows on it very recently, I will use it to help :-). Are you using wasmer.dll from our release package? Are you building Wasmer from the sources?

from wasm.

The scripts are here: https://github.com/dart-lang/wasm/tree/main/bin

We're building the wasmer-c-api as a staticlib using cargo (output is wasmer.lib) and then linking it together with some extra functions (finalizers.cc and some dart API stuff) into a dll.

One thing I noticed that you might be able to help with, @Hywan, is that when I do dumpbin /exports wasmer.lib, it doesn't seem to be exporting any functions. This might be the root of the problem. Are we missing something in the Cargo.toml to get this staticlib to work on Windows?

from wasm.

Yeah, I've noticed that too with @jubianchi, and we need to spend time debugging that. I'll try to find time to focus on that :-)!

from wasm.

OK I remember; forget my previous comment, it was a particular usecase. Your issue is very likely to be related to the RFC 2771 Re-exporting C symbols for cdylib. When you're doing https://github.com/dart-lang/wasm/blob/866e8c3beb45b2a6ceceb4cccf7a10a1c296662e/bin/wasmer.rs#L1, it means you want to re-export C symbols from wasmer_c_api (marked with extern "C" + #[no_mangle]), and Rust doesn't re-export them. It's unfortunate :-/.

Some solutionx exist, like wrapping all functions one by one, but it's not ideal.
There is other solution rust-lang/rfcs#2771 (comment) but it doesn't work in all cases.

I've found an old discussion we had together in the Wasmer's repository, wasmerio/wasmer#1631 (comment), where we suggest to you use rlib as the crate-type. Can you add rlib in addition to staticlib inside https://github.com/dart-lang/wasm/blob/866e8c3beb45b2a6ceceb4cccf7a10a1c296662e/bin/Cargo.toml#L7? Not sure it will solve the problem, but we can try!

Another documentation is https://doc.rust-lang.org/reference/linkage.html.

from wasm.

I tried adding rlib, and it produces a file called libwasmer.rlib, but it's only 5KB (wasmer.lib is 38MB). What's the rlib used for?

from wasm.

Quoting https://doc.rust-lang.org/reference/linkage.html:

--crate-type=rlib, #[crate_type = "rlib"] - A "Rust library" file will be produced. This is used as an intermediate artifact and can be thought of as a "static Rust library". These rlib files, unlike staticlib files, are interpreted by the compiler in future linkage. This essentially means that rustc will look for metadata in rlib files like it looks for metadata in dynamic libraries. This form of output is used to produce statically linked executables as well as staticlib outputs.

from wasm.

If we can do anything on the Wasmer' side to help you or save you time, please ping me.

from wasm.

Turns out this is more of a windows linker issue than a Wasmer or rust issue. Clang delegates linking to the system linker, and the Windows linker is very different to Mac and Linux. That's why on Windows we usually need to put __declspec(dllexport) in front of our C functions. I don't know what the Rust equivalent of that is, but there's a work around in case a library didn't do that.

You can pass a DEF file to the Windows linker to tell it what to export. This file has a very simple format, and I've updated our boilerplate generating python script to do this.

Most of the tests now pass on Windows, but I'm seeing issue #14 in fn_import_exception_test. No point blocking the other improvements on fixing that though, so I'll send a MR.

from wasm.

Thank for your investigation! Any contribution is welcome on wasmerio/wasmer :-). Is there anything I can do to help?

from wasm.