Darwin builds (server/plugin) about bwt HOT 22 CLOSED

@shesek Unfortunately I don't have answers as I stumbled upon osxcross yesterday just out of curiosity while looking for a solution to compile bwt in Docker.

Before I found this darwin builder Docker image, I was able to do all the steps manually from the fresh debian:buster-slim (this manual helped me a lot). The main obstacle to automating all the things that I discovered is that macOS SDK needs to be downloaded from somewhere.

One possibility is to fetch the XCode xip installer from Apple, then unpack it and repack SDK as described in osxcross readme. Unfortunately, Apple doesn't allow downloading XCode without login. Somebody has to download it manually anyway (all ~8Gb, of which 48Mb only is needed).

Another option is to fetch pre-packaged SDK from somewhere, but this leads to the same issue of trusting somebody.

from bwt.

@shesek I was able to successfully cross-compile for macOS in Docker using https://github.com/joseluisq/rust-linux-darwin-builder
Could this be integrated into the release script? Should work on CI too. It would be nice to have signed binaries for macOS.

from bwt.

I think I managed to cross-compile for macOS (via oxscross), but don't have a way to actually test this.

Would anyone like to be a guinea pig? 🐹

from bwt.

from bwt.

+1

from bwt.

The options for supporting macOS are: (1) finding a trusted maintainer to help build and publish the macOS releases, (2) getting myself a macOS, or (3) trusting some cloud provider to build it remotely.

I wouldn't feel comfortable trusting a cloud provider for this, so that leaves us with finding a trusted maintainer, or someone donating to sponsor a macOS that I could use for that (which would also enable me to make iOS releases for Spark 😁)

from bwt.

You can also setup the bwt electrum server using docker instead of using the plugin.

This is somewhat more complicated, but not terribly so. Basically docker run --net host -v ~/.bitcoin:/bitcoin shesek/bwt:electrum --xpub <xpub> and you should have the electrum server up and running.

from bwt.

@leshik Nice, this makes things much easier! I was under the impression that you need a mac for this. Is osxcross considered stable? Does it have any disadvantages compared to a real macOS host? Does it support signing too?

I'm a bit wary about introducing third party docker images into the build process, but its nothing that some auditing and replicating the dockerfile into this repo couldn't solve.

from bwt.

@shesek I develop mostly on a mac so I'm happy to help with releases. I believe we share a few connections (Tel Aviv) if you want to explore this option offline.

from bwt.

@pox Sounds good, happy to explore this! Being able to do an IRL exchange of release pubkeys is also a big plus.

Are you coming to the bitcoin embassy from time to time? Perhaps we could meet there?

I'm reachable at [email protected] or on freenode (same nickname).

from bwt.

@shesek sent an email.

from bwt.

Thanks @pox!

The macOS releases are available here: https://github.com/shesek/bwt/releases/tag/osx-rc

I recently also made the builds reproducible. If you'd like, you can verify that checksums against the builds made on Travis CI (which pulls the code from GitHub). Scroll to the very end of the build log and you'll see the checksums following -----BEGIN SHA256SUM-----.

from bwt.

Doesn't seem to quite work.

When I try executing the bwt binary:

dyld: Symbol not found: _rustsecp256k1_v0_1_1_context_no_precomp

😞

@shesek I see you're using Travis. Have you tried executing the binary as part of the build? Could probably be used as some basic smoke test.

from bwt.

Ugh. I've been running into similar errors in the complication step, I thought I solved them with some compiler flags but it looks like I just pushed it away to the runtime. :(

I'm using Travis, but it's also cross-compiling from Linux environment (to match the binaries produced by my local environment), so I can't run the binary as part of the build. I can, however, create a separate macOS Travis environment specifically for that purpose -- which is an excellent idea. Thanks for bringing that up!

I'll comment here when I make progress. Cheers.

from bwt.

Ok, I think I got it working for realz this time. :) Care to give it a try @pox? (the binaries are available on that same osx-rc tag)

from bwt.

@shesek binary seems to work. Is there any sort of acceptance test you'd like me to perform? Or are you just happy with knowing it's running (spitting out usage, for example)?

$ ./bwt
 ERROR bwt::hd > Please provide at least one xpub to track (via --xpub or --bare-xpub).
Error: no xpubs provided

from bwt.

@pox Nice, thanks!

Just seeing that its running without crashing down is probably good enough, but it would be cool if you have an handy bitcoin node and could give this a more throughout test.

Basically, if you have bitcoind running with the default directory and ports, ./bwt --xpub <xpub> should Just Work (tm). Otherwise you'll also also need to set --bitcoind-url and --bitcoind-dir. Also recommended that you use a separate bitcoind wallet (bitcoin-cli createwallet <name> true and --bitcoind-wallet <name>).

After it finishes rescanning, you can try connecting to the electrum server on port 50001 (no ssl, so --server 127.0.0.1:50001:t with the t), or try the http api with something like curl localhost:3060/utxos

from bwt.

So far so good, it started a rescan. Will update.

btw, definitely a good idea to have a CLI option to specify rescan block height.

from bwt.

@shesek I got Electrum to connect to it, but it didn't find any transactions.
In all likelihood this isn't an issue with the osx build. Probably just a matter of usage. I can elaborate if you want (I've used some known public xpubs that have transactions in the last few months, but they don't show up).

Also, I only tested the standalone binary, not the plugin. Is the plugin also available on osx, and if so would you like me to test it? I would need instructions how.

from bwt.

btw, definitely a good idea to have a CLI option to specify rescan block height.

This is possible via --xpub <xpub>:<height> or using a date with --xpub <xpub>:<yyyy-mm-dd>.

(I know you already figured that out, just mentioning it for others.)

from bwt.

Implemented in bc9162a.

from bwt.

Released in v0.1.5! https://github.com/shesek/bwt/releases/tag/v0.1.5

from bwt.