bwt-dev / bwt Goto Github PK

Not yet implemented in rust-bitcoincore-rpc: rust-bitcoin/rust-bitcoincore-rpc#27

Implement coin selection in bwt, to be able to fund transactions using a specific hd wallet or addresses. This is currently not possible using bitcoind's fundrawtransaction.

Inspired by EPS:

https://github.com/chris-belcher/electrum-personal-server/blob/master/electrumpersonalserver/server/electrumprotocol.py#L32-L51

I get this using with Eclair mobile app:

INFO bwt::electrum > [192.168.1.148:39430] connected peer
ERROR bwt::electrum > [192.168.1.148:39430] receiver failed: invalid request - maybe SSL-encrypted data?: [22, 3, 1, 0, 218, 1, 0, 0, 214, 3, 3, 141, 209, 205, 69, 67, 214, 195, 169, 204, 56, 252, 162, 84, 8, 37, 254, 214, 151, 104, 234, 116, 137, 158, 222, 68, 102, 229, 54, 104, 189, 206, 166, 32, 173, 223, 108, 58, 249, 196, 20, 103, 78, 187, 180, 252, 172, 199, 164, 57, 199, 61, 153, 146, 47, 11, 112, 245, 24, 215, 128, 150, 228, 249, 100, 32, 0, 24, 19, 1, 19, 2, 19, 3, 192, 44, 192, 43, 192, 47, 192, 48, 192, 19, 192, 20, 0, 156, 0, 47, 0, 53, 1, 0, 0, 117, 0, 23, 0, 0, 255, 1, 0, 1, 0, 0, 10]
INFO bwt::electrum > [192.168.1.148:39430] disconnected peer

Does it support bitcoind pruned mode, as eps does? In particular the Electrum plugin?

• Reproducible build environment in a Docker container (e992262)

• Build in Travis-CI and report checksums (72cfb45)

Docs: https://github.com/shesek/bwt#reproducible-builds

It is disabled because the OpenSSL dependency enabled by it causes the following error on ARM targets:

/lib/aarch64-linux-gnu/libc.so.6: version `GLIBC_2.25' not found (required by bwt)

See: chris-belcher/electrum-personal-server#174

Blocked by #14

Allow to specify a custom executable to be used for broadcasting transactions to the network, instead of using bitcoind's sendtawtransaction. This can be preferable for privacy reasons.

Electrum Personal Server has this implemented as the broadcast_method config option.

• Add --tx-broadcast-cmd <cmd> option to allow specifying a custom command for broadcasting txs (c9d76e8)
• Add --tx-broadcast-p2p <proxy-addr> option to broadcast directly to the P2P network using a socks proxy (or alternatively, using a public block explorer api, which is simpler)
• Add --tx-broadcast-tor option as a shortcut for setting --tx-broadcast-p2p with tor
• Enable --tx-broadcast-tor by default if tor is found on the machine?

Probably because the cookie changed.
bwt can maybe try rereading the cookie if auth fail.

Support for tracking multisig wallets.

Types:

• m-of-n p2sh + sort pubkeys for each script
• m-of-n p2sh-p2wsh + sort pubkeys for each script
• m-of-n p2wsh + sort pubkeys for each script
• m-of-n p2sh + do not sort pubkeys for each script
• m-of-n p2sh-p2wsh + do not sort pubkeys for each script
• m-of-n p2wsh + do not sort pubkeys for each script

The "xpub" format could be extended in a similar way to NBXplorer (the wallet backend for BTCPay).

https://github.com/dgarage/NBXplorer/blob/8715d3d1ba2a519bd67678ed4be050487f2c9994/docs/API.md#derivationScheme

Just asking here to see if there is some sort of conceptual ACK for starting working on this
romanz/electrs#227

It Would be nice if both implementations could expose a gRPC endpoint and the API could be defined by a typed schema.

Why?

1. gRPC gives HTTP/2 for free
2. http2 means binary encoding, more efficient than utf8 JSON strings
3. Protobuf is a perfect match, could give backward and forward compatibility when making changes to the APIs
4. Server-side streaming
5. Automatically generate stubs and clients from the single .proto model spec

• Add --tx-broadcast-p2p <proxy-addr> option to broadcast directly to the P2P network using a socks proxy
• Add --tx-broadcast-tor option as a shortcut for setting --tx-broadcast-p2p with tor
• Enable --tx-broadcast-tor by default if tor is found on the machine?

Provide a docker image that sets up nginx with certbot or a self-signed cert, as a reverse proxy to the http/electrum servers.

For the http api, this could also throw basic http auth into the mix (this does not solve authentication for the electrum server though, which cannot be done with the current electrum protocol and would require a separate authentication layer, like a vpn or an hidden service).

This could also potentially be implemented as part of the rust server itself (there are a few rust libaries for integrating with letsencrypt), but this is quite more complicated and doesn't seem to add much value over the simpler docker image option.

Already mostly implemented, they should get tracked properly once imported into bitcoind.

At startup:

• --address
• --addresses-file

At runtime:

• PUT /address/:address
• POST /address
• Don't lock the indexer/store while rescanning

I already pasted the bwt file from the compressed file on the directory said, I wasn't sure if it is /electrum/plugins like the root dir or ~/.electrum/plugins home dir one, both seems doesnt work atleast on my end, maybe I am missing something thanks!

I am using a standalone executable electrum 3.3.8

Since users are connecting to their own full node, there's little value in having Electrum verify the SPV proofs, and we can save some resources by not generating them. To be used with Electrum's --skipmerklecheck.

• Implement --electrum-skip-merkle (2390f2d)
• Auto-enable --skipmerklecheck and --electrum-skip-merkle in the Electrum plugin (e6a3399)

Due to the way ypubs/zpubs are handled to make them compatible with rust-bitcoin, their original version bytes information is lost, making them always serialize with BIP32's original version bytes and not with SLIP132's (i.e. as xpub... rather than ypub... or zpub...).

This effects the way xpubs are shown on console and in the hd wallets api, but works apart from that.

This could be fixed either by adding ypub/zpub support to rust-bitcoin (not sure if that's likely to happen), or by writing some custom serialization code to inject the original version bytes back in.

Tried to run it as stated in the documentation.

The command I used

docker run -it --net host -v ~/.bitcoin:/bitcoin shesek/bwt --xpub [...]

The output I got

bwt: error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory

Would be nice to have officially built bwt electrum plugin for MacOSX, without having to download rust toolchain, especially for normies. Thanks

Ubuntu 19.10, Core v0.19.1, see electrum.log

Given an xpub, there should be a way to override the path. Perhaps using some keywords.

If not specified the default is %CHANGE%/%INDEX%

ie

CUSTOM_PATH="123/%CHANGE%/1/8/23/%INDEX%/2/4"

will result in the first receive address paths of:

• 123/0/1/8/23/0/2/4
• 123/0/1/8/23/1/2/4
• 123/0/1/8/23/2/2/4

and the first change address paths of:

• 123/1/1/8/23/0/2/4
• 123/1/1/8/23/1/2/4
• 123/1/1/8/23/2/2/4

Some things to consider as design choices:

1. Should it be possible to have %CHANGE% come AFTER %INDEX%
2. Should it be possible to omit %CHANGE% (if ie. an exchange would rather explicitly specify change address(es) for whatever reason)
3. Should it be further abstracted where variables like %CHANGE% and %INDEX% can be created/added and given parameters (ie. Creating a non-hardened "BIP44 Account"-like system where you only check the next number if there is tx history in any receive address of the previous number)

I would say 1=no, 2=yes, 3="someday... maybe"

Bitcoin Core only lists the fee for transactions in the "send" category with inputs that all belong to the wallet (ie. not coinjoined). Figuring out the fee of "receive" transactions or transactions with external inputs require looking up the spent previous outputs and summing them up.

This could alternatively be implemented for unconfirmed transactions only using getmempoolentry, which would be much more efficient. This is probably acceptable because we only really care about the fee of unconfirmed transactions.

GET /block/tip returns the block height under :hash and the block hash under :height

bwt: error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory

Reported by nakomopoulos: https://twitter.com/nakomopoulos/status/1294915929684549632

I got this error when I tried to connect to my Bitcoin Node on plugin log box:

ERROR bwt » JSON-RPC error: RPC error response: RpcError { code: -13, message: "Error: Please enter the wallet passphrase with walletpassphrase first.", data: None }

I have only one wallet on my Bitcoin core which was imported by Electrum Personal Server, I left the wallet textbox blank.

My Bitcoin core is 0.19.1 and Electrum 3.3.8.

With the following regtest getblockchaininfo result bwt gets stuck at this loop. I don't know why Bitcoin Core is reporting initialblockdownload as true ... perhaps we should just compare blocks and headers ourselves? Although that could be prone to error the moment the node starts up and hasn't fetched new headers yet.

$ bitcoin-cli -regtest getblockchaininfo
{
  "chain": "regtest",
  "blocks": 106,
  "headers": 106,
  "bestblockhash": "584a14a4d71fa98be9297f7ff70754533f6778b9e3ab4ba9fb2340fbe3b1e88b",
  "difficulty": 4.656542373906925e-10,
  "mediantime": 1592948117,
  "verificationprogress": 1,
  "initialblockdownload": true,
  "chainwork": "00000000000000000000000000000000000000000000000000000000000000d6",
  "size_on_disk": 33373,
  "pruned": false,
  "softforks": {
    "bip34": {
      "type": "buried",
      "active": false,
      "height": 500
    },
    "bip66": {
      "type": "buried",
      "active": false,
      "height": 1251
    },
    "bip65": {
      "type": "buried",
      "active": false,
      "height": 1351
    },
    "csv": {
      "type": "buried",
      "active": false,
      "height": 432
    },
    "segwit": {
      "type": "buried",
      "active": true,
      "height": 0
    },
    "testdummy": {
      "type": "bip9",
      "bip9": {
        "status": "defined",
        "start_time": 0,
        "timeout": 9223372036854775807,
        "since": 0
      },
      "active": false
    }
  },
  "warnings": ""
}

Collisions are highly unlikely for typical bwt usage, but might become an issue in setups where there are many dynamically-managed xpubs, for example if each customer is given its own xpub for deposits.

This could be resolved by using the full 160 bit identifier, or possibly by using the descriptor checksums as the primary identifier instead of the bip32 fingerprint (once descriptor-based tracking is implemented, #1 / #37).

Similarly to /txs/since/:height, but returning events instead of transactions. Could be called /events/since/:height.

This would make it easier to catch-up with missed events, without having to implement separate code to extract relevant updates out of transactions.

Internally, bwt should create the events based on the transaction data, rather than keeping the events themselves around. This ensures that no events will be missed even if bwt is down while they happen.

bwt is waiting for completion of bitcoind's IBD(initial block download).
Connecting to bitcoind running regtestmode, this could be take long time.
I know bwt is not responsible for this. But I just want to know how can I avoid this.
Maybe I should ask how can I make finish IBD mode on regtest to bitcoin core community.

bwt currently constructs the electrum spv proofs for transactions using the list of txids in the confirming block (retrieved via getblock with verbose=1), which is a rather large list that needs to be fetched separately for every transaction.

This could be improved by converting the bitcoind merkleblock spv proof into the format expected by electrum, which is implemented by EPS here.

Ideally though, users should probably just configure electrum with --skipmerklecheck and bwt with --electrum-skip-merkle, as the spv proofs don't really add much in the context of a trusted self-hosted full node setup.

When running:
docker-compose up --remove-orphans I get the following error:

Recreating docker-bwt-shesek_bwt_1 ... done
Attaching to docker-bwt-shesek_bwt_1
bwt_1  | bwt: error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory
docker-bwt-shesek_bwt_1 exited with code 127

Would it be possible that anyone has a good docker-compose.yml file I could use as a starting point?