softwareverde / bitcoin-verde Goto Github PK

Bitcoin Verde is a Java full-node implementation of the Bitcoin Cash protocol. Fully indexed, Bitcoin Verde is a unique, from the ground-up, implementation. Bitcoin Verde provides a block explorer, development library, and network implementation diversification.

Home Page: https://bitcoinverde.org

License: MIT License

Shell 0.53% Python 0.01% Java 95.55% CSS 1.02% HTML 0.75% JavaScript 2.15%

bitcoin-verde's Introduction

Bitcoin Verde v2.4.0

Patch Notes

v2.4.0

May 2023 Support
- Cash Tokens
- P2SH32
- Transaction Size Rules
CHIPNet Support
Improved Stability
Improved IBD Syncing
Improved Pruning Mode
- Pruning Mode may now be run with Indexing Mode for limited explorer functionality.
Minor bug fixes.

v2.2.0

Electrum Server
- Run your own electron server with Bitcoin Verde.
- Enable bitcoin.indexBlocks = 1 in server.conf and start the node. (NOTE: Indexing may take a long time to complete and requires around 600GB of disk space.) Once indexing has completed (the status may be checked via the explorer), run the electrum module via the run-electrum.sh script. NOTE: Most electrum clients require an SSL certificate (self-signed certificates are supported). Self-signed SSL scripts may be generated via the ssl directory; the keyfile myst be converted via the pem2pkcs script and be configured within server.conf.
May 2022 Upgrade Support
- Upgrade and activation support for the 2022-05-15 BCH upgrade has been added.
- Includes 64-bit Script Integers, Multiplication opcode, and Transaction Introspection opcodes.
Added support for Pruning Mode + Indexing Mode enabled.
- The Pruning/Indexing combination is not (yet) supported with the Electrum module, however the Explorer module is supported.
Stratum/Pool Module improvements (this feature is still in early-release/beta).
Fixed an issue preventing the node from running on fresh installations of MacOS 11+.
Misc. minor IBD bug fixes.
Misc. minor explorer bug fixes.
Java 17 support.

v2.1.0

FastSync support and UTXO Commitment generation.
Implemented Electrum protocol server/support.
Enabled Testnet configuration.
Now supports configurable Block size.
Double Spend Proof support.
ECIES encrypt/decrypt module.
Prototype block validation / BitBalancer support.
Memo support for explorer/indexer.
SPV mode.
Fixed validation of SLP transactions with excessive SLP amounts.
Added support for preferred node peers.
Improved Block download logic/optimizations.
RPC now supports HTTP (i.e. Bitcoin Core) request format.

v2.0.1

Minor bug fixes.
Minor performance improvements.
- Updated for ASERT reference block.

v2.0.0

Non-Indexing Module
- Reduced disk footprint to less than 300GB.
- Reduced initial sync to less than 24hrs.
- Reduced required CPU/Memory resources.
- Resolves funded feature #7.
20201115 HF support (ASERT DAA).
- Stabilizes block time intervals at 10minutes/block.
CashAddr and SLP support for Explorer
- Partly resolves funded feature #10.
Improved logging performance and space-efficiency.
Added historic checkpoints support to improve security during IBD.
Fixed an issue preventing communication to Bitcoin Unlimited nodes.
Added peer-discovery via DNS.

NOTE: v2 is fundamentally incompatible with v1. If you are upgrading from v1, you must perform a full resynchronization from a clean directory.

v1.4.0

Support for ASERT difficulty adjustment algorithm.

v1.3.2

Fix for signature pre-image with OP_CODESEPARATOR.

v1.3.1

Reverting difficulty calculation to an older, more stable version.
Dependency updates and other minor improvements.

v1.3.0

20200515 HF support (OP_REVERSE).
The Explorer now returns results via pagination for better performance.
Added CashAddr and improved SLP support to the Explorer.
Misc. Wallet Updates & Bug Fixes.
Added Bitcoin-Sign-Message support.

v1.2.0

20191115 HF support (Multisig Schnorr Signatures).
SLP Validation.
Many SPV wallet SDK improvements.
Additional RPC commands including SLP validation.
Imported reference client test vectors to confirm compatibility.
Implemented a block cache for facilitating other nodes during their initial block download and improving performance of the explorer module.
NUM2BIN now follows ABC's quirk of allowing large byte arrays for encoding.
Migrated to Hikari database connection pool.
Optional block header bootstrapping for improved initial block download.
SPV nodes now receive matching mempool transactions when a bloom filter is set.

v1.1.0

HF20190515 rules are now supported. (Schnorr Signatures)
Headers are now bootstrapped during initial sync.
SPV Bloom Filters are supported.
Improved DOS defences against malicious nodes.
Adding ASIC Mining module.
Misc. Explorer improvements.

v1.0.2

FIX: Subsequent embedded-db restarts no long fail to start. (Broken dependency)
Added BAN_NODE and UNBAN_NODE RPC calls.
RPC scripts are now copied to the out directory.

v1.0.1

Added support for remote databases.
Added RPC ADD_HOOK function.
Explorer now lists new transactions and blocks on the home page.
Updated documentation to include node message throttling configuration.
Changed user-agent to use a space instead of hyphen.
Implemented getaddr message type to facilitate node discovery.
Fixed an issue causing TransactionBloomFilter to render significant false-positives due to integer overflow.

v1.0.0 - Initial beta release.

Description

Bitcoin-Verde is a ground-up implementation of the Bitcoin (Cash) (BCH) protocol. This project is a mining-enabled, all-in-one indexing full node, blockchain explorer, and library.

Purpose

Bitcoin Core (BTC) is the primary group responsible for development on BTC's client. In the past, lack of a diversified development team and node implementation, have caused bugs to become a part of the protocol. BCH currently has roughly multiple popular full-node implementations (BCHN, BCHD, Bitcoin Unlimited, Flowee, and more). However, many of these implementations are forked versions of Bitcoin Core, which means they may share the same (undiscovered) bugs. With a diverse network of nodes, bugs in the implementation of the protocol will result in incompatible blocks, causing a temporary fork. This situation is healthy for the network in the long term, as the temporary forks will resolve over time, with the intended implementation becoming the consensus.

Disclaimer

Windows Users: Bitcoin Verde has been tested heavily on Linux and OS X. On Windows, there may be many issues. If you are kind enough to run this implementation on Windows, please create an issue describing the problem you encountered.

Getting Started

To build the node, run the following:

./scripts/make.sh

This command will run the gradle command to download dependencies and build the jar file, its configuration, and run-scripts, located within the out directory.

If you are running on OSX, you may need to run ./scripts/osx/link-openssl.sh before running the node.

To run the node, you may cd into out and execute one of the run-* scripts. Or from the project directory, run ./scripts/run-node.sh (or ./run-node.sh from the out directory).

Review the out/conf/server.conf and change the node's settings as desired.

The default configuration requires ~6GB ram at its peak, and a 300GB SSD to perform optimally. Best performance is achieved by increasing the bitcoin.database.maxMemoryByteCount and by running on an NVME drive.

For more detailed instructions, please refer to https://explorer.bitcoinverde.org/documentation/

Upgrading to v2.0.0 from v1.+

Bitcoin Verde v2 is fundamentally incompatible with v1. If you are upgrading from v1, you must perform a full resynchronization from a clean directory.

The make.sh script will completely remove the out directory, which is where blocks, indexes, and configurations are stored. Running this command to upgrade will cause your node to re-sync from scratch. To upgrade, shutdown your node via ./scripts/shutdown.sh, then remove (or make a backup of) your existing out directory. Checkout v2.0.0 and run ./scripts/make.sh. Finally, start your node via ./scripts/rpc/run-node.sh.

Running the Node as a Service/Daemon

The ./scripts/run-node.sh script will run the node in the current shell, which will exit upon logout. To run the Node as a detached process, you can start the run-node script via nohup, like so: nohup ./scripts/run-node.sh &.

Logs are located within out/logs/node.log, and are gzipped and rotated. You can monitor your node's progress via the explorer or by tailing the logs directly via: tail -F out/logs/node.log.

Alternatively, you can install the daemon scripts located in daemons directory. Currently, Bitcoin Verde comes with init.d and systemd versions of these scripts. Installing them is dependent on your OS, and is out of scope of this README, but there are plenty of guides online for installing systemd or init.d processes.

Contributions

Any contributions are welcomed and will be reviewed via pull-requests. In order to be accepted, care must be taken for all immutable classes and their mutable counterparts. Additionally, PR resolving bugs preferrably be accepted along with a test proving their existence and fix.

Contact

Feel free to contact Software Verde, LLC at any appropriate softwareverde.com email address. Generic enquiries may be directed to [email protected]

Running on macOS Catalina

If the embedded database fails to start due to being unable to find the SSL lib, consider executing the following commands and trying again: ./scripts/osx/link-openssl.sh

bitcoin-verde's People

Stargazers

Watchers

Forkers

liu-peiwen echterago dennischiume jestevez limber0117 posicks baby636 imaginaryusername nirvananimbusa 1crazymoney muneebsa ambeeeee cculianu dfbsdb bit-hawk

bitcoin-verde's Issues

Java running out of memory? & More data base error

Environment:
Ubuntu 18.04
T3 medium ec2 instance: 2vCPU, 4Gb Memory
Java 8

After successfully starting to sync, the node seems to get stuck at 4.85%

This issue persists despite restarting the node. The node gets stuck at block 28270.

After running the 'run-repair' script, the node seemingly started again. It synced fine until it seem to crash a few hours later with the following error:

[2019-05-17 02:35:04] [BlockProcessor.java:151] Stored 14 transactions in 33.68ms (415.67 tps). 0000000000007ABDEA01673BAAAE8F591E4DF72087517B5D669C02C28162E22A
[2019-05-17 02:35:04] [BlockValidator.java:295] NOTE: Trusting Block Height: 116955
[2019-05-17 02:35:04] [AddressProcessor.java:72] Processed 189 LockingScript Addresses in 20480ms.
[2019-05-17 02:35:07] [SynchronizationStatusHandler.java:30] Synchronization State: SYNCHRONIZING
[2019-05-17 02:35:14] [NodeModule.java:819] Current Memory Usage: 337433504 bytes | MAX=1019215872 TOTAL=1019215872 FREE=681782368
[2019-05-17 02:35:14] [NodeModule.java:820] Utxo Cache Hit: 426578 vs 2894 (99.32615%)
[2019-05-17 02:35:14] [NodeModule.java:821] ThreadPool Queue: 0 | Active Thread Count: 1
[2019-05-17 02:35:14] [NodeModule.java:823] Alive Connections Count: 14
[2019-05-17 02:35:15] [NodeModule.java:824] Buffered Connections Count: 11
[2019-05-17 02:35:15] [NodeModule.java:825] In-Use Connections Count: 3
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x00007f846c8f8000, 12288, 0) failed; error='Cannot allocate memory' (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 12288 bytes for committing reserved memory.
# An error report file with more information is saved as:
# /home/ubuntu/bitcoin-verde/out/hs_err_pid13033.log

Note: the node did sync successfully 66gb.
I tried the earlier repair script and restarting the node, and got another data base error:

2019-05-17 17:43:54 139628934848640 [ERROR] InnoDB: Unable to lock ./ibdata1 error: 11
2019-05-17 17:43:55 139628934848640 [ERROR] InnoDB: Unable to lock ./ibdata1 error: 11
2019-05-17 17:43:56 139628934848640 [ERROR] InnoDB: Unable to lock ./ibdata1 error: 11
2019-05-17 17:43:57 139628934848640 [ERROR] InnoDB: Unable to lock ./ibdata1 error: 11
2019-05-17 17:43:58 139628934848640 [ERROR] InnoDB: Unable to lock ./ibdata1 error: 11
2019-05-17 17:43:59 139628934848640 [ERROR] InnoDB: Unable to lock ./ibdata1 error: 11
2019-05-17 17:44:00 139628934848640 [ERROR] InnoDB: Unable to lock ./ibdata1 error: 11
2019-05-17 17:44:01 139628934848640 [ERROR] InnoDB: Unable to lock ./ibdata1 error: 11
2019-05-17 17:44:02 139628934848640 [ERROR] InnoDB: Unable to lock ./ibdata1 error: 11
2019-05-17 17:44:03 139628934848640 [ERROR] InnoDB: Unable to lock ./ibdata1 error: 11
2019-05-17 17:44:04 139628934848640 [ERROR] InnoDB: Unable to lock ./ibdata1 error: 11
2019-05-17 17:44:05 139628934848640 [ERROR] InnoDB: Unable to lock ./ibdata1 error: 11
2019-05-17 17:44:06 139628934848640 [ERROR] InnoDB: Unable to lock ./ibdata1 error: 11
2019-05-17 17:44:07 139628934848640 [ERROR] InnoDB: Unable to lock ./ibdata1 error: 11
2019-05-17 17:44:08 139628934848640 [ERROR] InnoDB: Unable to lock ./ibdata1 error: 11
2019-05-17 17:44:09 139628934848640 [ERROR] InnoDB: Unable to lock ./ibdata1 error: 11
2019-05-17 17:44:10 139628934848640 [ERROR] InnoDB: Unable to lock ./ibdata1 error: 11
2019-05-17 17:44:11 139628934848640 [ERROR] InnoDB: Unable to lock ./ibdata1 error: 11
2019-05-17 17:44:12 139628934848640 [ERROR] InnoDB: Unable to lock ./ibdata1 error: 11
2019-05-17 17:44:13 139628934848640 [ERROR] InnoDB: Unable to lock ./ibdata1 error: 11
2019-05-17 17:44:14 139628934848640 [ERROR] InnoDB: Unable to lock ./ibdata1 error: 11
2019-05-17 17:44:15 139628934848640 [ERROR] InnoDB: Unable to lock ./ibdata1 error: 11
2019-05-17 17:44:16 139628934848640 [ERROR] InnoDB: Unable to lock ./ibdata1 error: 11
2019-05-17 17:44:17 139628934848640 [ERROR] InnoDB: Unable to lock ./ibdata1 error: 11
2019-05-17 17:44:18 139628934848640 [ERROR] InnoDB: Unable to lock ./ibdata1 error: 11
2019-05-17 17:44:19 139628934848640 [ERROR] InnoDB: Unable to lock ./ibdata1 error: 11
2019-05-17 17:44:19 139628934848640 [ERROR] InnoDB: Operating system error number 11 in a file operation.
2019-05-17 17:44:19 139628934848640 [ERROR] InnoDB: Error number 11 means 'Resource temporarily unavailable'
2019-05-17 17:44:19 139628934848640 [ERROR] InnoDB: Cannot open datafile './ibdata1'
2019-05-17 17:44:19 139628934848640 [ERROR] InnoDB: Could not open or create the system tablespace. If you tried to add new data files to the system tablespace, and it failed here, you should now edit innodb_data_file_path in my.cnf ba$
2019-05-17 17:44:19 139628934848640 [ERROR] InnoDB: Plugin initialization aborted with error Cannot open a file
2019-05-17 17:44:19 139628934848640 [ERROR] Plugin 'InnoDB' init function returned error.
2019-05-17 17:44:19 139628934848640 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
2019-05-17 17:44:19 139628934848640 [ERROR] Unknown/unsupported storage engine: InnoDB
2019-05-17 17:44:19 139628934848640 [ERROR] Aborting


Installation of system tables failed!  Examine the logs in
/home/ubuntu/bitcoin-verde/out/data for more information.

The problem could be conflicting information in an external
my.cnf files. You can ignore these by doing:

    shell> /tmp/MariaDB4j/base/scripts/mysql_install_db --defaults-file=~/.my.cnf

You can also try to start the mysqld daemon with:

    shell> /tmp/MariaDB4j/base/bin/mysqld --skip-grant --general-log &

and use the command line tool /tmp/MariaDB4j/base/bin/mysql
to connect to the mysql database and look at the grant tables:

The database error seems to be a permission issue...or rather redundant mysql process:

ubuntu@ip-172-31-39-229:~/bitcoin-verde/out/logs$ ps -ef | grep mysqld
ubuntu   13116     1  6 May16 pts/2    01:10:55 /tmp/MariaDB4j/base/bin/mysqld --no-defaults --console --basedir=/tmp/MariaDB4j/base --datadir=/home/ubuntu/bitcoin-verde/out/data --port=8336 --socket=/tmp/MariaDB4j.8336.sock --innodb-flush-log-at-trx-commit=0 --innodb-flush-method=O_DIRECT --max-connections=100000 --innodb-read-io-threads=8 --innodb-write-io-threads=8 --innodb-lru-scan-depth=256 --max_allowed_packet=134217728 --query_cache_size=0 --innodb_buffer_pool_instances=4 --innodb_buffer_pool_size=2147483648 --innodb_log_file_size=34359738368 --innodb_log_buffer_size=1073741824
ubuntu   20858 11978  0 18:06 pts/2    00:00:00 grep --color=auto mysqld

secp256k1 provided by Bitcoin-ABC

Hi, Amaury Séchet at Bitcoin-ABC have started a new git repo where he keeps a clone of the secp256k1 part from the main software.

I noticed that you ship with an old version from Bitcoin-core that does not support Schnorr.

I can help you if you need pre-built binaries from Fedora releases. Since we will start shipping the Bitcoin-ABC version for Fedora soon. https://koji.fedoraproject.org/koji/taskinfo?taskID=40070498

UTXO Fast-Sync: Consider using VARINT for all of the integers in the serialized UTXO

For example, UTXO output value is now 8-bytes fixes size. A quick scan of the UTXO set shows that fully 80% of the UTXOs right now, or ~41 million UTXOs, could be encoded in a 5-byte compactsize. That's a 150MB savings right there just for this 1 field. And there are a few other fields in the same data structure that likely can benefit (such as "height and is_coinbase"), as well as likely output number.

Together this may be around 1GB of savings or in that ballpark just for the current UTXO set.

It's something to think about doing.

Broken link in the README.md

The providerd link in the README.md is broken. Please check

UTXO Fast Sync spec: Perhaps recommend a minimum sub-bucket size?

As discussed in slack -- it might be advantageous for the UTXO fast-sync spec to recommend a minimum sub-bucket size for the "non-last" sub-bucket. This is to avoid a type of shenanigan/attack from badly or maliciously implemented peers that have a series of thousands of tiny sub-buckets each with 1 UTXO in them or worse a lot of 0-sized sub-buckets.

Ideally you want all of the sub-buckets to be just under 32 MiB (the largest recommended sub-bucket size), and only the last one can be any size (including as small as 1 UTXO or ~53 bytes.. or maybe even 0-bytes sub-buckets are ok for the last one since they are harmless so long as there is only 1 of them and not a million of them in a particular containing bucket).

Create Non-Indexing Module

GitHub Issue #7

Problem Description

Currently Bitcoin Verde indexes large parts of the blockchain.

Indexed components include (but are not limited to):

all transactions
all outputs and inputs (both spent and unspent)
P2PK/P2SH addresses
SLP tokens
SLP validation
contentious/orphaned blocks.

Keeping indexes on these components greatly increases the disk footprint of the node, as well as increases the initial block download (and validation) time.
Indexing these fields with modern hardware, including an M.2 SSD, can take over a week.
These indexes also increase the CPU and RAM resources required to optimally run the node.

Despite being very useful for explorers and wallet services, these indexes are mostly unneeded for blockchain validation and provide no value for mining nodes.

Value Proposition

This solution will provide the following benefits:

reduce the time for the initial block download
- allow mining pools to start running a Bitcoin Verde node in a more reasonable amount of time
reduce minimum required resources
- reduces barriers of entry for both miners and non-indexing node operators
- allows pool operators to run redundant/backup nodes for their pool at a lower cost
reduce overall infrastructure cost to run a Bitcoin Verde node

Solution Overview

Running the non-indexing module inherently runs the node with the cacheBlocks option enabled.
With cacheBlocks, blocks are stored serialized on-disk.
Many of the transaction-, address-, and slp-related SQL tables will be removed.
A new transactions table will be created with columns that can be used to point to the location on disk where the transaction's block is stored, and an offset of where the transaction is within the block flat file.
This migrates mined transactions from the database (which is indexed and normalized, which is space inefficient and less performant) to a more compact format in a flat file.

In order to keep mempool/unconfirmed-transaction logic consistent between modules, the former transaction_* tables will be renamed and reused for unconfirmed transactions.
Having unconfirmed transactions indexed and normalized allows Bitcoin Verde to keep its infinite transaction-chaining limit, without any additional development.
As transactions are mined in a block they will be removed from the unconfirmed_transactions_* table, and will be only stored in block flat files.
Since the transaction SQL tables will not be large, indexing only unconfirmed transactions nominally increases the disk footprint of the node, while still allowing extensibility for future features and complex decisions to be made regarding next-block inclusion of unconfirmed transactions.

Additionally, total transaction size and fee amount will be added to the transaction table to improve block template generation.

Changes will affect mostly the data-layer, which is encapsulated by the TransactionDatabaseManager and related classes. Validation logic and network logic may be minimally affected.

Many of the existing tests depend on direct manipulation of the database's data for their setup.
With a different schema, these tests will be broken when run with the schema changes for this module.
The above schema changes won't cause the original test suite to break since the test suite loads the indexing schema by default.
However, with this configuration, many of the tests will not be run against the new schema, causing a fairly large gap in test coverage for critical functionality.
This proposal also includes extension to existing tests to include coverage of the non-indexing schema as well as the indexing schema.

Solution Milestones

SQL schema refactoring and data-layer migrations.

The first milestone will consist of the SQL schema and logic changes discussed in the solution overview. This milestone is considered completed once the tables are removed and the node successfully completes its initial block download on main-net.
Updating tests for new data-layer.

The second milestone consists of updating all broken tests to ensure existing regression tests pass.
Additionally, the second milestone includes expanding the existing test suite to run against both indexing and non-indexing schemas.
Month-long main-net tests ran via the block template aggregator.

The third milestone will conclude after the node is synced to main-net and its template block is deemed compatible with both Bitcoin ABC and Bitcoin Unlimited nodes. Completion of this milestone requires the node be updated for the new sigops ruleset included in the 2020-05 upgrade, and requires the template block aggregator be completed so that the template block generated by Bitcoin Verde may be automatically validated by other node implementations against current main-net nodes. After a month of creating main-net template blocks without incompatibility, the milestone will be deemed completed.

Estimated Relative Complexity

Milestone 1 - 80 / 180 (45%)
Milestone 2 - 60 / 180 (33%)
Milestone 3 - 40 / 180 (22%)

Budget

This proposal does not have a minimum starting budget.

Completing this proposal will require approximately 180 hours.
At a rate of 0.5 BCH/hr, the total requested budget for this proposal is 90 BCH.

Funding Address

Funding this proposal may be sponsored by sending Bitcoin Cash to the following address:

1716CkGxHLVj2q9b4hKxRb3KgY11xReiv8
(bitcoincash:qpqa2x8cmrd2c6h6acnf7euqgkkl4prvwu7quzc4cc)

Authorization Signature:

The signature is signed with our primary donation address, 1VerdeMuXH1ApSZsQDkuHHZrwJaAtXTVn, which can be found on bitcoinverde.org.

The signature message consists of a Bitcoin Signed Message with the following format:

Issue-Number | Issue Title | Funding Address | Estimate Hours | Budget BCH

Notes:

The pre-image includes the concatenation symbol.

Pre-image:

7|Create Non-Indexing Module|1716CkGxHLVj2q9b4hKxRb3KgY11xReiv8|180|90

Signature:
HO1xu3BDJopi/PsAK2sqql0pHD62KtnmmhqoVyzJWkuAA9N6vhla4Pz4ABkCWU+3Qw/fpY+9wsLwM1zR1XO1oDQ=

Block Template Validation Service

GitHub Issue #8

Problem Description

There is a risk that blocks mined by miners could cause a chain split or be orphaned due to being incompatible between node implementations.

From a miner's perspective, even a small risk associated with these incompatibilities can have a large impact on profitability.
This incompatibility risk increases the incentive for miners and pools to all run the same implementation, which greatly reduces node-diversity between miners.

Incompatibilities in consensus, while generally very rare, are possible due to minute differences in the implementation details between nodes.
Generally, these differences are only challenged by uncommon edge-cases within the scripting and validation logic between the nodes.
Since the block template is only lacking the proper work before it is considered valid, nearly all of the possible incompatibilities between block candidates can be detected before any work is ever performed.

This block template validation service (TVS) proposal aims to reduce the risk of the miners creating an invalid block to near-zero by validating the template block against other implementations before any work is performed on the block template.

Value Proposition

This solution will provide the following benefits:

reduce the risk of unintentionally mining a block that would cause a network split
reduce the financial risk to miners of running different node implementations
increase miner confidence in node-diversity
alert developers of would-be incompatible blocks

Solution Overview

Create a service that is connected to the latest version of multiple node implementations ("validating nodes"):

BCHD
Bitcoin ABC
Bitcoin Unlimited
Bitcoin Verde
Flowee The Hub

This service shall accept a standard block template from getblocktemplate (as specified by BIP-22, BIP-23, BIP-9, and BIP-145 (if necessary)).
Once a block template received, the service ensures each validating node has seen each transaction within the template block.
Then the service attempts to validate the template block for each implementation.
The service then responds to the requester if any node finds the template invalid.
A best effort attempt by the service will be made to determine with transaction(s) trigger the invalid state of the template block, so the requester may choose to omit them.

Additional future extensions to this proposal may include returning a modified block template that excludes only the incompatible transactions.

Notes:

The validating nodes may not be equipped to validate a template block.
This proposal will define a formal BIP to extend getblocktemplate to allow its proposal mode to allow a flag to ignore the proof of work validation for the block data.

Additionally, this solution will create a reference implementation and pull request for Bitcoin ABC that fulfills the above extension to getblocktemplate.
Providing an implementation for Bitcoin ABC (while others are omitted) is considered within scope of this issue due to its current majority marketshare.

If other implementations provide similar required functionality without directly using getblocktemplate then this issue may be later extended to create a compatibility shim(s) for those implementations.

Bitcoin Verde does not currently support proposal mode of getblocktemplate.
This issue will modify the currently equivalent functionality to match the getblocktemplate RPC API, including proposal mode.

Solution Milestones

Service Creation

a. Define the service API to validate a block template.

b. Invoke multiple RPC getblocktemplate:proposal calls to connected node(s) to validate the template block, and return the validation status.

Create BIP

a. Create a formal BIP to extend the existing functionality of getblocktemplate:proposal.

b. Create reference implementation for Bitcoin ABC.

Modify Bitcoin Verde

a. Modify Bitcoin Verde to fulfill the proposed BIP.

Estimated Relative Complexity

Milestone 1 - 60 / 160 (37.5%)
Milestone 2 - 60 / 160 (37.5%)
Milestone 3 - 40 / 160 (25%)

Budget

This proposal does not have a minimum starting budget.

Completing this proposal will require approximately 160 hours.
At a rate of 0.5 BCH/hr, the total requested budget for this proposal is 80 BCH.

Funding Address

Funding this proposal may be sponsored by sending Bitcoin Cash to the following address:

18SSESYMBCsFkDGHwcvUFCoxwFcsCHgV73
(bitcoincash:qpges4u8lm9w6mekdkrn9cpy79lst2lukywwgpkf8r)

Authorization Signature:

The signature is signed with our primary donation address, 1VerdeMuXH1ApSZsQDkuHHZrwJaAtXTVn, which can be found on bitcoinverde.org.

The signature message consists of a Bitcoin Signed Message with the following format:

Issue-Number | Issue Title | Funding Address | Estimate Hours | Budget BCH

Notes:

The pre-image includes the concatenation symbol.

Pre-image:

8|Block Template Validation Service|18SSESYMBCsFkDGHwcvUFCoxwFcsCHgV73|160|80

Signature:

G8HaPOT1tLJE0JfcUnejdBKAuP+xzS5vu0ZZY85gEjS/WAQonhUI5kK94B1vb8cN4IFOSDDPKadmHzChy7xcgsg=

Occasional header merkle proof error with Electron Cash connecting to a bitcoin verde electrum-cash server

Sometimes I get this type of error in Electron Cash (off latest master):

  5.617| |05| [electrum.bitcoinverde.org] Sent incorrect merkle branch, expected: 60b3f9f9e439fd5f6c95ae380b91a480359657afd9206a9274f66fd42845273b, proved: e345e3dfa6c72b8751a57a06c0fed989f8aa4d86fa27532a10db1d8e6ecb3b9f

After which time my Electron Cash client bans the verde node for providing a bad merkle proof for headers. This is in response to the blockchain.block.headers Electrum Cash RPC method.

I added more debug code to Electron Cash to give you a sample method invocation that returns the wrong results on Verde:

The request was basically:

{'id': 65, 'jsonrpc': '2.0', 'method': 'blockchain.block.headers', 'params': [717025, 147, 717171]}

And the extra log prints I added showed:

  5.149| |05| [electrum.bitcoinverde.org] Sent incorrect merkle branch, expected: 60b3f9f9e439fd5f6c95ae380b91a480359657afd9206a9274f66fd42845273b, proved: e345e3dfa6c72b8751a57a06c0fed989f8aa4d86fa27532a10db1d8e6ecb3b9f
|  5.149| |05| [Network] bad server response for blockchain.block.headers: Exception('Got bad merkle branch') / {'result': {'max': 2000, 'root': '60b3f9f9e439fd5f6c95ae380b91a480359657afd9206a9274f66fd42845273b', 'count': 147, 'hex': '000000204da955af6b9f26858538197ad7cfd832553783f9a5feb0010000000000000000736596fccb160ba2a5e72ed2fe89e7668036dd12a977300579f245c7e617edda6a8bac61eb4f05184a302201000000208e78f460e6c9e11c4004abb6a92ceb11b641b0def54d4d02000000000000000060c24bff925a4a03140c34bbddbf5d06d4997d5159fac22e5b42f263925b6136e594ac61055105183eb32939000000207136eb940438b38a5c05d5c69792a88b207ad7ad5866b3010000000000000000b1dc57b5d610b4859f0a0590411e8e0bee2e50337b0d838a982a3b328f263fd78599ac61095b05181e857f8a00e00020997b37265951a33204ace20d3e7595a58ae72bc2ce1bea03000000000000000097101656049a47bf80a90b42b239ea8aca902974dbfa82f3ed3f1b3ab50573f531a1ac613d5e0518aa28506b00e00020d5f9e6a47f1c61eda726507c8649ad0188b6f3f7898c50040000000000000000ba1d2a8753738d8beeecd0261082053c9969a578b6a8309a6e0dbbabdd598ec96fa9ac61ca6505180b62339d00e0ff3f4fd59fceaa356ac979e68ee807d798970d4db6715b561c010000000000000000ee8f93af021774f55be23a5f3d32417e65ac781f38074e3d72888b62be029bc4e5a9ac61276e05182738fa67000040205a5f936d62524d165f392b59bc02a91db753fd24af83f5040000000000000000b74cabfdeb776c6ce7bbd5b37d765e10d542344b87eed8f062e2fcd0eed3d0e4f8aeac617f6b05185ca21a1900e0ff27aa90e8068b686f87f978a64c0161745ab4a363ef910ff102000000000000000097dc70a9576be48e63d6e16cadff803f35a2342bfdf1e5b59f797b0ed6a7e5231eb0ac61626f051869d46ec60000002096f584c150b6be85c5561e83d6dd284887e16456aea2a5040000000000000000af4c7b6f78a1517d24cd621cf9a796337c994737ad80629bf9770e3abf00d36245b0ac61ad6d05186fdfe08800000020612ae6f37a0924ac58a264b75f953eb227be448c6d386d000000000000000000a3d1a3063c86e49360fb46fb7ad7004c3eb90dfe260c4f25ff1bcff9919c93d7d5b2ac618c6a05184d67fb0b002000207e40b1a786f43a15ecb73c53b43273d9a614cb76b0792d040000000000000000292e32d02b3552d2320103ed1c0a8fbc23599f5df6113f0912b693fa174ddfd703b5ac61da6a0518ca891b5300008020838739913c83f07d458061265af27e2c415a9729d0890400000000000000000004b211f2e1850a29facbb90d56de268011aa8f57e3c735b44e27a49fb46d6db251baac61a06a05184ae47c500000ff3fd2e063e3f355d44907d09ea089087c935a47734ee24128020000000000000000ee4a72f352bbac0eadf9da23768608c87af7a88a15b10dbafe0b91e98762a90de4bfac61da6e0518b6083f10000000206fcd1c9d6e2ef094270dc736ea5f84861482ee2af779c904000000000000000068ebaba0a69da84f11b43f96f20552d2c492a48ee6128b7afd6de64cf8202c218fc1ac6176730518009849e5006035307d882b3c41d384c9d42809fbcfdcbef200137ce8bcafa20400000000000000003286019d26a8c32a026986f832e2c865906561cdc6005e2b7c6bd0e37ceb444494c1ac617f7205180518647c0000c02064fd74a0a9842153162fe91c2acafd6efa936b9d2659bd030000000000000000f325507aa255a384d383e3a3a2d76d7b8a934a6808c014c5cc20f7558735a8cce7c2ac612d6f0518a09b4862000040206c89d1916e581a8b386bf4a025ededa624eb68c1de5730020000000000000000c8566c9f2507c7179052bde33830f40ba860d4f6e8c01b73407be79a61982cdb56c4ac61b76d051868d2d201006000206984b89b2165fa8a24c7a340173e285cf4be8281df44f5010000000000000000563aab8146708e61c040fe99c102c3c9ed3e6035c11f1cad7632fac0f2a597ae26c5ac616d6c0518d323b731000000205f73eb887083cfd7b7f91918184f3ae9aca6dda0638b7c000000000000000000aeac09dbce25f562ba2e3cc41a362a9d766e5fc433c5a04c3e6d5dc428e922d2a6c7ac613f6a0518086ab138000000209ca723e611f78c0bca0d7caae70507bbfe5440d2d84af60000000000000000002bb104be3555a97caea60d31ba0c3587fd53fb7f3b8261f62337f1e8f5e7be9f0fceac61796a0518823a513800e0ff278e3839b5b779e8ec5c27fbf012262fcb74cb10c1725435000000000000000000d7fa0fc7b1566bb8063f6ecfa0c6c198b23689836f8f002db9ddb4b80f0a38b52dcfac61427005181c915a8f0000002005c8e5a891e97b74dd7c84b7e9651b6390ad59332ec4b2030000000000000000d3e0e3f4065ff6051eb63273d5f90da70605077bd7d8132540ea57d5f3b89b953ed0ac61836e051896401e3700004020bf3df2649b85d75d4722a777ca03ecc4d831a87dfe570f020000000000000000b7c3357362cc012a3416b3c76ae88966a8e1e9c66ef4d1e7667cfb0a405ef8eb69d3ac61b16c0518630982370000002099b0a9c51aa8a34b48958de6a605f1a93a540860a08eb4020000000000000000e00f7f10dc30c759dc1ce866d5d518bf8c72efdd39f717d42dc9f8e61202e04e1ed6ac61de6d0518c78e888c000000205e8465f770c0aa72303b6a27e71f4cc4cb329206a81dee000000000000000000484fae8d1e2044ffd0e9f9fbee796c321731c565e9b8bbfc15573bf18c2bed89dbd6ac61616e05182417a97e00e0ff2f9dcf211d5eaa024c4f83b7ab153052bf9720e9cae5f118040000000000000000d68d5e4820e7bbdd924463eb7012e11f9d0a6706f576fc5b37b5be6c74a712ed61d7ac61166c05185ffcbf4e00004020ad86bcfb68a98e11e2f165d4b26467f9c11db84241ee160200000000000000004d9b68aa46ba12402b358202c20fb0b5bb4154e81abbdcaa74a246661afab3740bd9ac61816905187549bc430000802058738b1774974f0d656d3d44b6ed6d831df5fd9427a57d040000000000000000a56f9a4e3046a3dc83d0bab9ef1aa717d9d1b866a36bff4682c9fb34137581676edeac618a680518399ef9650020002043e177ab84352e88c8331990f6429d8967799ec1ab8866050000000000000000f828e6ea95678e2ea1f9203488e15e06579d7c1426b99d71b17b8d8d91b26bf234e2ac61dd6c05183d0f52130000c0201a50d1cfd1207061d22a0954afa5660ff5dcbd4a10d5110000000000000000003ad46aae7d0111d640f9f6409dd7d7eeb285e9457c05ed388223454afe8fdc18ffe6ac61e96e051897535f0f00000020396adcc444b034a2d33f75ef924d89ebe37bc83510a512010000000000000000914e7afd640ee57701516f437ca2f6f9e7055468e7a96f8866e232311147e6870ceaac616b720518bceadf9f00e0ff2f70f0c9aeac572c005a71aeb207fef49d84cb903a5c3d69010000000000000000326fc6d26d98b22b06d6c42bd94bb61ba6593bebab09b85ee29193e4bb5a2d8a5deaac616d7305182da29ec1004000209788148295be4318151a48ecaeb359b6e48e22bea10df60100000000000000000e90f528a0f42b1f3fcdf6a6e0d35ec6a9d8f376f19e8889269715218d638d3b74ecac6186700518e4d466310000ff3f0665edfaa86d97c8607e824d5e7078be0f440f794e9fd4010000000000000000736b159d1cb3868e0f5920891f07eacbaaccb86981756325137291dcbb667edcb8ecac612a700518082302670000002080675e6caa4fe1f366d9e3e8e503e64c79d075cbc484700400000000000000001027789097b7e631a54d3c5708bf56524581d67af98a1b3d67ff5f2935dfa06ce5f3ac612f6d0518b143fa76000080202f68cfc87ea42fe5aff707dfad60a3cb276bd6577465100100000000000000001fe83385455b15f9bdbfd47f68ebeb3c8a6c3c2f6cd16435f8b6724a80e59d7d09f4ac611774051830d9254e000000206bb709b14543ffb691b40c169210c5182f2890274492100200000000000000006bc1fb12762a8b9d2bec854fb96198e7239b71ecbc495e430cdd7101964a037824f9ac61f17005183c216d8900000020faeea27754d11cdf2ec798ccadf4a6618200f905eca3c10400000000000000005e40fff841c60ab17d3ef4aa358565684e0c694092d5d8d9ad6deb03277e5743aef9ac61e3740518b0dcdd5100004020e521e6fe8e3503d933983073f815ccfb93c453578eefb6030000000000000000bd4775653a2f7f8da4ce3270556b97dada166a6ad622ad127e5030e15d86831d33fdac614e720518445c75bd004094272e18fbff76dda3c1802caa00f0f5f1af8d83e1f02051f9020000000000000000ba25ce501f6ff39c38a96f42200ebe92826d11a403b8bf9aa2b845f4cb5cd6dd07feac61fe7305182678496200e0ff374ac7aae66c6a74ab6f1d142cdb121105e61bae0cb7a1c90200000000000000000fbfe2816a5d15727db1356fc19daf001970181be0dcd97572d40129814a453149feac61d57105184237d8ac00e0ff3748690501df57eac8fe4439f75a4a1ef0548cd08f3515f8030000000000000000147ef31c56277745ca6cdf5f03b0bca9f693d0dd7a58b3e564bddd3b455006f32504ad61da6e05183fecc10200e0ff3f72b2dc6552bda8d6feeedf12fa31d9c14df6b30b03fcbe040000000000000000df2dae4ecaf53d1ea4d911acbb2baf2aa20f6ef12284e928a80b470bbbc431bce404ad61e173051826ca5ff600004020d8b741dead00e4f08ef7943f16499d7c159bd1794a8937010000000000000000755c944e3061ce580aea2250e77980d4844a592477a3ee8173df3192dfc270b64805ad6196710518306d036500200020fd31bf76ee43fd9d85f887e05aa08df29581639697cb05030000000000000000e19ee880d7d22b101280f9679013fccb67582708565facad71095b2e4508b40e6c15ad61cc6e05187e316bab0000c020ccb77d1682611f1637fc53f32ea8ac05f181a5edc23c39030000000000000000a199c250b3f5fa5c1ca17af032327bee69141be3c6972689e6b48ecc7b117e90cd15ad61a3820518c4f5116f000040207f32b4aefdb17b8676412ec22490eb42018101cbde961403000000000000000067512dd52654d11b90764a5a930b680a53f5ad625810f0a81ec554cedb47c71d4c17ad61ca7f05182be396af000000207cf4769bd10c0f26eeb5eb0e3b4c3afef37e4dd2a3034e040000000000000000c7d0aafc775ef26e9e9190deb960c8117d5f4380b0451bbeaa1a42b152936c350918ad618f7e051848c506d500008020f42a104e0820ae43bd172b900efd204b0ba4810fd57dc80200000000000000006ec04505b84753a19b924b70981100a8042d98e19997c7eb2fd44876f083aaf94e19ad613e7c051850d7cf9b0000602055aeaa6e1ea0f1ed936d438b249608a81dbff5d17f7b65010000000000000000fb40f1c14549e2e8057f053898b25b48df8d5b418ac9327a6c18b2f88fdd4fa4c319ad61b57a051851d8d9b30000a0208a131db7d69eb6104baeb1523665abbe36c535a6368b2b030000000000000000a1fc1acd98fa961d4b55a14dcd6585f068962fbc119639e9901855f82f4939ddac1bad61fa7705183293c46f00000020a8338b63c3bee8822015f9551ba9ef1c21d1062760b63201000000000000000002725813f8c6ab2f535e2e005e71e950a3e74a92de3da7aa0f3cd958fc654c6d331cad615a770518a713eb990000c0209a9ff359eda83136be53dd84177f5ac5ca78630d6df78602000000000000000055f7fe846111687d802addf505f4628f3d2a835110d80c56867911b9a962d83e281ead61c1740518a8852eef00e0ff3f7425bef8c9cbd42584d5ae1895f75cbfdf829bf127fd44040000000000000000d53c35f07bee73db84f9bc95d2587029a9b6b073e2762aabb984da81f9fe11e6ef1fad6134740518fdadbe2600e000209cf93c3a406e51ee74734c16736701102be0523a90929f00000000000000000076b551ea776b6989eb41e7abe832ea6fd9af7273d4425b254886362bb3a34c5a7620ad6163730518eaa2e66a000000205417279832816b845e8ad762d900b9b11835323a8ad8140400000000000000007e083c01afce2bf11fc9508f44ca4846c42ab21f68f6d33ea8083e636206844a7027ad61ca700518d9d547230000402020a0527a25f1234c1a1a70d54b57385e54ec4e21f5512804000000000000000041f689c239cc9c9dbadaee1bf6b00e2500b9dd7496330141b9581436992f82cf232aad616d77051822f792300000e43307b8fdd55ae0f2d2916b0d374b503c31e16d7b2517d54b030000000000000000a6d158bc735b8999d20c52a7bf6bdc3260305d2f8700bd97d0072d73e24e8e5f622aad61f0770518491d8a460000c02028d85c8c6e941a9a5a9650724b91397d0a00fbb62318a403000000000000000053bbbc3fd950fe5f5ef35f90438d15c03b724437f1fcb93917a08b79335d4f9d242bad61f1740518e90936800000ff3fc850aaf7f3e06e5350aa646acb016c3e050efe3ba39cab0200000000000000004cfd72f9513dffa285f63a8f12d10f8e5d85b46fa3f3a0a5ff2195294d2072d02733ad61aa7205187946254d00a03121c8c70d35986f4dc60684e3d3765173ed326c0781c27d330400000000000000005c7649cb37412059ab69377a555f2a24c3576a9eb5eb4242dbee09ba2d88b2179e35ad61cd7a05188f48cabc00e0ff2fd29557b2f25f707e1eaf20a2bd4c1225c0e97baa4d6bad040000000000000000deb402f1d69e17797fa23da899eb69e263db45624f08173f285194ca436d92594038ad61f97a05188dcd3a1b00008020a42aaad4e5ebf77a05db5ded78ff2b79f2ae49c67167c70100000000000000002700d96e1d23bfd509013d3defd27154e82220ed8bc44172e51ad3db8f422641463cad61647b0518106719590000002042a71159fb862382a4fe2bb4530f3ec15b394875395ae102000000000000000024d47134a547ddc5fe891ad2a89d82620d3b4d487d93fb9ccf2f7799e3537ccb693cad61cd7d0518f4bef79800e0ff271764299d0292c95fcae4bdcc9c65a821fd5aba7484a340040000000000000000d14e724606147f6dcdeb0ad17756cb81a7aa5039dc357e8a62285da3f637c591a93cad61a27a051804032ed700002020bc546b7b245a2a0b5fff5755c2ba73bff1cd43b04748760500000000000000004e74841fff6d00f2192b1143a6a064850a6387ca538c3b62d67c80c401e074c7453ead619e7705180f9150d7000000209d7a5d05162c3ac6f74d96877f19984a1d5eedc599c6d10300000000000000009bf7e082cfc0d0a7fbab53d4a38b242c473b19f3195e9e3f5a4945d0eb204948d73ead618e7605185b133b18000080203fc53c1196069cb974b0a0adc48732499faf2f839321ce020000000000000000de9f0fdb0b6542f73923fede2e79158306e1be2a0adf3d0eec4eb4a732480efdcc41ad61037405188584c5bd00000020e69458676c45012b90b1ca8240844c81087937c67acefb03000000000000000032977ac72baa4aa0ec2c82cc54fcc4a84bf67ac3e7e0d466fa2f8702ddd2810c5744ad61e37405186cac5ae90000002084ca5a66036ca63ca3e4483c5519beb720f9615307c908010000000000000000261f0a6a0250e27193421a3aae521f78c0fe396c4203999d596726e0894fdf21b84cad61307505180c81ed2800e0ff3f8cb59d6a0013d2de529f52bd5e5d4608544bccf82ad79500000000000000000031591a0e3bcaacc87f86b0e9fb81b4749e82cf299ed4d488a33fd63d0b8df910d44fad61db7d0518b0fa8cd50040cf247e5b83d26fe8b03de133f4317682299f36186606edd3ae0100000000000000005dbe8bbe61acc149ed7bffd409e36615324c9e69c5d17f7b17fed872d02aff0c0750ad61f57e0518dcf71a83000000202ba6261057cddc80e6021a9fb2cdd94a7b627cbfbf32490400000000000000006dbf169b1f3076ecdf3996cd962e4903c4dbd315709cc02777b45843bcd17551f051ad61e27b0518612772b40000ff3f5ddec1bcac821927c97b0afcc1ac2b1c9f92cb96a4e608000000000000000000e9c8037ccc4a8746ec8d6866ac8774be8774eee32cd903469828ba15a5820cc8025ead61427b0518518173e80000002082c3a27c0e1ead7c67dd38411c422717b90cfd91cf59640300000000000000005ef200fe195c2a026644b3b48fb531547cb0768fb07c0be5e6110ae67410b61a195ead61508905181e2e62030000002063eeb0d949e3f1c4abbf3cab9f67e710537c505d28dd0601000000000000000096fe8be347aac7dcb8d6922bcca1ec1592ea7442c1227bb336c4a9e32d1e25aa0861ad610d860518904afb6b0000802072eba1c3d01b26400b04ed136180e72adee4270035d37e02000000000000000046916c616f974c71a4de37f1a02c7b5e8f0fb91231f00e50bb83788e8e6d162cd762ad61e78605182b74790900600020b8047d6e520a037daf9cf998f4503afe46560000ac74ae02000000000000000046d109a554b8031426cb6019c02d6942bd8c1aae8d61765ad7298977571e1895a763ad612086051832421e65000080202c20a2c10510a8508000e33bea7112041869d17483656d04000000000000000091cff0866ca95c704a5e09cf789670a558ed0a09079987aec07c71e9af5643278d68ad61e88305185ac7287f00e0ff2742902549157c03e16cb1e19c7eff2b98e1f9d14818a01d04000000000000000076b02869f1c4550b8aefcd82d1e9a1ca758c4bf0e3779a499f0fe3e11ffb4eec9f69ad619b8705184459bf780000c020cd0af7c50466fbacc9841d8bc9e2fa3e1fb692a23f575a0400000000000000004f9262e137ff7dfdec081f468696320ab26b9a8ff425740057a2070e333e67ae716aad61c4850518e09f57d900e0002057d3dda2ca60a8c77bba9951bf8a43b8009bc4d20dbcf702000000000000000072d37311cf6d139c64f63b020cdf14849b0286e7306ab0ab6fce92c534df5a40c76bad618c83051827c50c51000000206d823a52223d9f94993ae016e5bfd47df4c9a59c349e84040000000000000000ebbbcc358f07971358b04203fbe26a7e4b914e02bb09120d353bb63947bea0279b71ad611682051887530f3b00000020de18a0c976acd6624e681a6dbe3dd68aa0a2d27cff33bc040000000000000000b51f684c34b851d0d83150d6ba608f4b5b3d88e1ee04070be18b8b089834e117dc76ad612687051881f1f3aa00e0ff3f303d59ec1c3f1606cf63afb98c39db0a2c2765714aa0c804000000000000000005f0a0a3697cea80501867918d058a4bacc18ae82b8c2c97bfdbbbe8bdcf1513f87aad61618b051802271a73000080208f2847a1a69c37c6f47c933a6dbde2d6890ff242eba239000000000000000000d7f8935f2a95f987935e2fe4a25c810fbdf8a75b293d6165ab5ed5c8d7e9e4d60a7bad61f58d0518323a06d8000000208c3c46c066b5f185fb64463198cfa0d4e2447d1bb126760000000000000000005897efd4ef47e19fb01b38dbccc76d1c56aaadc0e58f2f3eff95757cf7492928fb7bad61a48a0518213747f800e0ff3f68cbc182c91c157fc97994066e0d490fd3c45a686bc01e02000000000000000003963123ed03284ec88ad6db9994a6440faa1f318dcd9040144c361f4485032fbc7dad619c8805184883cc06000040201d67aad62bc18a9154a7bccc497c96b19cc665b2028cae010000000000000000bd73193a53f30cf583d352adb49f5402150bb0008df98ad6708a6223af5e928a2c85ad61c287051890c0c225000000207d4163d6af5f3f8e10ba426aed2767975c89f4e060e55904000000000000000000698fad853fe3422cdc806d0fe4df0f0c62dce9032c0040a4e1bd4ee52deca7bf86ad61278f0518b1103c13000080208d3b4be5bb0d758244472e8256901d9495913ccde65e510400000000000000004162ff04910dc4cc2fa790cbefa0da35c3228b39d61b2fc949b0daeb8e31ac0a768aad61098e0518a72c1c6500000020c82f4f33fd7a745571a57f0f3df58e2f79502892672f8c02000000000000000008d345961aefae5d1f38bfced92693743018af5927464592a4cb81dc486e72c4cf8ead610c9005180c8cd859000040203946912d3e392651171b64b6b7dc8720d513c7d0174201030000000000000000df4943a3475b23bab4138f40c335bf2d9e2f368136e56e45401e3d9eceac610f9c90ad61f7920518244cc0d3000000208546bf53e6d3c00550c81e0c09fc34de6b0a6d0749e165030000000000000000eef80524899d828bcae4b2a80b0bf0c9a6d309dd40be175a6a04e032e68a3f093c94ad612b92051860f644ee00004020be30c7181411de9a241b628a4e278679775d2d8f08cd7b000000000000000000fa632a441d9260204480c7d58efc9339090e1d14cff18471a21becf28f32926bb496ad610c940518b848161b000000204f1f15343b06506f252a27e4dd65b1a36801d74386f11905000000000000000045280a00df65f7f85a046d725a1f403e77140124b92e59411fdae017c24903b3b696ad613d940518f90ba37600e0ff2789bb89131fc4f915696e03e456b6446e9d16f232af604c050000000000000000b4c46bcee06e5f692ed15afeb957984e00de5c11f5ffc3bb308691491be7d3db8397ad61ce900518315750be00400020634e37bd4191545cd614a2ab5cc97eb9e2b0f955c6d32d0200000000000000005a8d5a5d79c6e55186eb1d412f6622c83e015e40d16fc8ddee8bf3f56f84e567c899ad618c8e0518e107f82100006321fa24a08aa366cff4b326a6a04d94cc2ffbf25998fb1fc4030000000000000000333d795d51f702b5d03707cd72ba1f4fb2bca89f67975c18db1d016ebe81793c069aad61748e05185a5d8a0500e0ff2f2c96c207c2c12760705e32304a385671b6d0ad271fe4ac02000000000000000075c8c5d8f4af7e739058f9c1f62f175446f417a8059f12583b74b585022a503e389aad61618b0518722b0f1300000020001b2fa43355497bbec52c3b87445da54d9433694ac6f9314d11f6268f82a1a2b98202ae6162ac0518a438b9710000402095bdfb9f5bb77a85b3f9d985ba28dd5913369e0586e0040000000000000000008e3b6a2332254482962545d8d443fd60867ae4d6522ad02a74d4ed929ff11292b406ae61fca80518708f16ae000080208872729830f26cdd8c2ae6b1661f603a0202f3b07bc610010000000000000000a19854f17e670c5f279ba9b723857a8a7a8a0d64564cc08ec6827b501f63b4bd8209ae61b8ab0518491c67f700e000207964f0591f5ca99c5baf3babe21cf0e86687ef31d9942903000000000000000068d62e81d1712e015ce037a23995f356f03d248f2dc46162aaba9777e2ce78c8b114ae616bac0518d28db3fb00e0ff2f9ee4350eec6e286b48ffab8c9011cf5a82ad9b1ba7e4bb0400000000000000005190f84ee511f6eaf563f0228527d7de7e39baf40a8e9263a0a54783974960838917ae61a3b9051869b4070a00000020b6ac2bd85802e8a8c10004267fa8aa77f6813a1cb10877040000000000000000b94bb8617ec2752b1278bcc5d755d8da6942f59aae2d1e5f45531d2196e4b9e20e19ae6166ba0518c9416d8d00e0ff3f738c2d94b7707685a2ffbe91bb4c26a052a1a3e78fdeb60200000000000000002f1175f2e17982928c942077bc753af5604a03b99589e70ebdc65e0e22b3d9e27a19ae6125b9051806e7410800000020d421aa428ab5ecc1bb4ccd6c92616d8f8efab1ab440c710300000000000000006309c635828e2dd02ee651a485994213bad5a677cbd1b6a5246042ad262883fd9021ae6143b6051821e8fb340000802015d1c3b91a4c07817b9fbe998821f6f6ea195733101782000000000000000000bb345c1490b8fa67923f0f63f298744aeafcf4fce609a858f5498aa35ca44c483c22ae61e9be05186a3103bf0000802094b0c11d9354880ea89d2404d23b820ab13df432db01a0030000000000000000d0e16006604a006ea3491ee4c71da5b06eb3f32a871d1df31bb6aa15b10f40497822ae615fbc05188d2349dc0000bf2d20a73e2bff8b8fef42ff971ac66b57606ee9c48a192c65050000000000000000da3da169c36605a75fae709ab9888bdaa50b68d00a6e2c97fa1ff2da9b5936838822ae6139b9051861a2049600c06720ca471b77079bb1e35c6f9ded658bd90583337478bc35be0400000000000000006e2b1f11dba060d2f6108d1f5e9ce2d437274eed5e0b38b991626281e8e34f18ab22ae61cab50518308570cd0000c02054bca7cb56d27c6055cd8a093ba8119a8d16069f6ceec2040000000000000000eb113721a842818d7d8efca536dd5c5b1f84183699515869982d5599f42387030523ae617db205184413c6d6', 'branch': ['00000000000000000289e55135664cae954121f20d9fffbe69317c4ccd568cb0', '000000000000000004c2ee6c9f06168d9a11a83b098acd55607cd256cba7bc54', '924ff54c7b4d0e2b2bdd57211efe9e8f5ae8f3d0838850a6b50bf856cbc431c7', 'f17336e3cc6012be74edecb05c638f0e5e2cb430f77fe53214c2d7ceccf1177b', '15ee34c90abd131b89fce6c10f923129a7eac920abe280bc6b4937d6d09308ec', 'e3d511b70242097d125e308139869c7472311dd153c32ae2ccba188b43583d1e', 'a574d3ff20337972f74e45ea590acb0e75dfdde0b9efb7637f8b5663ef4f4005', '194b58f5d5f8c9a7f769732a6f7812ef78d7f7986934484547f958f1794bab65', '7ab5768a40b90c43fbdd69928d27e67eeddbb9c9c8ccb685f354550b4d2f694a', '2d340b561c654db914e2aed82669d5e49a40fcb5b1d05f77cba3a76d0dacd85b', 'b6dc54e3155cd7a22a9bf10338a2dcd39516476b94ca5c829e67eca638d811fc', '616fd6f3444e8eca69b32c66f663601601d243e00e20d4477916f6320a98e60c', 'a9aae1b4fe64be994a885726b1bd50f9e69829c00c88c40417e95b5b74ae0ab9', '0572f13477921067e53bef1acde850b9f648c2dea4dfeea94836e151817e33a0', '4d7395a037d2924e0b50d5ccd93ff31d8f7defef03addb4b8b6bd0d1f5ad2a6e', '7189a36ec434065339bc78692bc72115ceb7f225cd7ae4fa6f21a60f670a9d83', '8f9ccaff3ac11105f690159421bc89272633bf21988a78e66391e5aa8ec40675', '604a2701df87074726373ee7c1f51bf608d7d83b9c45b48f64d261df31ab1ec2', 'd4b47668b68484d88924af7ebead7ae2e6aec3a0136d3f45650739009cc0a8e4', 'fedf9ef433406d606ab36adf7479b6b79eb8c90977c7ab8ea08d5aab5ed1e6d5', '7c55b292c8f83ffd916ffbbd041e7a396c1d90bafc47db1408592ebc783ccf07']}, 'id': 65, 'jsonrpc': '2.0', 'method': 'blockchain.block.headers', 'params': [717025, 147, 717171]}

FWIW, occasionally electrs servers return the wrong merkle proofs too, and they get banned too. This shows that the header merkle proof stuff is tricky.

Broken readme.md link

Under the getting started section of readme.md, the documentation link is 404 not found.
For more detailed instructions, please refer to https://bitcoinverde.org/documentation/

Unable to access jar file

:~/bitcoin-verde$ Error: Unable to access jarfile bin/main.jar

Following steps from the instructions:

git clone https://github.com/softwareverde/bitcoin-verde
cd bitcoin-verde
./scripts/make.sh
./scripts/run-node.sh > logs/node.log &

Environment: Ubuntu 18.04
Java & Javac: 11.0.3

This is a fresh ec2 instance. I had tried this on an earlier build and it failed on the instantiating MariaDB part.

Implement Testnet Configuration

GitHub Issue #9

Problem Description

Currently Bitcoin Verde only allows for connections to mainnet.
While historically testing of edge cases has largely been does via unit tests with public test vectors, there is additional integration testing that Bitcoin Verde could benefit from by connecting to testnet, particularly in the time leading up to hard-forks where it appears testnet is more heavily used.
Connecting to testnet may also provide a way for Bitcoin Verde's unique perspective on testing Bitcoin Cash to benefit other node implementations in the event that the many difference between Bitcoin Verde and other implementations may lead to different kinds of test transactions.
One of the key reasons this isn't already implemented is that testnet has a number of differences in terms of how transactions and blocks are relayed, validated, and mined.
As such, these differences will all need to be implemented as toggle-able based on a new configuration field.

Value Proposition

This solution will provide the following benefits:

An additional mode of testing that should provide benefits over current modes.
Increased exposure to edge-case transactions that are less prohibited on testnet.
Improved coordination with other node implementations for hard-fork testing.

Solution Overview

In order to operate with the testnet, the following updates will be required:

Alternative port numbers, magic number, and DNS seeds
Different address version number and prefix
Different genesis block
Additional difficulty adjustment rules

It is also worth noting that, although transaction standardness is not enforced by Bitcoin Verde currently currently, steps should also be taken to ensure that if and when standardness checks are added to Bitcoin Verde, they will still be disabled when connecting to the testnet.

Solution Milestones

Update components with different static content.

The first milestone consists of the changes needed purely to communicate with the BCH testnet. This includes port numbers, anything affecting protocol message contents, and information about the genesis block.

Ensure full synchronization is possible.

Update transaction and block validation rules to ensure that Bitcoin Verde will in fact accept the content it is now able to request and receive.

Estimated Relative Complexity

Milestone 1 - 20 / 60 (33%)
Milestone 2 - 40 / 60 (67%)

Budget

This proposal does not have a minimum starting budget.

Completing this proposal will require approximately 60 hours.

At a rate of 0.5 BCH/hr, the total budget for this proposal is 30 BCH.

Funding Address

Funding this proposal may be sponsored by sending Bitcoin Cash to the following address:
196JBcMUrnekHYvnWXcy6mGN5xsGqiKZud
(bitcoincash:qpvvzm07gv38cyy47frutk5tj0er9vvjvvdzl3xggk)

Authorization Signature:

The signature is signed with our primary donation address, 1VerdeMuXH1ApSZsQDkuHHZrwJaAtXTVn, which can be found on bitcoinverde.org.

The signature message consists of a Bitcoin Signed Message with the following format:

Issue-Number | Issue Title | Funding Address | Estimate Hours | Budget BCH

Notes:

The pre-image includes the concatenation symbol.

Pre-image:

9|Implement Testnet Configuration|196JBcMUrnekHYvnWXcy6mGN5xsGqiKZud|60|30

Signature:

G8fitCCem2B9Cy61I2Aub7A1Ge0QvUcBFYxNik5tQJkKaPFtAgnx0G8684BBeBdVdysVC8L50WUXXcCawsQLmLU=

DAA is not compatible with reference implementation code.

Bitcoin Verde DAA implementation is not compatible with of the other implementations.
This is not the fault of Bitcoin Verde, but the fault of how badly specified DAA is and how poorly implemented by ABC.
Verde and Knuth are the only 2 node implementations that have written DAA from scratch, the rest of the nodes have only copied or translated the Bitcoin ABC code, so they don't have this issue.

Here is a reference in this regard:
https://read.cash/@Fernando/on-daa-implementation-algorithms-and-specifications-b739e631

Given the following code, Bitcoin Verde prints "1" and Bitcoin ABC prints "2", so the consensus rules differ.
I would have liked to try this on testnet to probe it in a better way, generating a similar 3-blocks pattern, but I haven't had enough HP to do it.

package app;

import java.math.BigInteger;
import java.util.Arrays;
import java.util.Comparator;

final class BlockHeader {
    public Long Height;
    public Long Timestamp;
}

final class DifficultyCalculator {
    public BlockHeader[]  _calculateNewBitcoinCashTarget(final BlockHeader[] firstBlockHeaders, final BlockHeader[] lastBlockHeaders) {
        final Comparator<BlockHeader> sortBlockHeaderByTimestampDescending = new Comparator<BlockHeader>() {
            @Override
            public int compare(final BlockHeader blockHeader0, final BlockHeader blockHeader1) {
                return blockHeader1.Timestamp.compareTo(blockHeader0.Timestamp);
            }
        };

        Arrays.sort(lastBlockHeaders, sortBlockHeaderByTimestampDescending);
        Arrays.sort(firstBlockHeaders, sortBlockHeaderByTimestampDescending);

        final BlockHeader firstBlockHeader = firstBlockHeaders[1];
        final BlockHeader lastBlockHeader = lastBlockHeaders[1];

        final BlockHeader[] res = {firstBlockHeader, lastBlockHeader};
        return res;
    }
}

public class App {
    public static void main(final String[] args) throws Exception {

        final BlockHeader[] firstBlockHeaders = new BlockHeader[3]; // The oldest BlockHeaders...

        firstBlockHeaders[0] = new BlockHeader();
        firstBlockHeaders[0].Timestamp = (long) 1558731500;
        firstBlockHeaders[0].Height = (long) 3;
        firstBlockHeaders[1] = new BlockHeader();
        firstBlockHeaders[1].Timestamp = (long) 1558731501;
        firstBlockHeaders[1].Height = (long) 2;
        firstBlockHeaders[2] = new BlockHeader();
        firstBlockHeaders[2].Timestamp = (long) 1558731501;
        firstBlockHeaders[2].Height = (long) 1;

        final BlockHeader[] lastBlockHeaders = new BlockHeader[3]; // The newest BlockHeaders...
        lastBlockHeaders[0] = new BlockHeader();
        lastBlockHeaders[0].Timestamp = (long) 1558731500;
        lastBlockHeaders[0].Height = (long) 3;
        lastBlockHeaders[1] = new BlockHeader();
        lastBlockHeaders[1].Timestamp = (long) 1558731500;
        lastBlockHeaders[1].Height = (long) 2;
        lastBlockHeaders[2] = new BlockHeader();
        lastBlockHeaders[2].Timestamp = (long) 1558730000;
        lastBlockHeaders[2].Height = (long) 1;

        DifficultyCalculator c = new DifficultyCalculator();
        final BlockHeader[] selected = c._calculateNewBitcoinCashTarget(firstBlockHeaders, lastBlockHeaders);

        System.out.println(selected[0].Height);
    }
}

Bitcoin Verde Explorer: CashAddr, SLP, and Memo Support

GitHub Issue #10

Problem Description

Currently Bitcoin Verde's Block Explorer only provides basic support for CashAddr and SLP.
The Bitcoin Verde node itself supports rich support for CashAddr and SLP, which means users without access to the node via RPC have a degraded experience when compared to readily available alternatives.

The Bitcoin Verde node and explorer currently do not have any support for Memo.

End-users and developers rely on block explorers to check the validity of their actions on the blockchain.
The support provided by the current state of the explorer is considered minimal, which deters users from using the explorer to its full potential.

Fully supporting the SLP and Memo actions taken by users would provide viable redundancy for other block explorers, and increases the choice of platform for users and those running their own block explorer.

These additional features enable other developers to easily review what Bitcoin Verde considers valid.
This is especially valuable for OP_RETURN-like applications, since their status is not inherently validated by miners.

Value Proposition

This solution will provide the following benefits:

attract users to the Bitcoin Verde explorer that prefer the use of the CashAddr format
attract users to the Bitcoin Verde explorer that use SLP for tokens
attract users to the Bitcoin Verde explorer that use the Memo protocol for messaging
allow other developers to easily review the validity of their OP_RETURN-based transactions to cross-validate implementations and consensus
keep support for the legacy address format while also adding CashAddr support

Solution Overview

Since the Bitcoin Verde node has rich support for SLP and CashAddr, most of the improvements performed will only affect the User Interface of the Bitcoin Verde Explorer.

CashAddr Support

The explorer will enable a toggle setting to display between Legacy addresses and CashAddr addresses.
This preference will be remembered by application for each user.
The user will be able to query for both Legacy and CashAddr addresses.

SLP Support

The explorer will display a tabulated view for all SLP functions (Genesis, Send, Mint, etc) per transaction.
The explorer will display an icon associated with the SLP token, similar to simpleledger.info.
The explorer will continue to display the SLP validity of SLP-like transactions.

Memo Support

Bitcoin Verde will be extended to parse Memo protocol transactions.
Memo transactions will be indexed by the node.
A routine will be implemented to back-port the indexes for nodes that are currently synced.
RPC calls will be updated to include Memo data, similar to the functionality provided for SLP.
The explorer API will be updated to include transaction Memo data.
The explorer will display tabulated Memo data, similar to bitcoin.com.

Solution Milestones

CashAddr Support

The first milestone will consist of all User Interface changes to the explorer to facilitate CashAddr.
SLP Support

The second milestone will consist of all User Interface changes to the explorer to improve existing SLP support.
Bitcoin Verde (Node) Memo Support

The third milestone will consist of all node changes required to support the Memo protocol, including RPC calls.
This milestone excludes all Memo support for the explorer.
Bitcoin Verde (Node) Explorer Support

The fourth milestone will consist of all explorer changes required to support the Memo protocol.

Estimated Relative Complexity

Milestone 1 - 8 / 80 (10%)
Milestone 2 - 16 / 80 (20%)
Milestone 3 - 40 / 80 (50%)
Milestone 4 - 16 / 80 (20%)

Budget

This proposal does not have a minimum starting budget.

Completing this proposal will require approximately 80 hours.
At a rate of 0.5 BCH/hr, the total requested budget for this proposal is 40 BCH.

Funding Address

Funding this proposal may be sponsored by sending Bitcoin Cash to the following address:

1MEM6WAbHmDc2yi44HoneBtjc5xoeT9hcy
(bitcoincash:qrw73v53ewavdd4ezr5wd8tn2pyqeyhmxgekjtjr0x)

Authorization Signature:

The signature is signed with our primary donation address which can be found on bitcoinverde.org.

The signature message consists of the double-sha256 of the following format:

Issue-Number | Issue Title | Funding Address | Estimate Hours | Budget BCH

Notes:

The pre-image includes the concatenation symbol.

Pre-image:

10|Bitcoin Verde Explorer: CashAddr, SLP, and Memo Support|1MEM6WAbHmDc2yi44HoneBtjc5xoeT9hcy|80|40

Signature:

HHaC5XDFXl7YWtabXH6PCGafq0BTZvq3Xa0yqOIpxwkYBRR4s6uhcXX/G6IrNYQvkqUa62Jhuo7X7HIXNb1IcwA=

softwareverde / bitcoin-verde Goto Github PK

bitcoin-verde's Introduction

Bitcoin Verde v2.4.0

Patch Notes

Description

Purpose

Disclaimer

Getting Started

Upgrading to v2.0.0 from v1.+

Running the Node as a Service/Daemon

Contributions

Contact

Running on macOS Catalina

bitcoin-verde's People

Stargazers

Watchers

Forkers

bitcoin-verde's Issues

Create Non-Indexing Module

GitHub Issue #7

Problem Description

Value Proposition

Solution Overview

Solution Milestones

Estimated Relative Complexity

Budget

Funding Address

Authorization Signature:

Block Template Validation Service

GitHub Issue #8

Problem Description

Value Proposition

Solution Overview

Solution Milestones

Estimated Relative Complexity

Budget

Funding Address

Authorization Signature:

Implement Testnet Configuration

GitHub Issue #9

Problem Description

Value Proposition

Solution Overview

Solution Milestones

Estimated Relative Complexity

Budget

Funding Address

Authorization Signature:

Bitcoin Verde Explorer: CashAddr, SLP, and Memo Support

GitHub Issue #10

Problem Description

Value Proposition

Solution Overview

Solution Milestones

Estimated Relative Complexity

Budget

Funding Address

Authorization Signature:

Recommend Projects

Recommend Topics

Recommend Org