Nodechain is a service that allows you to create nodes on a blockchain and connect to them natively through APIs. The goal is to allow the user to build their own nodes natively without having to rely on external services.
License: MIT License
We want to create a Dockerfile to build a Terra node. It's mandatory to accept RPC request in the node.
We want to add support for a private network in development stage. Monero has no "regtest" native network, we will need to set this network from scratch.
Usefull link: Private network XMR
The method broadcastTransaction should returns broadcasted transaction hash for all chains, not only on ETH.
For BTC:
{"id": 1, "jsonrpc": "2.0", "result": {}}
For ETH:
{"id": 1, "jsonrpc": "2.0", "result": {"broadcasted": "0x5f568e62405b620fa7e8bfc52fa7a8fbdc733e804a9a044eb6202cf7f6d87e85"}}
We want to show broadcasted tx data in the UI
We need to configure Linting, building and Testing for the test/staging environment.
We need to implement the methods:
subscribeAddressBalanceunsubscribeAddressBalanceThrough WebSocket connection to be able to have the balance of an address in real-time.
Both methods logic and JSON RPC schemas need to be implemented.
I think nodechain.py should have an option to (start/stop) the application.
The option would be used like:
nodechain.py [arguments] start
nodechain.py [arguments] stop
nodechain.py stop
When the container is already started/stopped, the script would just throw an error.
If we invoke "stop" only with the currency, all instances ran with that currency would be stopped.
Also, the option to auto detect whether the instance is running, would be preserved. It would be only ran if invoked with (start/stop) arguments.
Imagine that we use nodechain.py in any script, and the intention of the script is, for example, restarting nodechain. The supposed script wouldn't know the initial status of nodechain in the way that it is acutally made, so it is impossible to create an script that ends in defined state.
If this issue is implemented, it would be as easy as run nodechain.py [arguments] stop, and nodechain.py [arguments] start afterwards, even if the first command prints some kind of error.
We need to configure Jenkins for the production environment.
We need to extract the new blocks that are consolidated in existing blockchains.
This needs to be a Websocket method that listens for new blocks.
Add Web socket support for ETH and BTC for new blocks
I find the organization of the repos a little messy. Sometimes I want to run NodeChain with my own RPC endpoints, and I have to modify the code manually. To improve the workflow, I'm suggesting three different changes.
1. Create a configuration file
Instead of having all RPC endpoints in the code, we could attach a "configuration.json" file that will be parsed by NC. The standard could be something like:
{
"networks": {
"bitcoin": {
"enabled": true,
"config": {
"rpc": "http://blahblah",
"electrumrpc": "http://blahblah"
}
},
"ethereum": {
"enabled": true,
"config": {
"rpc": "https://blabla"
}
}
}
NC will take all configurations from that file, and the configuration file will be set up by a configuration script. The first node ("ethereum", "bitcoin")... could be matching with the name of the package
2. Split into two repositories
Not everyone that runs the Connector wants to run the Dockers, and not everyone running the Dockers wants to run the Connector. Instead of this, we could split everything inti two repos. We could, also, update the init script, allowing the user to setup their own RPC endpoints (for those who do not want to run the Connector and just the native Nodes). It could have as default the RPC of the Connectors, to maintain a compatibility.
3. Upload NodeChain-Connector to Docker Hub
I propose, for better organization, uploading the Dockerfiles (and software) to Docker hub. That way, we will keep splitting the Connector code, the Docker-compose and init script code, and the code of the external software.
Docker-composes will have the image of NC, Bitcoin, Electrum... directly pulled from Docker Hub, which will keep the code and workflow cleaner.
There are some errors in the finalization of buildapi.py when you press Ctrl+C. We want to handle this signal to not show any exceptions.
We need to implement the methods:
subscribeToAddressBalanceunsubscribeFromAddressBalancesubscribeToNewBlocksunsubscribeFromNewBlocksThrough WebSocket connection to be able to have the balance of an address in real-time.
Both methods logic and JSON RPC schemas need to be implemented.
Documentation must be updated before closing this issue.
We need to extract the new blocks that are consolidated in existing blockchains.
This needs to be a Websocket method that listens for new blocks.
The setup and running of the application are better understood if there are explanatory GIFs in the README.md.
The method getFeePerByte always returns "1000" it's a very high fee. The method returns "1000" without taking into account the confirmations parameter, returns the same for confirmations: 1 and confirmations: 25
Improve and use utils.ensureHash (Connector/eth/utils.py) in every input to ensure hash values
cat /etc/os-release):NAME="Ubuntu"
VERSION="18.04.5 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.5 LTS"
VERSION_ID="18.04"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionicuname -sr): Linux 4.15.0-143-genericuname -m): x86_64When you make a request to BTC API, this error is appearing if the method is using Electrum:
E | daemon.CommandsServer | internal error while executing RPC
Traceback (most recent call last):
File "/usr/src/app/Electrum/electrum/daemon.py", line 209, in handle
response['result'] = await f(*params)
File "/usr/src/app/Electrum/electrum/commands.py", line 144, in func_wrapper
return await func(*args, **kwargs)
File "/usr/src/app/Electrum/electrum/commands.py", line 492, in getaddressbalance
out = await self.network.get_balance_for_scripthash(sh)
File "/usr/src/app/Electrum/electrum/network.py", line 839, in make_reliable_wrapper
raise BestEffortRequestFailed('no interface to do request on... gave up.')
electrum.network.BestEffortRequestFailed: no interface to do request on... gave up.We should know what is happening while running the startup script. It is proposed to change the version flag to -V and implement a verbose mode for logging (-v)
When starting an api, the user may be waiting for a while without knowing whether Nodechain is working or stuck.
Every "stage" environmental variable or others, should be changed to network
There's a couple of "Swapper" names into the repository. We should change this "Swapper*" to "Nodechain*" without breaking any functionality.
P.e: packages/swapper-bitcoin-core/... to packages/nodechain-bitcoin-core/...
We need to plan a strategy to follow for the development of the environments to be followed by the CI/CD.
Every development (referring of networks) should be changed to stagenet
The enpoint "getTransaction" needs to be standardized for all coins. For that, I've created the following standard, subject to changes.
"transaction":{
"fee":"xxx",
"block":"xxx",
"data":"xxx",
"transfers":[
{
"fromAddress":"xxx",
"toAddress":"xxx",
"fee":"xxx",
"amount":"xxx"
},
{
"fromAddress":"xxx",
"toAddress":"xxx",
"fee":"xxx",
"amount":"xxx"
}
]
}
All of the objets above are self-explaining.
We propose the creation of a docker-compose for the development environment for each of the currencies supported by NodeChain.
These docker-compose shall have the local blockchain instead of using a production or testing blockchain.
Some of the proposed options for the current currencies are:
Having local blockchains will allow development to not have long waiting times between transactions.
Create new method getTransactionCountForAddressess as we do with method getAddresBalance and getAddressesBalance. The method will receive an array of addresses as param and return something like that:
[
{ address: 'xxxx', transactionCount: '10' },
{ address: 'yyyy', transactionCount: '0' },
{ address: 'zzzz', transactionCount: '3' }
.....
]Bitcoin wallet needs to discover all addresses used generated from a secret phrase. To address discovery use the BIP44 strategy. This strategy needs to find 20 unused addresses in a row so the wallet consult transaction count information about tens or hundreds of addresses
We want to add every step to execute common commands into Bitcoin, Bitcoin Cash and Ethereum nodes in development mode.
For example, if we execute Bitcoin in development mode, we'll need to create wallets and mine some blocks before executing any transaction.
We want to create:
We propose the modification of the architecture in docker-compose for the staging and production environments for each of the currencies supported by NodeChain.
Current {COIN}TESTNET packages, docker-composes and code should be changed to delete the "TESTNET" part.
This change needs to work with the stages planing in #1
We should also change README to explain the new way to use the stages. Also we need to modify buildapi.py to fit the new changes
Having local blockchains will allow development to not have long waiting times between transactions.
To improve the development experience we need to add logs of the following types:
It is intended to separate the logic of the Connector from the logic of the APIs. For this we need to take the connector out of all docker-composes and make it standalone.
In addition, the Connector should have a new /refresh endpoint where it will receive some parameters from the startup script with which it will change the variables of the connector.py files of each corresponding API.
Finally, the endpoint system will change from http://any_server/rpc to http://any_server/<COIN>/rpc to be able to route requests to the different blockchains from a single instance of the connector.
We need to implement the following endpoints:
checkTxProof( )checkSpendProof( )getHeight( )getTransaction( )getBlockByHash( )getBlockByNumber( )estimateFee( )getAddressBalance( )getAddressesBalance( )Useful links: Wallet-RPC Daemon-RPC
NodeChain could support everything following HTTP API (along with JSON RPC), which is more modern and in line with standards.
To accomplish this, I suggest making a new decorator, and attaching it to every call. It could look like:
@rpcutils.httpRestMethod
@rpcutils.rpcMethod
def getAddressHistory(id, params):
(...)
return responseThat way, we could accomplish this without much effort.
In my opinion, HTTP REST convention should not be followed to the letter. I do not know a reason to do this, that will add a lot of lines of codes (At least by now.)
It would be a good idea to implement an endpoint to check if any blockchain is syncing.
A good response template could be:
coin: ETH
syncing: true
<!-- If syncing -->
startingBlock: '0x384',
currentBlock: '0x386',
highestBlock: '0x454'
It proposes to add a basic support for Monero. We will consider a basic support:
The following methods will be implemented:
If you request a balance from an address that is in the process of spending its balance getBalance method returns error 400.
You can test it from https://swapper-dev.netlify.app :
getBalance
You can reproduce it from the test on this PR:
Después de #60 el Conector estará preparado para aceptar peticiones de actualización. Estas peticiones deberán ser enviadas cada vez que se levante una nueva API.
/refresh to the Connector.To facilitate the visualization of the documentation, it is proposed to transcribe all the documentation to Github Pages.
Unify ETH/BTC websocket responses
ConnectionResetError: Cannot write to closing transport
TypeError: 'NoneType' object is not subcriptable in getBlockByNumber
address field to subscribeAddressBalance, getAddressBalance and getAddressesBalance in BTC/BCH/ETHgetBlockByNumberIn #28 some fields return a decimal. This fields are typed by number, this makes things hard when you need to parse this into a JS code.
We propose to change this number type to string.
Create new method getAddressesUnspent as we do with method getAddresBalance and getAddressesBalance. The method will receive an array of addresses as param and return something like that:
[
{
address: 'xxxx',
utxos: [
{
status: UtxoStatus,
txHash: string,
value: string,
vout: string
},
{
status: UtxoStatus,
txHash: string,
value: string,
vout: string
}
]
},
{
address: 'xxxx',
utxos: [
{
status: UtxoStatus,
txHash: string,
value: string,
vout: string
},
{
status: UtxoStatus,
txHash: string,
value: string,
vout: string
}
]
}
...
]Bitcoin wallet needs to get all UTXOs from a secret phrase. We need to make tens or hundreds of requests to the getAddressUnspent endpoint so it is best to group them into a single request.
After #13 test getAddressUnspent is failing:
========================================================================================== FAILURES ==========================================================================================
___________________________________________________________________________________ testGetAddressUnspent ____________________________________________________________________________________
def testGetAddressUnspent():
if "getAddressUnspent" not in RPCMethods:
logger.printError("getAddressUnspent not loaded in RPCMethods")
assert False
expected = makeElectrumRequest(GET_ADDRESS_UNSPENT_METHOD, [address1])
got = RPCMethods["getAddressUnspent"](0, {
ADDRESS: address1
})
txs = []
for tx in expected:
txs.append(
{
TX_HASH: tx[TX_HASH_SNAKE_CASE],
VOUT: str(tx[TX_POS_SNAKE_CASE]),
STATUS: {
CONFIRMED: tx[HEIGHT] != 0,
BLOCK_HEIGHT: str(tx[HEIGHT])
},
VALUE: str(tx[VALUE])
}
)
> assert json.dumps(got, sort_keys=True) == json.dumps(txs, sort_keys=True)
E assert '[{"status": ..."vout": "0"}]' == '[{"status": ..."vout": "0"}]'
E - [{"status": {"blockHeight": "4", "confirmed": true}, "txHash": "35875b91efecac90129100f70a03a782cff22f8701b2ca626d76c0bd4b81e3bb", "value": "5000000000", "vout": "0"}, {"status": {"blockHeight": "130", "confirmed": true}, "txHash": "2fc9d214f2c2c9d56fcc2c61512f95c2d998b9fecbcc9918ff46469666dc3ff0", "value": "5000000000", "vout": "0"}, {"status": {"blockHeight": "12", "confirmed": true}, "txHash": "90b891f1552dda17737d4f00446a94dc6ebe27bbde7f89801ddb28a2223c27fe", "value": "5000000000", "vout": "0"}, {"status": {"blockHeight": "93", "confirmed": true}, "txHash": "2a055...
E
E ...Full output truncated (2 lines hidden), use '-vv' to show
tests/btc/test_btc.py:410: AssertionError
====================================================================================== warnings summary ======================================================================================
../../../local/lib/python3.7/abc.py:126
/usr/local/lib/python3.7/abc.py:126: DeprecationWarning: Inheritance class App from web.Application is discouraged
cls = super().__new__(mcls, name, bases, namespace, **kwargs)
-- Docs: https://docs.pytest.org/en/stable/warnings.html
---------- coverage: platform linux, python 3.7.12-final-0 -----------
Name Stmts Miss Cover
------------------------------------------------
btc/__init__.py 1 0 100%
btc/apirpc.py 239 50 79%
btc/apiws.py 51 8 84%
btc/connector.py 15 0 100%
btc/constants.py 93 0 100%
btc/utils.py 54 16 70%
btc/websockets.py 81 27 67%
logger/__init__.py 0 0 100%
logger/logger.py 8 1 88%
rpcutils/__init__.py 1 0 100%
rpcutils/constants.py 19 0 100%
rpcutils/errorhandler.py 24 9 62%
rpcutils/rpcconnector.py 23 8 65%
rpcutils/rpcutils.py 56 33 41%
wsutils/__init__.py 1 0 100%
wsutils/broker.py 72 25 65%
wsutils/clientwebsocket.py 24 24 0%
wsutils/constants.py 5 0 100%
wsutils/publishers.py 9 1 89%
wsutils/singleton.py 6 0 100%
wsutils/subscribers.py 54 16 70%
wsutils/topics.py 7 0 100%
wsutils/wsutils.py 15 0 100%
------------------------------------------------
TOTAL 858 218 75%
================================================================================== short test summary info ===================================================================================
FAILED tests/btc/test_btc.py::testGetAddressUnspent - assert '[{"status": ..."vout": "0"}]' == '[{"status": ..."vout": "0"}]'When looking for information about small details of the implementation of the code as well as the use of the application, it is proposed to collect all this information in a Wiki in the NodeChain repository.
We should add some basic commands for every API in development stage.
We want to update every package to the last stable version.
To improve the structure of the Docker-compose it is proposed to separate the services in different networks to communicate only with necessary services.
Add header: 'Access-Control-Allow-Origin': '*' in API responses for supporting CORS in browsers.
As a web client, I want to request NodeChain API from my own domain. Now I'm receiving this error on API requests:
To improve the execution of NodeChain, it is proposed that the startup script accepts parameters by console
buildapi.py --coin="eth" --network="test" --port=60001 --ssl
We should fix this linting problems
To standardize all responses that return a decimal number it is proposed to return all such responses in JSON number type.
Due to the poor implementation of the standard number type in languages like Javascript it may cause some problems in the future to return data as Number instead of String.
It is proposed to change ElectrumX to ElectRS due to its higher performance.
This task should create its corresponding Dockerfile.
Instead of having all RPC endpoints in the code, we could attach a "configuration.json" file that will be parsed by NC. The standard could be something like:
{
"name": "bitcoin",
"token": "btc",
"networks": {
"regtest": {
"dockerComposePath": "./docker-compose/regtest/btc.yml",
"configurable": [
"bitcoincoreRpc",
"electrumRpc",
"bitcoincorePort",
"electrumPort"
]
...
},
...
},NC will take all configurations from that file, and the configuration file will be set up by a configuration script. The startup script will pass this information to the connector by making a request
I want to run NodeChain with my own RPC endpoints, and I have to modify the code manually. To improve the workflow, I'm suggesting to have a file
We need to implement tests for the application that cover at least 80% of its functionalities.
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.