Decodes, packs, encodes, proves, stores and uploads block-replicas (primarily "block-specimens") produced by EVM or non-EVM byte code based blockchains.
Home Page: https://www.covalenthq.com/network/
License: MIT License
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
BSP agent crashes (after some retries) when there is any issue in producing BS proofs
From the last conversation a while back, we didn't have any monitoring, hence crashing the bsp-agent helped devs to be aware of some issue and help them debugging
But now that we have the metrics for successful block specimens, it would be nice to properly catch the errors, and avoid crashing the application
Specially recently we have been facing alot of crashes due to chain congestion
Some error logs recently:
bsp-agent-chorusone | 2024-04-14T12:47:17.858870813Z time="2024-04-14T12:47:17Z" level=info msg="Process pending streams at: 2024-04-14T12:47:17Z" function=consumePendingEvents line=120
bsp-agent-chorusone | 2024-04-14T12:47:19.621061918Z time="2024-04-14T12:47:19Z" level=error msg="error sending tx to deployed contract: submit transaction to pool failed: Ok(ImmediatelyDropped)" function=executeWithRetry line=153
bsp-agent-chorusone | 2024-04-14T12:47:19.621102596Z time="2024-04-14T12:47:19Z" level=error msg="failed to avro encode, prove and upload block-result segment with err: failed to prove & upload block-replica segment event: 1-19653865-replica" function=processStream line=155
bsp-agent-chorusone | 2024-04-14T12:47:19.623321289Z panic: failed to prove & upload block-replica segment event: 1-19653865-replica
bsp-agent-chorusone | 2024-04-14T12:47:19.623383788Z
bsp-agent-chorusone | 2024-04-14T12:47:19.623388126Z goroutine 7320 [running]:
bsp-agent-chorusone | 2024-04-14T12:47:19.623391242Z github.com/covalenthq/bsp-agent/internal/node.(*ethAgentNode).processStream(0xc00044b520, {{0xc000028510?, 0xc0003c0fd0?}, 0xc00012e090?}, 0xc00049b700)
bsp-agent-chorusone | 2024-04-14T12:47:19.623394519Z /bsp-agent/internal/node/eth_node.go:156 +0x95a
bsp-agent-chorusone | 2024-04-14T12:47:19.623397534Z created by github.com/covalenthq/bsp-agent/internal/node.(*ethAgentNode).consumeEvents
bsp-agent-chorusone | 2024-04-14T12:47:19.623404828Z /bsp-agent/internal/node/eth_node.go:69 +0x38a
bsp-agent-chorusone exited with code 0
bsp-agent-chorusone | 2024-04-14T14:15:26.879654466Z time="2024-04-14T14:15:26Z" level=info msg="tip cap: 2000000000 fee cap: 252000000000" function=setTransactionFeeParams line=104
bsp-agent-chorusone | 2024-04-14T14:15:37.065873908Z time="2024-04-14T14:15:37Z" level=error msg="error sending tx to deployed contract: replacement transaction underpriced" function=executeWithRetry line=153
bsp-agent-chorusone | 2024-04-14T14:15:37.065928362Z time="2024-04-14T14:15:37Z" level=error msg="failed to avro encode, prove and upload block-result segment with err: failed to prove & upload block-replica segment event: 1-19654040-replica" function=processStream line=155
bsp-agent-chorusone | 2024-04-14T14:15:37.068130283Z panic: failed to prove & upload block-replica segment event: 1-19654040-replica
bsp-agent-chorusone | 2024-04-14T14:15:37.068172384Z
bsp-agent-chorusone | 2024-04-14T14:15:37.068179037Z goroutine 2775 [running]:
bsp-agent-chorusone | 2024-04-14T14:15:37.068184176Z github.com/covalenthq/bsp-agent/internal/node.(*ethAgentNode).processStream(0xc00049e2e0, {{0xc0033f57a0?, 0x2e72fd?}, 0xc000c102a0?}, 0xc000127460)
bsp-agent-chorusone | 2024-04-14T14:15:37.068190168Z /bsp-agent/internal/node/eth_node.go:156 +0x95a
bsp-agent-chorusone | 2024-04-14T14:15:37.068195037Z created by github.com/covalenthq/bsp-agent/internal/node.(*ethAgentNode).consumePendingEvents
bsp-agent-chorusone | 2024-04-14T14:15:37.068199977Z /bsp-agent/internal/node/eth_node.go:116 +0x625
bsp-agent-chorusone exited with code 0
From Discord user soph#5873:
docker-compose runfails am not able to pull everything:
pops@covalent-testnet:/home/pops/bsp-geth$ docker-compose -f "docker-compose-testnet.yml" up
Creating network "bsp-geth_cqt-net" with the default driver
Pulling redis (redis:alpine)...
alpine: Pulling from library/redis
59bf1c3509f3: Pull complete
719adce26c52: Pull complete
b8f35e378c31: Pull complete
d034517f789c: Pull complete
3772d4d76753: Pull complete
211a7f52febb: Pull complete
Digest: sha256:4bed291aa5efb9f0d77b76ff7d4ab71eee410962965d052552db1fb80576431d
Status: Downloaded newer image for redis:alpine
Pulling redis-commander (rediscommander/redis-commander:latest)...
latest: Pulling from rediscommander/redis-commander
339de151aab4: Pull complete
f732fa32fc61: Pull complete
ef60bb667d5a: Pull complete
4f50fa5032a4: Pull complete
Digest: sha256:19cd0c49f418779fa2822a0496c5e6516d0c792effc39ed20089e6268477e40a
Status: Downloaded newer image for rediscommander/redis-commander:latest
Pulling geth (ghcr.io/covalenthq/bsp-geth:latest)...
ERROR: Head "https://ghcr.io/v2/covalenthq/bsp-geth/manifests/latest": unauthorized
Elrond has provided us observers that serve data from their 4 shards. They provide us a docker-compose.yml which start an observer for each shard and provide data on a websocket with a specific AVRO schema.
We want to add the possibility to retrieve data from websocket in the agent so that it can be used for an other blockchain and add more features to the agent.
You can communicate with the observers that running on this IP : 34.66.210.112.
Wesockets addresses :
Observer 0 : http://34.66.210.112:20000
Observer 1 : http://34.66.210.112:20001
Observer 2 : http://34.66.210.112:20002
Observer Metachain : http://34.66.210.112:20003
NODE_TAG in .env as following :NODE_TAG=elrondnetwork/elrond-go-node-devnet:D1.2.30.0
./start_stack_with_covalent.shElrond provided us a little client :
covalent_websocket_client_example.zip
This client interact with one observer only so you have to run 4 if you want to interact with all of them.
The websocket as two routes /block in the one that you receive blocks from. The blocks are formatted following this AVRO schema : https://github.com/ElrondNetwork/covalent-indexer-go/blob/main/schema/block.elrond.avsc
Once you have received a block you have to acknowledge in the second route /acknowledge. And then you will receive a new block on /block and so on.
The goal of the implementation is to be able to read from a pool of websocket (in this case 4) and match the protocol to pull blocks from the websockets to our message queue without changing the format to not loose data proper to Elrond.
Running rudder for binary specimen with the current cid ipfs://QmZ14k5FhCC4T8kgcaXrKa3ar24QsaBST7F7Ybv3k7rA4L
fails with -
%Porcelain.Process{err: nil, out: :string, pid: #PID<0.488.0>}
ERROR(10): error unmarshalling replica file: json: cannot unmarshal number 15891574.0 into Go struct field BlockhashRead.State.BlockhashRead.BlockNumber of type uint64Codec upgrade is required for fields that are encoding "double" to "long"
Two instances of mismatched codec were found
"name":"BlockhashRead_record",
"type":"record",
"fields":[
{
"name":"BlockNumber",
"type":"double"
},
...
..
{
"name":"blockNumber",
"type":"int"
},
having two different encodings
we should aim for values on blocknumbers up to - 9223372036854775807 using long and not 2147483647 using int and definitely not with decimal place usingdouble since blocknumbers don't have decimals
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.