the stage of execution too slow about bsc-erigon HOT 12 CLOSED

from bsc-erigon.

Stuck at 26605878? Can you provide more logs

from bsc-erigon.

For bsc testnet, suggest specify bootnode in the command, such as:
nohup ./build/bin/erigon --datadir ./data --chain=chapel --port=30303 --http.port=8545 --authrpc.port=8551 --torrent.port=42069 --private.api.addr=127.0.0.1:9090 --http --ws --http.api=web3,net,eth,debug,trace,txpool --http.addr=0.0.0.0 --torrent.download.rate=256mb --bootnodes=enode://0637d1e62026e0c8685b1db0ca1c767c78c95c3fab64abc468d1a64b12ca4b530b46b8f80c915aec96f74f7ffc5999e8ad6d1484476f420f0c10e3d42361914b@52.199.214.252:30311","enode://330d768f6de90e7825f0ea6fe59611ce9d50712e73547306846a9304663f9912bf1611037f7f90f21606242ded7fb476c7285cb7cd792836b8c0c5ef0365855c@18.181.52.189:30311","enode://df1e8eb59e42cad3c4551b2a53e31a7e55a2fdde1287babd1e94b0836550b489ba16c40932e4dacb16cba346bd442c432265a299c4aca63ee7bb0f832b9f45eb@52.51.80.128:30311","enode://0bd566a7fd136ecd19414a601bfdc530d5de161e3014033951dd603e72b1a8959eb5b70b06c87a5a75cbf45e4055c387d2a842bd6b1bd8b5041b3a61bab615cf@34.242.33.165:30311","enode://ecd664250ca19b1074dcfbfb48576a487cc18d052064222a363adacd2650f8e08fb3db9de7a7aecb48afa410eaeb3285e92e516ead01fb62598553aed91ee15e@3.209.122.123:30311","enode://665cf77ca26a8421cfe61a52ac312958308d4912e78ce8e0f61d6902e4494d4cc38f9b0dd1b23a427a7a5734e27e5d9729231426b06bb9c73b56a142f83f6b68@52.72.123.113:30311 &

from bsc-erigon.

i use bsc mainnet not chapel

i need 1h 44mins when the execution stage, is too slow, forever catch up the latest block

from bsc-erigon.

Because of execution time, execution stage is the most time-consuming stage within all sync stages of Erigon, suggest download mdbx snapshot and restart, it'll be faster. Also, in the future, it'll be possible to download state from bit-torrrent instead of execute locally.

from bsc-erigon.

Because of execution time, execution stage is the most time-consuming stage within all sync stages of Erigon, suggest download mdbx snapshot and restart, it'll be faster. Also, in the future, it'll be possible to download state from bit-torrrent instead of execute locally.

you mean delete chaindata mdbx.dat keep snapshots file?and restart erigon?

from bsc-erigon.

Because of execution time, execution stage is the most time-consuming stage within all sync stages of Erigon, suggest download mdbx snapshot and restart, it'll be faster. Also, in the future, it'll be possible to download state from bit-torrrent instead of execute locally.

you mean delete chaindata mdbx.dat keep snapshots file?and restart erigon?

Download the mdbx.dat with full archive data and replace the current chaindata/mdbx.dat, then restart erigon.

from bsc-erigon.

Because of execution time, execution stage is the most time-consuming stage within all sync stages of Erigon, suggest download mdbx snapshot and restart, it'll be faster. Also, in the future, it'll be possible to download state from bit-torrrent instead of execute locally.

you mean delete chaindata mdbx.dat keep snapshots file?and restart erigon?

Download the mdbx.dat with full archive data and replace the current chaindata/mdbx.dat, then restart erigon.

I use the mdbx.dat with full archive data, can't still catch up the latest block

from bsc-erigon.

You might be interested in trying this out: https://github.com/sentioxyz/erigon

This is a custom fork from sentio.xyz, the motivation came from the heavy utilization of cloud disks in our production.
We are currently using this for eth-mainnet, bsc-mainnet and polygon-mainnet.

Cloud disks provide same throughout but much higher latency comparing to a local NVMe disk. The current design of erigon is not really friendly to cloud disks, as most sync stages are only leveraging sequential I/O (which is very slow: https://cloud.google.com/compute/docs/disks/optimizing-pd-performance#io-queue-depth).

So we decided to introduce parallel execution within selected stages, to fully utilize I/O queue of cloud disks.
These are the main improvements:

• HashState Stage: Parallel data extraction. This improves performance when page cache misses.

• Intermediate Hash Stage: Parallel trie hash calculation, by calculating the hash of 16 subtrees then merge. This could improve speed of that stage significantly.

• LogIndex/CallTraceIndex/AccountHistory/StorageHistory: These stages tend to be very slow when updating bitmaps. Bitmaps need to be read from mdbx and rewritten, all sequential. We've changed that stage to parallel extract data into collector, then designed another database (bitmapdb2) to offload bitmaps from mdbx. (This requires --bitmapdb2 flag. Once turned on, you cannot use a separated rpcdaemon, as bitmapdb2 cannot be opened by multiple processes. Also you cannot enable otterscan APIs, as the new database do not yet support these.)

Issues/Criticism/PRs are welcome.

from bsc-erigon.

@ty-sentio-xyz Does it support BSC hard fork?

from bsc-erigon.

You might be interested in trying this out: https://github.com/sentioxyz/erigon

This is a custom fork from sentio.xyz, the motivation came from the heavy utilization of cloud disks in our production. We are currently using this for eth-mainnet, bsc-mainnet and polygon-mainnet.

Cloud disks provide same throughout but much higher latency comparing to a local NVMe disk. The current design of erigon is not really friendly to cloud disks, as most sync stages are only leveraging sequential I/O (which is very slow: https://cloud.google.com/compute/docs/disks/optimizing-pd-performance#io-queue-depth).

So we decided to introduce parallel execution within selected stages, to fully utilize I/O queue of cloud disks. These are the main improvements:

• HashState Stage: Parallel data extraction. This improves performance when page cache misses.
• Intermediate Hash Stage: Parallel trie hash calculation, by calculating the hash of 16 subtrees then merge. This could improve speed of that stage significantly.
• LogIndex/CallTraceIndex/AccountHistory/StorageHistory: These stages tend to be very slow when updating bitmaps. Bitmaps need to be read from mdbx and rewritten, all sequential. We've changed that stage to parallel extract data into collector, then designed another database (bitmapdb2) to offload bitmaps from mdbx. (This requires --bitmapdb2 flag. Once turned on, you cannot use a separated rpcdaemon, as bitmapdb2 cannot be opened by multiple processes. Also you cannot enable otterscan APIs, as the new database do not yet support these.)

Issues/Criticism/PRs are welcome.

polygon mainnet and ethereum mainnet fast than bsc mainnet at execution stage

from bsc-erigon.

same with #66 , will close this one and track performance issue there.

from bsc-erigon.