Setting batch-size value above 100 for export_receipts_and_logs crush the program about ethereum-etl HOT 3 CLOSED

Just checked and this issue is happening also with export_blocks_and_transactions, probably happen with other commands as well

from ethereum-etl.

Btw in regards to the performance, I also played a bit with --max-workers (20, 12 & 8 workers) and the diff was negligible.

In general the CPU's r mostly waiting/idling, I still think that it's because I connect to Erigon using HTTP, will try to check tomorrow with IPC (need a separate RPC daemon for that)

from ethereum-etl.

So I found out that the problem was on Erigon side as I didn't set a flag of --rpc.batch.limit which default to 100.
I guess that ethereum etl error can be improved to return Erigon code instead of just crushing.

In regards to the performance, it did improve the performance of export_blocks_and_transactions considerably but it doesn't seems to change anything for export_receipts_and_logs which is my main issue (Seems like I'm reaching iops limit on the SSD).

from ethereum-etl.