Date based hive partition is more useful than Block No. based in analyzing. about ethereum-etl HOT 3 CLOSED

Totally agree with you. Partitioning by date is much more convenient. I wrote Airflow dag scripts where I used partitioning by date https://github.com/medvedev1088/ethereum-export-pipeline/blob/master/ethereumetl/airflow_dags/export_dag.py

BLOCK_RANGE=$($PYTHON3 get_block_range_for_date.py -d $EXECUTION_DATE -p $WEB3_PROVIDER_URI) && BLOCK_RANGE_ARRAY=(${BLOCK_RANGE//,/ }) && START_BLOCK=${BLOCK_RANGE_ARRAY[0]} && END_BLOCK=${BLOCK_RANGE_ARRAY[1]}

$PYTHON3 export_blocks_and_transactions.py -s $START_BLOCK -e $END_BLOCK -p $WEB3_PROVIDER_URI --blocks-output blocks.csv --transactions-output transactions.csv

I'd appreciate if you could add export_all_partition_by_date.sh .

from ethereum-etl.

Could I upload AWS schemas for partition by date?

from ethereum-etl.

Sure, go ahead, it will be useful to other people also.

from ethereum-etl.