Trino Query getting Hung after reading 400~600M rows (10~12GB data) of Parquet from object storage about trino HOT 3 CLOSED

@pavan-upadhyay I suggest increasing the query.max-memory-per-node config to 15Gb or more for experimental. Because your table is unpartitioned, so maybe it needs to load onto one node instead of sharing it across 5 nodes.

However, for long-term I still recommend that this table should be rewritten with an optimal partition.

Edit: Don't forget to specify SELECT necessary, columns instead of SELECT * which is also a simple and effective solution.

from trino.

Hi @tlm365 , Thanks for the response!! i did try with -Xmx28G , query.max-memory-per-node=18G config but still the same issue. My query does have column names in SELECT instead of SELECT * . The query which I am testing in Trino, is working fine in Impala with same cluster configurations.

from trino.

Hi @tlm365 .. this time i set enough memory including JVM, However , it still hangs after 600M rows and our Reserved Memory goes down to 100 MB from 19 GB.

from trino.