impala-insert failing about impala-tpcds-kit HOT 6 CLOSED

impala-insert.sql fails with a memory limit exceeded which I am trying to increase but it is still failing. So far at 40G and failing. I also get errors: Failed to close HDFS file: ... and Error(255): Unknown error 255. Also the hint clustered is not used with my version (impalad version 2.5.0-cdh5.7.0) . any ideas of an alternative I could use to clustered?

from impala-tpcds-kit.

With such an old version (almost 3 years old) you will have to load the fact tables by limiting the number of partitions being loaded at one time in a multi-pass format.

from impala-tpcds-kit.

from impala-tpcds-kit.

I can't get impala updated on our work system which is annoying. So in impala-insert.sql I am currently trying to split the table for example, catalog_sales, by splitting the partitioning further. Is that the right way to go about it, and could you recommend the best way to do that split. Many thanks for your cooperation.

from impala-tpcds-kit.

The problem is that it takes a significant amount of memory to insert into every partition from every host, so the hints deal with that. On older versions you will have to limit the number of partitions being inserted into by adding a predicate on the text table. For example, taking 20 values of ss_sold_date_sk from store_sales at a time.

from impala-tpcds-kit.

Thanks Greg. I have found another database that I might use if I can't get this working. Thanks for the help.

from impala-tpcds-kit.