[FEAT] Support Array of Struct Type about parquetviewer HOT 8 CLOSED

@mukunku
I could reproduce the error. It happens when every array of the column is empty.
I created an example file (same columns as the previous one but this one has empty arrays):

emptyArrayError.zip

from parquetviewer.

Thanks. I will take a look 👍🏼

from parquetviewer.

Looks good! File is loading and showing the data correctly.
Thanks for the fast fix.

from parquetviewer.

Implemented and fixed. Thx

from parquetviewer.

Hey @ChristianDu, as you saw I've added support for struct arrays in v3.0.0 . Thanks for opening the issue and sharing a sample file.

Going to close out this issue but feel free to re-open if the issue persists.

from parquetviewer.

Hi @mukunku ,

thank you very much for implementing the feature that fast.
It works with a few files but i get an error for one column.

createStructField(COLUMN_NAME, createArrayType(createStructType(Arrays.asList(
createStructField(SUB_COL_1, TimestampType, true),
createStructField(SUB_COL_2, DoubleType, true),
createStructField(SUB_COL_3, DoubleType, true) )), false), true);

I will try to provide you with an example file because i can't upload the exact file.

Error:

Specified cast is not valid.

Something went wrong (CTRL+C to copy):

System.InvalidCastException: Specified cast is not valid.

at ParquetViewer.Engine.ParquetEngine.ReadListField(DataTableLite dataTable, ParquetRowGroupReader groupReader, Int32 rowBeginIndex, ParquetSchemaElement itemField, Int32 fieldIndex, Int64 skipRecords, Int64 readRecords, Boolean isFirstColumn, CancellationToken cancellationToken, IProgress`1 progress)

at ParquetViewer.Engine.ParquetEngine.ProcessRowGroup(DataTableLite dataTable, ParquetRowGroupReader groupReader, Int64 skipRecords, Int64 readRecords, CancellationToken cancellationToken, IProgress`1 progress)

at ParquetViewer.Engine.ParquetEngine.PopulateDataTable(DataTableLite dataTable, ParquetReader parquetReader, Int64 offset, Int64 recordCount, CancellationToken cancellationToken, IProgress`1 progress)

at ParquetViewer.Engine.ParquetEngine.ReadRowsAsync(List1 selectedFields, Int32 offset, Int32 recordCount, CancellationToken cancellationToken, IProgress1 progress)

at ParquetViewer.MainForm.<>c__DisplayClass33_0.<b__1>d.MoveNext()

--- End of stack trace from previous location ---

at ParquetViewer.MainForm.LoadFileToGridview()

at System.Threading.Tasks.Task.<>c.b__128_0(Object state)

at InvokeStub_SendOrPostCallback.Invoke(Object, Span`1)

at System.Reflection.MethodBaseInvoker.InvokeWithOneArg(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)

OK

from parquetviewer.

@ChristianDu Can you try this alpha version with your file?

ParquetViewer_#108.zip

Not sure if you use regular exe or self-contained but I can't upload self-contained in a comment so I shared the regular one.

If you're not comfortable testing the exe that's okay too! I can add it to the release as usual.

from parquetviewer.

Thanks for confirming! https://github.com/mukunku/ParquetViewer/releases/tag/v3.0.0.1 has the fix now 💪🏼

from parquetviewer.