How do I use an existing MessageType? about parquet4s HOT 4 CLOSED

Hi!

I do not know your use case but I am not sure if using own MessageType is a good idea. You still have to provide a case class to write data. Library will resolve the schema on its own. If your custom MessageType differs in field types from what Parquet4S generates then it will just break. Unless you see that there is something wrong with generated schema I recommend using what is provided by the lib. Otherwise please create an issue - let's fix it!
If your case is that you need something really specific to your use case you may try implementing your own instance of type class of ParquetSchemaResolver that will take field definitions out from your MessageType. It should be very simple. Just place it in a proper implicit scope. I hope it will work then!

from parquet4s.

I was specifically looking for a way I could have a parquet schema from a file and be able to write to that without having to rely on avro or thrift. Since you can parse the message schema using parquet's library I already had a message type to use, hopefully that makes sense. I'll give the parquet schema resolver impl a try and see if that does it.

from parquet4s.

from parquet4s.

I see, okay I appreciate your very fast response, I'm going to have to write my own implementation of this it would seem, but this library is awesome and I plan to definitely use it in the future when I have concrete case classes to use. thanks!

from parquet4s.