[RFC] Refactor timestamp codecs about parquet4s HOT 2 OPEN

How naive of me: as always, time-related machinery is like a rabbit hole 😅

Ambiguity

Given

• Date Time: 2019-01-01T00:00
• Time Zone: Africa/Nairobi

Current LocalDateTime-based encoder "output"

• Julian day: 2458485
• Nanos: -10800000000000

Suggested Instant-based encoder "output"

• Julian day: 2458484
• Nanos: 75600000000000

Both these outputs denote the same point in time. Instant-based decoder correctly restores value produced by LocalDateTime-based encoder and vice versa. But this is still an observable behavior change. I suppose this is undesirable (or not?). Anyway, I have no idea which of the outputs is correct/preferable because of the next issue.

INT96 format itself

The INT96 format is deprecated (see here) and even lacks any documentation (see discussion here).

With all that said, wouldn't it be better to soft-deprecate default timestamp codecs in parquet4s and encourage users to choose INT64-based ones? Also, we could switch to an Instant internally and use the LocalDateTime-based implementation only as a fallback for the INT96 format.

from parquet4s.

I suppose this is undesirable (or not?)

I guess there must be a small bug that's causing it. Definitely, it is undesirable because produces different data.

The INT96 format is deprecated (see here) and even lacks any documentation (see discussion apache/parquet-format#49).

Yes, it is deprecated, but yet (!) it is still a default format in such priminent tools as Spark, Impala and many others.

from parquet4s.