Comments (4)
flipp5b
commented on June 21, 2024
It looks like the issue only manifests for some time zones.
TZ=Europe/London sbt "core/IntegrationTest/testOnly -z com.github.mjakubowski84.parquet4s.FilteringSpec"- Success.TZ=Europe/Berlin sbt "core/IntegrationTest/testOnly -z com.github.mjakubowski84.parquet4s.FilteringSpec"- Success.TZ=Europe/Warsaw sbt "core/IntegrationTest/testOnly -z com.github.mjakubowski84.parquet4s.FilteringSpec"- Failure (1900-01-01T00:58:08 was not equal to 1900-01-01T00:34:08).TZ=Europe/Moscow sbt "core/IntegrationTest/testOnly -z com.github.mjakubowski84.parquet4s.FilteringSpec"- Failure (1900-01-01T00:04:25 was not equal to 1900-01-01T00:34:08).
from parquet4s.
marcinaylien
commented on June 21, 2024
Ha!
Europe/Warsaw is treated by JVM (?) as the timezone used in Polish Commonwealth several hundred years ago :D
from parquet4s.
mjakubowski84
commented on June 21, 2024
So, it is actually a bug in Java: https://stackoverflow.com/questions/47823081/java-sql-timestamp-wrong-time-parsing
In order to fix, one must avoid using Timestamp.valueOf or similar
from parquet4s.
flipp5b
commented on June 21, 2024
Well, yeah :) Switching zeroDate to a more recent date (like LocalDate.of(2000, 1, 1)) works for all the time zones above.
from parquet4s.
Related Issues (20)
- Add Pekko support HOT 4
- Is it possible to write a file without Akka or Fs2 integration? HOT 2
- `ParquetReader.projectedGeneric` does not work when selecting more than one column from a same group HOT 5
- Reading from gcs bucket HOT 1
- Do not publish a pekko/akko versions of scapapb module HOT 1
- missing tail records of large(~193M) parquet files HOT 4
- Protobuf enums deserialisation HOT 3
- compatible parquet-hadoop with spark3.1 HOT 3
- Unsure how to use for 'semiauto' approach HOT 2
- ParquetSchemaResolver test fails on recent JVMs HOT 1
- [akka/pekko] Too many paths created during record partitioning HOT 2
- [RFC] Refactor timestamp codecs HOT 2
- Feature request: Expose partitions as a `Stream[F, Stream[F, Record]]` for FS2 HOT 5
- Incorrect value after reading parquet HOT 7
- [Question] get a listing of parquet files? HOT 4
- [Question] Is there a mechanism to detect when the `rotatingWriter` finishes writing to a file and to be notified of the file that was written? HOT 1
- Support vectored io introduced in Parquet 1.14 HOT 1
- Efficent way to read big files? HOT 2
- Possiblity to write avro IndexedRecords to Parquet using ParquetStreams HOT 1
- feat(akkaPekko): Retry mechanism for ParquetPartitioningFlow HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from parquet4s.