Comments (9)
mjakubowski84
commented on July 2, 2024
Hi @flipp5b !
Thank you for the heads up! I would say that you found the bug, and parquet4s-fs2 should not close the file system. Feel free to create a PR with a fix.
from parquet4s.
flipp5b
commented on July 2, 2024
Hi @mjakubowski84,
Thank you for the great library! Sure, I'll prepare the PR.
from parquet4s.
flipp5b
commented on July 2, 2024
@mjakubowski84, I'd like to add a test to reproduce the bug, but it looks like the issue is specific to a distributed file system. So I wonder if it's acceptable to add a new test suite with testcontainers running a custom single-node Hadoop cluster image?
from parquet4s.
mjakubowski84
commented on July 2, 2024
sure! please do!
from parquet4s.
mjakubowski84
commented on July 2, 2024
@flipp5b Thanks for the contribution! Would you like to add a fix also to core & akka so that the bug fix release solves the problem completely?
from parquet4s.
flipp5b
commented on July 2, 2024
Sure, I'll do that tomorrow.
…On Tue, Jan 17, 2023, 22:32 Marcin Jakubowski ***@***.***> wrote:
@flipp5b <https://github.com/flipp5b> Thanks for the contribution! Would
you like to add a fix also to core & akka so that the bug fix release
solves the problem completely?
—
Reply to this email directly, view it on GitHub
<#285 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABKAT3SHJMTG5KQZTADEIYTWS3XWBANCNFSM6AAAAAAT2SDKTM>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
from parquet4s.
mjakubowski84
commented on July 2, 2024
Thanks, I appreciate it!
from parquet4s.
mjakubowski84
commented on July 2, 2024
@flipp5b Your fixes are released as v2.8.0. Thank you for your contribution!
from parquet4s.
flipp5b
commented on July 2, 2024
@mjakubowski84, it was a pleasure! Thank you for the instant feedback and the bugfix release!
from parquet4s.
Related Issues (20)
- Decoding of timestamp fields generated by pandas/pyarrow HOT 4
- Expose all new Parquet options in Options HOT 1
- Support for map Parquet legacy format HOT 4
- Reading of files with fields of type required AND optional Int64 HOT 5
- update to hadoop 3.3.4 HOT 2
- Feature Request: parquet-protobuf HOT 8
- Some of FilteringSpec tests fail when run locally HOT 4
- Overwrite mode not overwriting partitions in S3 with `viaParquet`? HOT 5
- Applying `not in` (`in` negation) filter always resulting in 0 rows HOT 4
- Question. How to define a TypedSchemaDef for a Map[String, Any] HOT 8
- [question] is there a way to have a single parquet writer that writes for a class hierarchy? HOT 1
- [question] is there a clever way to register writers for lots of case classes HOT 1
- Question: How to provide Hadoop Path for reading files from S3 HOT 3
- Feature Request: scalapb support HOT 4
- An example for writing a required or optional value codec for custom type with more than one field. HOT 6
- Is schema backwards compatibility on projections possible? HOT 1
- Options `ParquetFileWriter.Mode.OVERWRITE` not deleting old parquet files in S3 HOT 1
- failed to read parquet generated by pandas HOT 5
- Add Pekko support HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from parquet4s.