Comments (5)
@eyala - any update here?
from dione.
I can verify that this fix works for the sample data I have. I have given a snapshot version for the PayPal team with this bug, and if it works for them, too, we can make the sync interval a param and include it in all three of our Spark flavors.
from dione.
Just a small note on that - the sync interval of the b-tree file is an internal implementation detail.. If it is exposed externally, it should be clear that this is just a workaround, and not some meaningful user-facing configuration that the user needs to consider.
from dione.
@eyala - can we close this issue?
from dione.
Thank you for the support and taking this up.
We have conducted thorough manual testing of various scenarios and it appears that the issue has been successfully fixed.
However, I would like to highlight that the fix has not yet been rolled out to the production environment. And will inform here once the production rollout happens and it becomes stable.
Thanks
Abhishek Jain
from dione.
Related Issues (20)
- Add option to use IndexManager without Hive
- Add API to easily create AvroBtreeHash tables for non-index use cases
- ParquetIndexer currently doesn't support start_pos, end_pos
- Use Hadoop's file splits instead of start_pos,end_pos
- Switch filesDF to HadoopRDD
- Consider datasource v2
- Add missing partitions - add optional "min" partition
- Trim common hdfs prefix in index DF
- Add simple HTTP server implementation for fetches
- Add IT script
- Support non-partitioned tables
- Fix Array type value when fetching from Parquet data
- Any support for Delta files ? HOT 2
- Add support for ORC files
- Add support for split files in all file formats
- Support time range query HOT 1
- Record data_size column may have null issue HOT 2
- Support for spark 3 HOT 1
- Any plan to upgrade the spark version to 3.3.0 ?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dione.