Comments (4)
I would like to work on this.
from arrow-datafusion.
As I mentioned here #7845 (comment), the most useful thing for me would be less ugly errors when you do try to compare a union with something else.
Also it would be good to be able to force people to cast unions in my case, so it would be nice if we could switch off comparison_coercion
somehow.
@vaibhawvipul if you're working on this, you might be interested in my logic in datafusion-contrib/datafusion-functions-json/src/common_union.rs
from arrow-datafusion.
As I mentioned here #7845 (comment), the most useful thing for me would be less ugly errors when you do try to compare a union with something else.
Also it would be good to be able to force people to cast unions in my case, so it would be nice if we could switch off
comparison_coercion
somehow.@vaibhawvipul if you're working on this, you might be interested in my logic in
datafusion-contrib/datafusion-functions-json/src/common_union.rs
This was helpful @samuelcolvin. Thank you.
My approach is similar -
- adding support for arrow union datatype
- update planner
- need to support cast
- will be extending functions to have union functions like
union_extract
.
from arrow-datafusion.
take
from arrow-datafusion.
Related Issues (20)
- ScalarValue serialization does not handle nested dictionary values
- Using `Expr::field` panics HOT 3
- Improve signature of `get_field` is function
- UserDefinedLogicalNode::from_template does not return a Result<...> >
- UserDefindedLogicalNode::from_template does not return a Result<...>. HOT 3
- Row groups are read out of order or with completely different values HOT 4
- DataFusion weekly project plan (Andrew Lamb) - May 20, 2024
- Advanced example for building an external index for Row Groups *within* parquet files
- DataFusion HashJoin LeftAnti doesn't support null aware anti join
- Incorrect statistics read for `i8` `i16` columns in parquet HOT 3
- DataFusion ignores "column order" parquet statistics specification
- DataFusion reads Date32 and Date64 parquet statistics in as Int32Array HOT 2
- Pass per-field BigQuery `OPTIONS` values to the LogicalPlan's Arrow Schema
- Expand Test Coverage for ScalarUDF's
- Make the configuration for `StreamTable` more generic to support more stream sources
- Support `date_bin` on timestamps with timezone, properly accounting for Daylight Savings Time HOT 14
- Incorrect statistics read for unsigned integer columns in parquet HOT 1
- Incorrect statistics read for binary columns in parquet
- Implement a benchmark for extracting arrow statistics from parquet HOT 1
- Incorrect statistics read for struct array in parquet HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from arrow-datafusion.