Comments (6)
I've checked-out the project but I don't understand how would we integrate with it.
As I understand, the project defines a data format, not a query language.
from prql-query.
@aljazerzen - Yes, indeed Iceberg defines the table data format and for now recommends using SQL engines like Apache Spark or Apache Flink to read / write data - as described in their Engine Support document
So one thought here woud be when PRQL integrates with Dataframes (as mentioned in the PRQL roadmap), then one of the (potential) candidates would be Spark Dataframe.
from prql-query.
I see.
So in terms of code, Iceberg has connectors that allow different engines to access tables from other engines/storage locations.
It doesn't specify anything about the query language and leaves that to your query engine. So if you are using anything that takes SQL, you should be able to PRQL with Iceberg :D
But I hear you, a guide on how to do this would be nice to have :D
Do you have a specific query engine in mind or are you asking just in general?
from prql-query.
My sense is that we've tightened our focus in this repo since that version of the Roadmap, and so will focus on the language here (updated roadmap is PRQL/prql#1374), and leave the execution to other tools, possibly https://github.com/PRQL/prql-query
@snth shall we transfer this issue there?
from prql-query.
Sure, I am happy with that.
I am quite interested in Apache Iceberg myself and motivated to support it in pq.
from prql-query.
Hi @ramkumarkb ,
The next integration for prql-query
will probably Polars. I would like to support Apache Iceberg and initially that will probably come through the DuckDB backend (who I believe are working on this) and possibly also Data Fusion.
from prql-query.
Related Issues (20)
- Testing Discord Webhook HOT 2
- Add support for homebrew
- Publish image to DockerHub HOT 1
- Add basic integration tests
- error: failed to run custom build command for `libduckdb-sys v0.5.1` HOT 31
- Error: Failed to create directory "//.duckdb"! HOT 8
- Fails when there are spaces in the filenames
- Add a docker-compose.yaml file to simplify docker run command
- Files with spaces in the filename still failing with the DataFusion backend
- sqlite: Expected integer, got something else HOT 15
- Add prql-query to website? HOT 1
- prql-query 0.0.14 doesn't build with duckdb 0.6.0 HOT 9
- 0.0.14 binary broken? HOT 1
- "Unsupported Postgres type oid" error
- Backtick-quoted column name in SQL output from query with `count_distinct` HOT 3
- method to show_query to show what SQL PRQL will generate? HOT 1
- readme example fails
- pq "from a | select { b, c }" fails with syntax error
- This repository should be archived? HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from prql-query.