Comments (3)
quentingodeau
commented on June 2, 2024
1
Yes exactly GEN2 is managing the paths with the notion of directories.
Yes that also true I can, in fact, as first approach managing only the listing of directories and keep the reading with the blob sdk they should be compatible in fact under the hood the datalake also use a blob client.
Thx for the tips I didn't known that Duck had a discord I join it! Feel also free to ping me there even if I'm away or disconnected !
from duckdb_azure.
samansmink
commented on June 2, 2024
Hey @quentingodeau I'm not super familiar with the dfs endpoints, so please correct me if im wrong here.
Afaik these endpoints are for Azure Data Lake Storage Gen2 which aims to add a file system layer on top of azure blob storage. This means that operations like globs would be more efficient through ADLS than through raw blob storage.
Assuming the files that are listed through ADLS are also accessible through raw blob urls, perhaps a nice 1st PR regarding this would be to:
- add support for adls url scheme
- add special glob handling for adls urls
Feel free to ping me by mail or discord for a chat!
from duckdb_azure.
quentingodeau
commented on June 2, 2024
Close has pr has been merged
from duckdb_azure.
Related Issues (20)
- Unable to query entire parquet directory using anonymous authenticatoin HOT 3
- Sample query to read Parquet/CSV from Azure Blob Storage HOT 4
- Performance compared to querying virtual filesystem through `rclone mount`
- Test issue
- Support for specifying token directly HOT 13
- add support for AD Service Principal client/secret HOT 2
- Segfault with extension built on Ubuntu HOT 3
- Connecting without connection string. HOT 2
- Duck DB 0.10 seems to be broken HOT 3
- Segmentation fault when copying to Azure storage HOT 1
- MainDistributionPipeline build failed in main HOT 4
- Support write operation HOT 12
- Querying data from public $web container without authentication HOT 5
- AzureStorageFileSystem Directory Exists not implemented HOT 2
- Support for Device Code Flow Authentication HOT 6
- Connection timeout behind proxy network
- Cannot read using abfss from fabric lakehouse, Error while getting a connection handle. Error Code: 12005: The URL is invalid
- Support R windows_amd64_rtools build HOT 1
- MalformedJsonError due to Databricks identity column
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from duckdb_azure.