agstephens / can-it-kerchunk Goto Github PK
View Code? Open in Web Editor NEWTesting kerchunk with some of our data and services
License: BSD 2-Clause "Simplified" License
Testing kerchunk with some of our data and services
License: BSD 2-Clause "Simplified" License
Need to think about:
:)
Please let me know how testing has gone for you.
Note that there is more care taken for CF times in both the single-HDF and dataset combine steps now.
We have a demonstration Quobyte volume (the acclim
Group Workspace - /gws/nopw/j04/acclim/outputs
) mounted on a JASMIN server. We are able to interact with it using existing S3-compliant libraries, including s3fs
and xarray
.
The README in this repository demonstrates how to generate a kerchunk
index file for (1) a single file, and for (2) multiple files.
NOTE: There is an update to kerchunk
since the scripts were created - so that you can create the aggregated files directly (rather than in a 2-stage process.
That example reads the files from object store, rather than as POSIX files.
During early experimentation with the Quobyte S3 interface we have not managed to work out how to configure the authentication components of the s3fs
and kerchunk
libraries to be able to run the kerchunk scripts.
Our recommended solution to overcome these problems is:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.