coiled / coiled-resources Goto Github PK

There is CSV data here.

Probably best to move that to S3.

I'd be nice to organize the notebooks in accordance with the blog URLs, so they're easy to find.

The notebook for https://coiled.io/blog/dask-read-parquet-into-dataframe/ could be stored in blog/dask-read-parquet-into-dataframe for example.

That'd make it easy to find all the notebooks that correspond with the blog posts.

The coiled-datasets should be abstracted to a separate repo. That'll make them easier to document and will make it easier to setup best practices. Lots of the notebooks in this repo depend on datasets created by coiled-datasets, but we don't need the coiled-datasets creation code in this repo.

This repo has lots of top level directories that make it hard to follow. We want to have less top-level directories, but also minimize breaking changes to notebook links.

The repo name indicates that the notebooks should pertain to Coiled. We may want to abstract "Dask only" notebooks to another repo to facilitate organization.

Here's a possible organization structure:

dask/
  bag/
    json-to-parquet.ipynb
  dataframe/
    parquet/
      column-pruning.ipynb
      predicate-pushdown.ipynb
     memory-usage.ipynb
integrations/
  prefect/
    exmple-workflow.ipynb
  xgboost/
datasets/
pandas/

Let me know what you think.

this NB says do this

conda env create -f environment_pytorch-example.yml

but the env in the repo has a different name

@pavithraes