Code Monkey home page Code Monkey logo

datreant.cli's Introduction

datreant: persistent, pythonic trees for heterogeneous data

Documentation Status Build Status Code Coverage

In many fields of science, especially those analyzing experimental or simulation data, there is often an existing ecosystem of specialized tools and file formats which new tools must work around, for better or worse. Furthermore, centralized database solutions may be suboptimal for data storage for a number of reasons, including insufficient hardware infrastructure, variety and heterogeneity of raw data, the need for data portability, etc. This is particularly the case for fields centered around simulation: simulation systems can vary widely in size, composition, rules, paramaters, and starting conditions. And with increases in computational power, it is often necessary to store intermediate results obtained from large amounts of simulation data so it can be accessed and explored interactively.

These problems make data management difficult, and serve as a barrier to answering scientific questions. To make things easier, datreant is a Python package that addresses the tedious and time-consuming logistics of intermediate data storage and retrieval. It solves a boring problem, so we can focus on interesting ones.

For more information on what datreant is and what it does, check out the official documentation.

Getting datreant

See the installation instructions for installation details. The package itself is pure Python.

If you want to work on the code, either for yourself or to contribute back to the project, clone the repository to your local machine with:

git clone https://github.com/datreant/datreant.git

Contributing

This project is still under heavy development, and there are certainly rough edges and bugs. Issues and pull requests welcome!

Check out our contributor's guide to learn how to get started with contributing back.

datreant.cli's People

Contributors

kain88-de avatar

Stargazers

 avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

datreant.cli's Issues

add tests

I should add tests that the cli args are parsed correctly.

find treants of other packages.

If I want to find other datreant specialized packages as well I would have to include them all at the beginning. Since I can't possibly know all of them this is a problem. One solution is to create a ~/.datreantrc which contains a list of specialized datreant packages. These would be loaded when dtr is started so that I can find the other Treants as well.

make installable

The script is basically done. But right now I can't figure out how to write the setup.py correctly

@datreant/coredevs any idea?

Prompt for category key/value pairs

I just started using datreant again and came up with one use-case that might also help others. Given a variety of simulation systems that are already setup without datreant, I want to provide a list of paths to datreant (or call datreant seperately for each path) and be provided an input prompt for key/values I would like to set in the Treant.

For example, I have a variety of systems with different membrane compositions. Now I want to use datreant to its fullest abilities and start quickly categorizing these old simulations.

Running something like datreant create-category ./some/path/ CATEGORY1 CATEGORY2 CATEGORY3, should create a new Treant in ./some/path/ and ask me what I want to set the value for the keys CATEGORY1, CATEGORY2 and CATEGORY3.

On a related manner, I wrote a small script that looks at all files in the path and asks which filename I want to use as the value for some key.

Improve usability of the CLI

The CLI is pretty crude right now. dtr init does not provide any output, while creating the .datreant directory immediately. I do like how we currently handle things for MDBenchmark, where we try to format all error messages in some easy to read and understand manner.

I think the CLI should:

  1. Provide output to the user, what it is going to do.
    a) Maybe the user should also be prompted (Y/n) and informed what the CLI is planning, before executing? Adding a --yes or --force flag to skip this prompt.
  2. Show a progress bar, so the user knows what's up. Click provides something already: http://click.palletsprojects.com/en/7.x/utils/?highlight=pager#showing-progress-bars
  3. Never fail with a stacktrace, but use exit codes and print a nicely formatted error.

I just read this post and think the author makes some very good points on usability and user experience.

I'll be creating some pull requests for this, as I start using this more in the following days.

Core functions

I guess it's necessary to talk about where the core functionality of this package should be.

I started using it more and, besides the functionality discussed in #6 (PR #7), I guess something more useful is a command like dtr get --category ABC where ABC is the name of a given category. This would return the associated value or raise an error.

So given that the current implementation is very basic, which functions should the CLI provide to the user? What should and shouldn't they be able to do?

I was thinking of first forking this, so I can get my use-cases up and running, but developing it for this main package would be neater.

add categorie filter

allow to specify a category to look for in the bundle. I'm not sure what the best syntax for that would be though.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.