Code Monkey home page Code Monkey logo

signac-dashboard's Introduction

signac-dashboard - data visualization for signac

Affiliated with NumFOCUS PyPI conda-forge RTD License PyPI-downloads Slack Twitter GitHub Stars

The signac framework helps users manage and scale file-based workflows, facilitating data reuse, sharing, and reproducibility.

The signac-dashboard package allows users to rapidly visualize and analyze data from a signac project in a web browser.

Resources

Installation

The recommended installation method for signac-dashboard is through conda or pip. The software is tested for Python 3.8+ and is built for all major platforms.

To install signac-dashboard via the conda-forge channel, execute:

conda install -c conda-forge signac-dashboard

To install signac-dashboard via pip, execute:

pip install signac-dashboard

Detailed information about alternative installation methods can be found in the documentation.

Quickstart

In an existing signac project directory, create a file dashboard.py:

from signac_dashboard import Dashboard
from signac_dashboard.modules import StatepointList, DocumentList, ImageViewer, Schema

if __name__ == '__main__':
    modules = [StatepointList(), DocumentList(), ImageViewer(context="JobContext"),
               ImageViewer(context="ProjectContext"), Schema()]
    Dashboard(modules=modules).main()

Then launch the dashboard:

$ python dashboard.py run

signac-dashboard's People

Contributors

b-butler avatar bdice avatar cbkerr avatar charlottez112 avatar csadorf avatar dependabot[bot] avatar joaander avatar pre-commit-ci[bot] avatar rohanbabbar04 avatar tcmoore3 avatar vyasr avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

signac-dashboard's Issues

File "preview" for FileList

Feature description

When looking at files with the FileList module, it might be nice to have a "preview" page that works kind of like GitHub's previews for text (and potentially binary) files.

Proposed solution

Create a new view that includes a download button and does syntax highlighting with pygments. The current FileList buttons could link to this preview instead of direct download.

Text box to go straight to a specific job

Feature description

It would be great if there was a text box into which I could enter the first few characters of a statepoint id and go directly to that job. I can currently go to localhost:port/jobs/abc123, but it would be nicer if I didn't have to click in the address bar to do that.

List View / Grid View / Jobs buttons

Feature description

jobs_icon

In my experience, new users sometimes find the following confusing: after clicking on one job in the dashboard, they expect that clicking the List View / Grid View buttons in the top-left navbar would take them back to the list of all jobs. However, that only happens when one clicks on Jobs in the corner, and it is not apparent that is a clickable button.

Proposed solution

Perhaps the Jobs button can be made more apparent, or if not, the List View / Grid View could take one back to the initial list showing all jobs?

video viewer bugs

  1. should probably be poster='none', otherwise we get a TypeError: join() argument must be str or bytes, not 'NoneType' on
    poster=self.poster if self.poster and job.isfile(

  2. The dictionary returned by make_card should also contain the key jobid=job._id, otherwise the path to the video files is incorrect: http://localhost:<port>/jobs//file/<fname>.mp4 (notice missing job hash).

Smooth shut-down

Original report by Carl Simon Adorf (Bitbucket: csadorf, GitHub: csadorf).


The shut-down via KeyboardInterrupt sometimes fails for no apparent reason.
Repeated interrupts will eventually shut down the server.

Mobile-friendly view

Feature description

It would be nice if the navbar showing the module list would be visible when browsing from a small-screen mobile device.

Proposed solution

Perhaps it could be toggled on/off for saving screen real-estate?

Job tags

Original report by Bradley Dice (Bitbucket: bdice, GitHub: bdice).


Users should be able to "tag" jobs, i.e. for further review. Tag names and colors should be stored in the project document. Each job will have a set of its tags in the job document.

Search history

Feature description

It would be dope if searches were saved and easily accessible from the dashboard. I find myself entering fairly complicated search strings and then needing to reuse the searches.

Proposed solution

Having a search history in the dashboard page would be nice, and being able to somehow label searches would be a further enhancement.

Additional context

I'm happy to try to implement this kind of feature but I really have no idea where to start.

signac-dashboard init

Feature description

It would be nice to have something analogous to flow init, as a starting point for modifying things. e.g. I wanted to just change the titles, but then running python dashboard.py meant that I no longer had the default modules showing. It'd be much easier to have the default settings generated with an init call that users can then build on.

Proposed solution

signac-dasboard init that generates a dashboard.py script with all the default modules

Running dashboard from the source directory causes `sass.CompileError`

Original report by Carl Simon Adorf (Bitbucket: csadorf, GitHub: csadorf).


Pointing the python path to the source directory in order to run the dashboard without installation causes a sass.CompileError unless the user executes git submodule update --init before starting the dashboard.

It might be nice to trigger the submodule update when executing either python setup.py build or python setup.py develop to avoid confusion.

Job grouping operations

Original report by Bradley Dice (Bitbucket: bdice, GitHub: bdice).


Create one or more module(s) with "JobGroup" as their context, which take an iterable of jobs as an argument and return cards. i.e. matplotlib plots or averaging a field in job.document.

Hosting a dashboard on a webserver

In order to share analysis results with my team members, I would find it useful to be able to host signac dashboards on an nginx/apache web server instance hosting our domain. Is that possible/documented?

Unable to launch dashboard due to missing module: `flask.ext`

Original report by Carl Simon Adorf (Bitbucket: csadorf, GitHub: csadorf).


Trying to launch a dashboard results in the following error:

$ ./dashboard/dashboard.py
Adding '.' to dashboard paths.
Traceback (most recent call last):
  File "./dashboard/dashboard.py", line 56, in <module>
    config=config
  File "/home/csadorf/.local/lib/python3.6/site-packages/signac_dashboard/dashboard.py", line 36, in __init__
    cache.init_app(self.app)
  File "/home/csadorf/.local/lib/python3.6/site-packages/Flask_Cache-0.13.1-py3.6.egg/flask_cache/__init__.py", line 156, in init_app
    from .jinja2ext import CacheExtension, JINJA_CACHE_ATTR_NAME
  File "/home/csadorf/.local/lib/python3.6/site-packages/Flask_Cache-0.13.1-py3.6.egg/flask_cache/jinja2ext.py", line 33, in <module>
    from flask.ext.cache import make_template_fragment_key
ModuleNotFoundError: No module named 'flask.ext'

This happens with fresh installs with either pip, conda, or local install.

Display pdfs in dashboard

Feature description

Dashboard should be able to display pdfs or vector graphics formats.

Additional context

@tcmoore3 was outputting project level phase diagrams as pdfs

Only 1 Notes instance appears on dashboard

Description

When attempting to add multiple instances of the Notes module to a dashboard, only the first one (passed to the modules argument of PlotDashboard) actually appears on the dashboard.

To reproduce

from signac_dashboard import Dashboard
from signac_dashboard.modules import StatepointList, ImageViewer, VideoViewer
from signac_dashboard.modules import Notes, DocumentEditor

class PlotDashboard(Dashboard):
    def job_sorter(self, job):
        return job.sp.get('x', 0)


if __name__ == '__main__':
    modules = []
    notes_mod = Notes(name='notes 1', key='notes1')
    modules.append(notes_mod)
    other_notes_mod = Notes(name='notes 2', key='notes2')
    modules.append(other_notes_mod)
    PlotDashboard(modules=modules).main()

System configuration

$ python -c 'import platform; print(platform.platform()); import sys; print(sys.version); import signac_dashboard; print(signac_dashboard.__version__)'
Darwin-17.7.0-x86_64-i386-64bit
3.6.7 | packaged by conda-forge | (default, Jul  2 2019, 02:07:37) 
[GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)]
0.2.5

Keyboard shortcuts

Original report by Bradley Dice (Bitbucket: bdice, GitHub: bdice).


Every job on a list/grid view should have an anchor tag, and basic keyboard shortcuts should be available, roughly mirroring vim and/or Gmail, for scrolling through jobs, tagging jobs, opening jobs in a new tab, and navigating pages.

Avoid browser caching of images

Original report by Carl Simon Adorf (Bitbucket: csadorf, GitHub: csadorf).


Summary

With standard browser settings, images are locally cached, which means users might accidentally see outdated versions of plots etc.. unless they do a "deep refresh".

It might be a good idea to instruct the browser to not cache these images, or only for a very short time.
This could possibly be a module option, but default behavior should be no caching.

Observed Behavior

Hitting a simple refresh shows outdated images.

Expected Behavior

Hitting a simple refresh should always show the current version of each image.

Copy text button for TextDisplay

Feature description

It would be nice to have a button to copy the text in a TextDisplay box, instead of having to select the text and click copy or ctrl-c. Something similar to how you can copy the git repo url to make it easy to clone a repo from github.

Proposed solution

I'm not sure how to implement this feature, but I'm guessing there's some simple html thing to make it work given the ubiquity of this feature throughout the internet.

VideoViewer Partial Content

Original report by Bradley Dice (Bitbucket: bdice, GitHub: bdice).


When I play videos in the VideoViewer module, and try to seek back and forth, it refuses to work and just keeps playing from where it was before. This is probably due to Chrome sending a request for 206 Partial Content. Firefox does fine, probably because it's caching the entire video.

More info:
https://stackoverflow.com/questions/8088364/html5-video-will-not-loop

The dashboard server needs to support partial content responses for file requests, especially for the case of larger/longer videos.

Hide constant statepoint parameters on the statepoint parameters card

Feature description

It would be nice if there was an option to hide the statepoint parameters that are constant across all jobs in the Statepoint Parameters card. This would be especially useful in the exploration stages of a project when you add sp params that you think may change but have only initialized a subset of all possible jobs.

Proposed solution

I'm not sure how all of this works on the back end, but it's just loading a signac project somewhere, so there should be a way to call project.detect_schema(exclude_const=True) and pass that information along to wherever it needs to go.

Remove duplicate results when activating multiple `ImageViewer` modules

Feature description

ImageViewer(name="One plot", img_globs=["my_amazing_plot.png"]),
ImageViewer(name="All Plots"),

It would be nice if "my_amazing_plot.png" would not appear twice on the dashboard page.

Proposed solution

Perhaps one one module is activated, it could check the output from the other modules and eliminate the duplicates?

Create releases via CircleCI

Currently, Bitbucket Pipelines generate releases for new tags pushed to the repository. I need to update the CircleCI configuration to perform this release, now that we use GitHub.

filter dashboard jobs

I know I can search inside a dashboard, but can I select a subset of jobs to be shown by default in a particular project dashbard, based on a statepoint/doc filter?

Create `dashboard` executable

Original report by Bradley Dice (Bitbucket: bdice, GitHub: bdice).


The package should provide an executable dashboard that runs the dashboard file specified in the project configuration, and creates a default dashboard.py in the project directory if one does not exist.

Read modules and config from project document

Original report by Bradley Dice (Bitbucket: bdice, GitHub: bdice).


If not provided in the dashboard script, modules and config should be read from the project document.

A JSON file import/export should be available through the executable (#20).

Settings changed by the user should be kept in the user session, and written to the project document with a "save" feature.

Create conda-recipe

Original report by Carl Simon Adorf (Bitbucket: csadorf, GitHub: csadorf).


The signac-dashboard has currently the following depedencies:

  • flask
  • jsmin
  • flask-assets
  • flask-cache
  • libsass
  • cssmin
  • flask-turbolinks

All dependencies available on conda-forge!

Long file names are partly hidden

Long file names as part of the ImageViewer module (but I assume that applies to similar modules as well), are partially hidden, wheras they should probably be either broken or just cut of with some kind of placeholder such as ....

I assume changes would need to be made here (?):

<img src="{{ url_for('get_file', jobid=jobid, filename=filename) }}" alt="{{ filename }}" title="{{ filename }}" />

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.