lgsinnovations / jupyter-sigplot Goto Github PK

The example notebook tries to show that you can overlay a 2d array, ask for a 2d plot, and not have to specify the subsize. But it doesn't work! This may have changed in the recent array refactor.

We removed the versions to fix #47, but it seems this is breaking fresh installations of Jupyter-SigPlot in Python 2 (Python 3 works).

@RinconResearch @sterre what versions are you and your customers able to use? My environment is very flexible, so I can use the latest version, but one of my customers is constrained by Jupyter 5.2.2 and ipywidgets==6.0.2 (still need to fix this, see #40.

cc @jrmims

Per @sterre's recommendation:

Explore generically dispatching server-side calls to the client as much as possible; for example, a Python call to sp.change_settings({'xmin': 0}) would be sent to the browser side as something like a json {"function": "change_settings", "arguments": { "xmin": 0 } (ideally without special Python-side logic for the function name, either). It might be able to replace oldArrays, oldHrefs, and inputs, and possibly make future API changes to the Javascript SigPlot object transparently available to to widget.

This would require some pre-processing for those cases where we're currently moving files into a space accessible to the notebook server--this needs to be done generically.

Setup for #4. Will need:

• Setup travis in general, add badge, etc.
• Python
  • Tests
  • Flake8/pep8
  • Coverage
• [ ] JS Replacing with Selenium

Tied to #12.

The example notebook (example/JupyterSigplot.ipynb) refers to amatma's fork instead of the main repo in its Binder link.

• git URL is advertised as: git clone https://github.com/LGS Innovations/jupyterSigplot.git
• But the whitespace in LGS Innovations appears to be erroneous
• The URL that worked for me was: git clone https://github.com/LGSInnovations/jupyterSigplot.git

Client-side code in _done creates, but then immediately removes, an element for the PNG rendering of the plot's canvas.

I think we can just delete them...

There appears to be an issue with the following environment:

ipywidgets==6.0.1
notebook==5.2.2

Both Python and JS tests. The JS tests should simply make sure the right values are being passed to SigPlot.js and the Python tests should make sure the right objects are being created (that end up getting synched with the JS side).

• Python Tests
• JS Tests
• Python Selenium Tests to test JS

If a Jupyter Notebook server started in ~/foo, but there's a notebook "my_notebook.ipynb" in ~/foo/bar/my_notebook.ipynb, overlay_href will download the file "myfile.tmp" to ~/foo/bar, but will send the client /files/myfile.tmp instead of files/bar/myfile.tmp.

There's still an import from the removed example module in __init__.py that prevents successful import of jupyter_sigplot at version 0.1.1:

Removing this line allows successful import and instantiation of a widget.

It seems likely that testing was done in a clone that still had the removed module in place, suggesting an opportunity for automated testing.

Running the MyBinder demo, I get lots of error pop-ups from hub.mybinder.org saying:

Failed to load data

whenever I try to load data in, with the result that the demo notebook does not show off the full range of capabilities of the SigPlot package.

Some of the logic for caching URLs and symlinking files assumes that the kernel's cwd matches that of the jupyter notebook server. I believe some things will break if the kernel issues a chdir.

Some old documentation (e.g., jupyter_sigplot.sigplot.SigPlot was modified to .Plot)

We've talked about this some in person and on Slack. This issue is just trying to capture some of what we've discussed, with no strong organization.

Currently, when a saved notebook is re-opened, SigPlot widgets do not reliably show a rendered image without re-evaluating the generating cell and all dependencies. This is especially vexing in cases like nbviewer or GitHub / GitLab, and is likely a showstopper if the original data is no longer available.

There's some nascent logic in the extension around done and imageOutput that looks like it wants to capture a png from SigPlot and save it to the client for rich representation. This seems like a solid approach, with the only question being how to make that PNG repr display at the right time. It's possible that widgets and rich representation don't mix--this from a very quick experiment where I tried to add an HTML representation to the hello world widget.

Libraries like Matplotlib/Seaborn and Bokeh seem to address this by using a Javascript rich representation instead of a bona fide widget. I make this claim based on observing what's saved with a notebook containing figures from each library.

• Matplotlib with %matplotlib notebook generates a Javascript and image representation. On load, the image is displayed until the cell is re-evaluated.
• Matplotlib with %matplotlib inline just generates an image repr.
• (Seaborn is a wrapper around Matplotlib, and its save semantics are the same.)
• Bokeh generates a Javascript representation that appears to load the required library from a CDN (I believe it can also be inlined), plus a JSON representation that the Javascript code renders.

I thought D3 might be a reasonable analog to SigPlot, so went looking for some examples of D3 in a notebook. Here's what I found. None of these is as complete as we might like.

• https://www.stefaanlippens.net/jupyter-custom-d3-visualization.html
• https://medium.com/@stallonejacob/d3-in-juypter-notebook-685d6dca75c8
• https://gist.github.com/jakevdp/289be7d8c6e3a1248921

It appears that Javascript reprs take precedence over other rich reprs. This may depend on whether the notebook is trusted. If you return None from a _repr_* function, that repr is not used, which could potentially allow us to wait until a PNG was available before rendering it.

Whatever the representation in the saved notebook, it needs to deal gracefully with very large input data. Matplotlib and Bokeh do this by serializing the figure instead of the data. (There's a size inflation for small data sets, but a big saving on larger data.) A DataShader-style approach may also be relevant.

JupyterLab has a different extension model, and also restricts Javascript content.

The overlay_array, overlay_href, overlay_file, and plot functions all have the @register_line_cell_magic decorator, but they don't actually seem to work as magics. They should be fixed or removed.

We have several checks like isinstance(arg, str) that return False in Python 2 if the argument is a Unicode string. This happens in practice if you're using Jupyter magics for plotting, and passing the magic argument to __init__, overlay_href, or overlay_file.

You can get the right answer in Python 2 by checking isinstance(..., basestring) instead, but the basestring type does not exist in Python 3, where all strings are Unicode. Neither does the unicode type.

One way to fix this would be to replace those base isinstance calls with calls to a function that makes a version-appropriate check. Another is to have your magic call str(line), which will do the right thing in both Python 2 and Python 3, but at the expense of requiring the magic author to remember to do this.

Recommend we decide how we want to address this (as code or documentation), then implement.

@sterre On macOS, /tmp is actually a link to /private/tmp, so test_unravel_path fails.

>               assert(actual == expected)
E               AssertionError: assert '/private/tmp' == '/tmp'
E                 - /private/tmp
E                 + /tmp

Jupyter Notebook intercepts keyboard shortcuts

When I use overlay_array to plot a large array (e.g. 20M points) it takes a significant amount of time (i.e. over one minute) for the data to be displayed in the widget. This is basically what my notebook looks like:

In[1]:

from jupyter_sigplot.sigplot import SigPlot
import numpy as np

x = np.arange(0, 200e3, 0.01)
sinx = np.sin(x)
cosx = np.cos(x)

In[2]:

plot = SigPlot(sinx, cosx, xi=True, cross=True, all=True)
plot.plot()

In[4]:

# After writing sinx and cosx to Type 1000 'SF' files ...
plot = SigPlot("sinx.tmp", "cosx.tmp", xi=True, cross=True, all=True)
plot.plot()

When I evaluate the second cell, a blank plot is displayed right away, but the data takes over a minute to show up, and there is no indication that any background work is taking place.

When I evaluate the fourth cell, the full plot is displayed almost instantly, including the data.