Make deployed dashboards secure, scalable about dashboards HOT 15 CLOSED

A "final" update on our mini-roadmap: https://github.com/jupyter-incubator/dashboards/wiki/Deployment-Roadmap

Returning to the original list of items in this issue:

There's no API auth on the kernel provider at the moment (tmpnb, Jupyter)

jupyter/kernel_gateway supports token auth.

There's no concept of authentication in the generated dashboard app

jupyter-incubator/dashboard_server has a pluggable auth model based on passportjs (https://github.com/jupyter-incubator/dashboards_server/wiki/Authentication)

The original code is exposed in the dashboard frontend HTML and can be manipulated arbitrarily by any user

jupyter-incubator/dashboard_server keeps all of code in the backend and only allows comm messages to travel between the client and kernel.

@JanSchulz said:

I would still love to see a "jupyter notebook only" solutions, so that I could send out an URL to my students and they could play with some dashboard without me fiddling with node.js, php or whatever :-)

jupyter-incubator/dashboards_setup has a handful of recipes for easily bringing up a notebook server, kernel gateway, and dashboard server in Docker or otherwise. The docker_deploy recipe preconfigures the notebook server to one-click deploy a notebook to the dashboard server for use by others.

In addition, the jupyter/roadmap has a section about adding dashboard rendering to the notebook server itself. How that might manifest is TBD.

@lmeyerov said:

To provide security, we need a security model (threat model + what properties we're trying to preserve). It's not clear what those are here.

There's a baseline threat analysis on the wiki written against our initial release v0.1.0. I'm about to go update it to take into account all of the improvements we've made beyond our initial prototype (jupyter/dashboards_server#196).

I'm going to close out this item. @JanSchulz, @lmeyerov if you're using or still considering using any of this work and have ideas or find issues, don't hesitate to open new items.

from dashboards.

I see this project as a potential shiny clone, meaning that I would like to use this to deploy the frontend to arbitrary users without authentications. For this the current way isn't working, as the link to the kernel is exposed and one can basically run any python code via the javascript console.

My idea for that would be version 3) -> a backend which executes the code, sends the results to the frontend and then only accepts the widget communications (if that's only "slider positions" and so on -- haven't looked into the details of the widget to backend communications) from the frontend. (

The Dashboard would then become a renamed ipynb file, which the proxy would execute and send the output frontend and would proxy the widget communication.

See also here: jupyter/notebook#399, but for that the widgets have to decoupled from the kernel, too: jupyter-widgets/ipywidgets#151)

from dashboards.

I agree the dashboard frontend -> dashboard backend -> kernel provider is the correct way to secure things. But I think 1, 2 and 3 all need to be done to get it right. There's really no action to take on this issue against this repo at this time: it's the dependencies the code here uses that need to change or swapped out:

• tmpnb or jupyter notebook -> some new kernel provider that does proper API auth (e.g., https://github.com/jupyter-incubator/kernel-gateway, potentially in conjunction with jupyter/enhancement-proposals#5)
• thebe.js -> some new client using jupyter-js-services that knows how to disentangle "execute this code" from "execute the code that was in the original notebook cell X" (e.g., a reference impl of a kernel-gateway client)

Once those building blocks exist, this project needs to adapt to use the new client / server APIs appropriately. For instance, instead of generating a dummy PHP app to serve up thebe.js, notebook code, and dashboard layout, generate a nodejs app to service up the dashboard layout and frontend js that comms with the backend js to request code execution on a remote kernel.

from dashboards.

Sounds reasonable...

I would still love to see a "jupyter notebook only" solutions, so that I could send out an URL to my students and they could play with some dashboard without me fiddling with node.js, php or whatever :-)

from dashboards.

Two updates on this topic.

First, jupyter-incubator/kernel_gateway prototype is seeded.

Second, I've done a bit of research into limiting what a frontend can do to a kernel by reading through the protocol docs more thoroughly (http://jupyter-client.readthedocs.org/en/latest/messaging.html). At the very end, there's a section stating that custom Comm channels (which is what interactive widgets use) can execute arbitrary user code. So even if we introduce a backend between the frontend and kernel, there's no way to prevent a malicious user from potentially gaining full access to the kernel on the back by limiting what message channels are available to the client.

Current thinking:

If the goal is to make it trivial for a notebook author to turn a notebook containing arbitrary code, widgets, markup, etc. into a dashboard, then the security envelope has to be put around the kernel itself in the back. If the goal puts more constraints on the notebook author about what widgets and code he/she can use, then the security story can be improved. But at that point, I do not really see a benefit of notebooks-to-dashboards. Just use shiny, bokeh, pyxley, etc. and write a new dashboard app outside the notebook.

from dashboards.

First of all, awesome :)

To provide security, we need a security model (threat model + what properties we're trying to preserve). It's not clear what those are here.

As an example, Splunk dashboarding focuses on read/write controls for both the data sources and the dashboard. To translate that here, the focus on authentication might be used to do data security: an app can inspect the user auth and propagate to an external query system. That does not address dashboard read/write protection, which therefore sounds like something that must be natively supported.

from dashboards.

@lmeyerov Thanks for commenting. I'll admit right up front that while I'm not totally in the dark with respect to security, I'm also by no means a pro at it. You clearly have some experience here. I'd love to collaborate on it.

Are you by chance in NYC / going to Jupyter Day next week there?

from dashboards.

Definitely a can of worms. Wish I was able to make it to NYC with you all.

from dashboards.

Hi @parente -- @thibaudh, @padentomasello & I are in SF. I'll be in NYC either nov ~11th or around thanksgiving, and am in DC quite a bit. Any of that help? Happy to do a call on this, and deployment & data app authoring in general. Likewise, let us know if a preferred way to help in general.

FWIW, after we finish prototyping a couple spark apps, we should have a better feel for MVP issues. E.g., we're already talking about needing decl-widgets to be cross-browser, adding/allowing a global dashboard nav, and exposing URL querystring parameters to apps. Getting remote deploys (Thebe?) up seems like a reach target for a December release, so we've been wondering what pared down hub-only version would look. (So closer to what @JanSchulz is expecting, but still secured.)

from dashboards.

Looks like we'll miss by a few weeks on a face to face. But we can still work virtually on security and deployment scenarios, maybe on the wiki associated with this repo. I've not given it my full attention yet, (and can't this week either) but I hope to soon. In the mean time, if you have thoughts / details on deployment, threat model, etc. feel free to seed a wiki page or some such and point to it from here. When I get cycles to work on this in earnest, we can setup some time to chat.

from dashboards.

Start of a threat analysis: https://github.com/jupyter-incubator/dashboards/wiki/Deployed-Dashboard-Threat-Analysis

from dashboards.

Wiki page now has diagrams for current state of affairs and what things might look like after the first round of countermeasures are implemented. Feedback / edits greatly appreciated.

from dashboards.

Work on #135 and #105 move the ball forward on deploying more secure dashboards outside of the Jupyter Notebook application (e.g., code no longer in HTML, only comm channel messages allowed after initial execute, user login to dashboard app, API key from dashboard backend to kernel provider, ...).

Work on #117 starts to make it slightly less hacky to manage kernels for dashboards deployed right within Jupyter Notebook and securing access to them with the built-in Jupyter login. No security story for preventing arbitrary code execution from users that access dashboards right within Jupyter yet. (That requires changes to Jupyter Notebook server itself to filter kernel messages which is why we're first attempting it on separately deployed dashboards where we have more freedom to muck with the backend code.)

from dashboards.

https://github.com/jupyter-incubator/dashboards/wiki/Deployment-Roadmap published and posted to the google group. Lots of other issues opened leading to minimally secured, deployed dashboards.

from dashboards.

I've been scratching things off the roadmap as we make progress. Most of the remaining work is in in the jupyter-incubator/dashboards_nodejs_app repo at this point. We're living at the bleeding edge there trying to use jupyter-js-widgets, phosphor-widget, etc. which are still largely being defined as part of the Jupyter Lab work. Might take some time to stabilize.

from dashboards.