Code Monkey home page Code Monkey logo

colaboratory's Introduction

CoLaboratory

This repository, jupyter/colaboratory is no longer being actively developed or maintained. For the latest in Colaboratory, check out Colab from google research. That site is an up to date version maintained by the Google Colaboratory team. For new issues see: https://github.com/googlecolab/colabtools


This repo contains two related tools:

  1. The CoLaboratory Chrome App.

  2. CoLaboratory with Classic Jupyter Kernels.

Both of these create and store notebooks in Google Drive and allow for collaborative editing of notebooks. The difference is that the Chrome App executes all code inside the Chrome browser using the PNaCl Sandbox), while coLaboratory Classic executes code via local Jupyter kernels (such as the IPython kernel) that have complete access to the host system and files.

Setup

First clone this repo:

git clone --recursive https://github.com/jupyter/colaboratory

Installing CoLaboratory Classic Frontend for Local IPython Kernels

Run

cd colaboratory
pip install -r requirements.txt

to install the dependencies.

NOTE: This must be run from the colaboratory directory.

If you did not use the --recursive flag when cloning, you will get errors like:

[tornado.access] WARNING | 404 GET /static/closure/css/common.css

To fix this, run git submodule init && git submodule update.

Start IPython notebook:

python -m colaboratory

This launches the web application.

Navigate to http://127.0.0.1:8888/static/colab/welcome.html in your browser.

Installing the CoLaboratory Chrome App

Run

python colaboratory/install_chrome.py

This creates an unpacked Chrome App, in the build_chrome/ directory.

To install this app in Chrome, follow the instructions for installing an unpacked extension (extensions are apps for these purposes), at https://developer.chrome.com/extensions/getstarted#unpacked. The extension is located in colaboratory/build_chrome/.

Caveats/Requirements

IPython Version

If you already had IPython installed, and the version you had installed was not 2.x, then Colaboratory will not work. You can either upgrade (or downgrade) to 2.x, or use virtual env. To check the version of IPython you have installed, run ipython --version.

Loading Python Libraries

Currently there is no way to install new libraries in the PNaCl kernel.

Running from colaboratory.jupyter.org

The website colaboratory.jupyter.org can be used for interactive editing of notebooks, but cannot yet connect to a local kernel. This is due to https issues that we are currently working on. However, running the Classic frontend as described above already works, as the website runs on localhost.

The Collaboration Model

CoLaboratory's collaboration model is evolving. The current model is a single collaborative notebook with separate kernels. This can lead to a mismatch between a user's kernel state and the state of the notebook.

To understand how a state mismatch can manifest, consider the scenario below. Bob and Sue are working on the same notebook at the same time. Both Bob and Sue will have their own kernel state. Bobs changes will change the notebook Sue sees, but Sue's state is unchanged. If sue tries to access the variable Bob created, she will get an error.

Collaboration Error

Development

You can simply edit JavaScript code in place and these changes will be reflected automatically. You may need to refresh the browser several times to clear the cache after making changes.

If you add new files or add/remove goog.require statements at the start of JavaScript files, you need to run the script build_colab_deps.py in order to compile the list of dependencies.

colaboratory's People

Contributors

briancullinan2 avatar captainsafia avatar carreau avatar craigcitro avatar fperez avatar ivanov avatar jdfreder avatar kayur avatar kestertong avatar marksandler2 avatar matthewturk avatar minrk avatar mr-c avatar rgbkrk avatar wernight avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

colaboratory's Issues

Reconcile kernel connection with IPython 2.0 protocol

Right now the same notebook will connect to a new kernel each time a connection is made, rather than connecting to a single kernel for that notebook, and only launching a new kernel if no kernel for that notebook is running. This is because we don't use session.js, only kernel.js, in the front end.

remove Google-y references

welcome.html

  • link to this internal Google page in static/frontend/welcome.html
  • "Learn More" is a go/colab link
  • library logos refer to dremel and big table in static/v2/img/library-logos.png
  • Feedback menu -> Report bug should point to the issues for the new location of this repo

Confusion in the execution model while working collaboratively

Right now, when two or more users are editing a single notebook collaboratively, the result can be quite confusing when execution is also mixed in: everyone sees a single, common output view, but that view is really the mix of the states of everybody's kernels.

The end-user experience can be quite confusing.

We'll need to sort out a simpler execution model, perhaps with an 'execution baton' that can be passed from one user to another but can only be held by one person at a time.

Kernel cannot reconnect after failed connection

If the user tries to connect to the wrong kernel address, then attempts to connect to a correct address, the second connection will still fail, as the function Session.delete() fails at session.js line 79.

This problem would be easily solved by changing that line, but since it is IPython code, we should instead try to understand why this is happening and address the root cause.

Open Source push - Agenda (7/27 and 7/30)

put agenda here:

  • setup colaboratory.jupyter.org to use as an entry point for the webapp. Validate that someone other than @fperez can do it with the common dreamhost credentials. See #20.
  • review history of the current repo and see whether we need to filter big binaries out of it first. See #26.
  • test install instructions/readme on a clean machine for accuracy. See #46.
  • list any current deviations of file format and sketch out future plans for each.
  • include in README a clear explanation of the differences in experience/capabilities between this and a 'regular' ipython install.
  • upgrade coLaboratory to v2
  • create plan for adding collaborative functionality within core IPython
  • Remove modified IPython source code from repo and figure out a better way of doing that

Cannot read property 'modal' of undefined on starting local colaboratory

I installed the coLaboratory Drive app from http://colaboratory.jupyter.org/welcome/.

I then ran:

git clone --recursive https://github.com/jupyter/colaboratory
cd colaboratory
pip install -r requirements.txt
python -m colaboratory

Visiting http://127.0.0.1:8888/welcome, I click "Create" to create a new notebook.

I see a popup with error Cannot read property 'modal' of undefined on starting local colaboratory:

0-first-load

but then the kernel seems to work fine:

1-executes-correctly

Purge repo of large binary blobs

Over the history of the repo, some binary blobs from PNaCl builds may have made their way into the git history. We should run git filter-branch and remove them to make the repo as small as possible before making it public.

https requests do not work in PNaCl kernel

HTTPS requests result in an error inside the PNaCl kernel. Example:

Input:

urllib2.urlopen('https://www.google.com')

Error (not full stack trace):

/usr/lib/python2.7/urllib2.py in https_open(self, req)
   1220 
   1221         def https_open(self, req):
-> 1222             return self.do_open(httplib.HTTPSConnection, req)
   1223 
   1224         https_request = AbstractHTTPHandler.do_request_

/usr/lib/python2.7/urllib2.py in do_open(self, http_class, req)
   1182         except socket.error, err: # XXX what error?
   1183             h.close()
-> 1184             raise URLError(err)
   1185         else:
   1186             try:

URLError: <urlopen error [Errno -1] Invalid value for `ai_flags' field.>

@matthewturk, I'm assigning this to you as you know more about this than me, but I'm happy to help debug this issue.

Google Closure library submodule does not work

The latest merge changes the location of the submodule containing the Google Closure library. However cloning the repo now fails to create the Closure library submodule, even when using the --recursive flag.

I believe this can be fixed by deleting the .gitmodules file and then rerunning the git submodule add command, but I'm not sure if there were reasons for not doing this.

Establishing procedures for pull requests?

Now would be a good time to establish conventions for changes to the repo.

I would propose the following procedure: every PR should be approved by

  • an Jupyter developer, to ensure changes remain compatible with Jupyter code and vision,
  • one of @KesterTong or @matthewturk, to prevent breakages of the Chrome App, and
  • one of @kayur or @marksandler2, to prevent breakages of the Classic web app.

If this is excessive, we can just have anyone from the above approve PR's, on the condition they get approval from one of the above people where needed.

How does that sound? @fperez @ellisonbg @minrk @matthewturk @kayur @marksandler2

chrome extension: C++ Error >Aborting: _Unwind_RaiseException called

Installed as instructed, local IPython technique worked fine. NACL failed. Running OSX 10.9.2

nacl>Access: /home == -1
 pnacl_kernel.js:92
nacl>Successfully created /home
 pnacl_kernel.js:92
nacl>Access: /home/zeropy == -1
 pnacl_kernel.js:92
nacl>Successfully created /home/zeropy
 pnacl_kernel.js:92
nacl>Mounted html5fs, moving on.
 pnacl_kernel.js:92
nacl>Access: /usr == -1
 pnacl_kernel.js:92
nacl>Successfully created /usr
 pnacl_kernel.js:92
nacl>Access: /tmp == -1
 pnacl_kernel.js:92
nacl>Successfully created /tmp
 pnacl_kernel.js:92
nacl>Access: /etc == -1
 pnacl_kernel.js:92
nacl>Successfully created /etc
 pnacl_kernel.js:92
nacl>Access: /mnt == -1
 pnacl_kernel.js:92
nacl>Successfully created /mnt
 pnacl_kernel.js:92
nacl>Access: /mnt/http == -1
 pnacl_kernel.js:92
nacl>Successfully created /mnt/http
 pnacl_kernel.js:92
nacl>Grabbing env var.
 pnacl_kernel.js:92
nacl>Got something.
 pnacl_kernel.js:92
nacl>DATA URL: zeropy_20140520.tar.gz
 pnacl_kernel.js:92
nacl>Extracting ZeroPy environment.
 pnacl_kernel.js:92
nacl>Extracting /mnt/http/zeropy_20140520.tar.gz
 pnacl_kernel.js:92
nacl>Asked for O_RDONLY
 pnacl_kernel.js:92
nacl>RETURNING GZF: 1055416664
 pnacl_kernel.js:92
nacl>Opened '/mnt/http/zeropy_20140520.tar.gz' successfully (into /usr)
 pnacl_kernel.js:92
nacl>Aborting: _Unwind_RaiseException called (C++ exception handling is disabled)
 pnacl_kernel.js:92
nacl>ted:1 

FilePicker API does not work in Chrome App

The FilePicker API is used to display an "open" dialog box, to open files from Google Drive. This is used in both the welcome screen and the regular notebook window (file > open). Currently the FilePicker API does not work in the Chrome App, for reasons unknown.

Delete Cloud Console Project containing old Client IDs

Previously we were using client IDs in a different project from Google's internal build. However this caused realtime API conflicts, so all Client IDs are now moved into the same project as the one used by Google's internal build.
Deleting the project containing the old Client IDs will cause any forks using the old Client IDs to stop working, but will prevent any further realtime conflicts

Basic bokeh support via output_notebook()

When trying to execute this in a cell:
from bokeh.plotting import *
output_notebook()

I get this:

RuntimeError Traceback (most recent call last)
in ()
1 from bokeh.plotting import *
----> 2 output_notebook()

/home/dave/anaconda/lib/python2.7/site-packages/bokeh/plotting.pyc in output_notebook(url, docname, session, name)
155 else:
156 from . import load_notebook
--> 157 load_notebook()
158 global _default_notebook
159 _default_notebook = True

/home/dave/anaconda/lib/python2.7/site-packages/bokeh/init.pyc in load_notebook(resources, verbose, force, skip)
36 pass
37 if not notebook:
---> 38 raise RuntimeError('load_notebook() only works inside an IPython Notebook.')
39
40 import IPython.core.displaypub as displaypub

RuntimeError: load_notebook() only works inside an IPython Notebook.

Testing framework

Figure out how to test this code in a way that integrates with the rest of Jupyter testing machinery.

Host the static content at colaboratory.jupyter.org

Create a site with all the same static content that the chrome app needs, so that Drive-hosted notebooks can be rendered without users having to install anything other than registering colaboratory.jupyter.org as a known Drive app.

Google Drive OAuth: Origin Mismatch

Got this error when I tried to install coLab in Google Drive. I know I need to change the OAuth tokens Google provides - just can't figure out where I change those, in the code.

Check in PNaCl kernel built in naclports

Check in the files for the PNaCl kernel into chrome/pnacl directory. This will ensure the Chrome App can be built from the repo, and is using the latest PNaCl build from naclports.

Change Client IDs

Current Client ID's belong to a different Google Cloud Console project to the internal Google version. This will cause realtime conflicts.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.