livebook-dev / livebook Goto Github PK

LiveBook is meant to be the Elixir version of notebooks with a focus on:

• Real-time collaboration
• Live features - a cell can be alive and stream values as it runs

The biggest challenges we have ahead of us is to address the drawbacks from Jupyter notebooks:

• Do not depend on evaluation order. We need to attach some ordering to cells, either explicitly or implicitly.
• How to persist the notebooks? JSON is not executable, not friendly to version control software.

Other considerations:

• Can the notebooks be executable or even deployed to prod?
• Security: how can we sandbox the execution?

Today, when starting a notebook, you end-up with this:

Unless you know you have to click on the sidebar, you will get stuck. There are a couple options we can choose to improve this (perhaps even more than one):

1. Have new notebooks already start with a section
2. Start notebooks the sidebar open
3. ???

On Notebooks settings, the "Save" button is on the right side for the File tab but on the left side for the Runtime tab.

My suggestion is to move it to the left. For the file button, we can show the filename that will be saved close to the save button (and I think having them close together is a good idea in the first place). :)

Show autocomplete suggestion relevant for the given cell. Probably something similar to IEx.Autocomplete. The editor has support for showing more info for each suggestion, so it can be nicely used to show some docs/specs.

You can choose the root of a Mix project and we will call "mix deps.get" and start something with elixir --sname -S mix run automatically for you.

Because compiling the project can fail in many ways, we will need to report the output if executing the commands above fail.

Since we have a markdown source, we should make it easier to download the markdown source if we want. The big question is where we are going to have those buttons. I believe we have two options right now:

1. Put it under notebook settings in the sidebar

2. Put those as buttons on the notebook title, similar to what we do to sections, see the red circle here:

Another notebook-wide button that we may want to make accessible is the fork button.

We need some basic security features:

1. Only localhost access unless a flag is given
2. Password (and/or token) required by default (unless a flag is given on the CLI)
3. When executing a new notebook, what should be the trust model?

More info: https://jupyter-notebook.readthedocs.io/en/stable/security.html

• Inside markdown cells, if I press [, it correctly adds ] and puts the cursor in the middle. However, if I press ", it generates two quotes and does not put the text in the middle.
• Pressing cmd+enter should default to a no-op (similar to Elixir)

Evaluated code is slow, so we likely want to use compile_quoted instead. We have to investigate. The main difficulty is passing the values of bindings back and forth but we can use the process dictionary for that.

Use case: all notebooks will need to execute some code to prepare for the sections/cells that come next. For example, code that runs Mix.install or some imports/requires. We need to support this somehow.

One idea is to mark a section (or multiple) sections as header sections. Those sections are always executed before any other section and their bindings are passed forward to the other ones. Alternatively, we can support header cells but I think marking a whole section is probably closer in intent to how we want folks to design the notebooks.

Each section and cell should have an anchor icon that changes the URL to that cell, so we can copy and paste it and share it.

Evaluating the following code takes a long time:

for i <- 1..1000 do
  IO.puts i
end

I assume this is because the browser is busy handling each line and doing virtualization, we may need to revisit the approach here. Should we also automatically scroll to the bottom, like a terminal would?

This is pending a feature to be implemented upstream on LiveView. We should probably have a small part of the cell that is draggable.

We already store the deleted sections/cells, we should have like a modal dialog that shows these and allows to insert them back somewhere. I think this is much better than a global undo/redo, because if the user wants to get back some old code, they doesn't have to undo all the work they did since then.

I have been playing with Livebook to give my presentation on Nx and MNIST and I noticed a problem related to the current working directory. For the Elixir executable, we are running it on the Livebook server directory but in this case I think it makes the most sense for it to be the in the same directory as the Livebook file itself.

This will work fine for when you open up an existing Livebook, but what to do when you start with the Livebook in memory and then you save it? Should we change cwd of the existing evaluator? I would say so.

For the Mix evaluator, we will document the root is the Mix project root. For the distributed one, it is whatever the existing node is in.

I think we can remove one new line at the end so we don't have the extra space. Additional newlines should appear though.

Try evaluating this:

Task.async(fn -> raise "oops" end) |> Task.await()

Today pressing tab consecutively behaves weirdly, as we bounce around elements without a visual indication, and we eventually end-up inside the editor, where pressing becomes 2 spaces. perhaps we should make it jump from cell to cell, with an outline, but without actually entering them?

We should allow names to be assigned to users and give each of them a cursor color.

• Implement user profiles UI (profiles should be stored on cookies that will be shared across notebooks).
• Profiles should appear on all pages. Changes to profile are broadcast.
• Show user cursors based on the user color. Mousing over the cursor should show the user name.
• Add a users button to the sidebar. Clicking on the user should take to their cursor if they are on insert mode anywhere.

Then there is a v2 of collaborative features, which is comments and allow read-only users, but we are not going to discuss those for now. We will open up issue once this one is closed.

This is just a reminder so we don't forget to incorporate them. We need tooltips on sidebar icons and on the icons on the focused cells.

That's because the output is temporarily discarded and then added back. I wonder if we can do something to address this? Perhaps we can avoid removing the output until the first stream comes? Or is that too tricky? 🤔

For a running/queued cells the run icon should be changed to a cancellation icon. Also a keybinding for that.

Since we are going to ship with an executable, we should probably rename the app from live_book to livebook. This is a matter of replacing live_book -> livebook in the code and renaming a couple files. We will likely still style it as "LiveBook". I can rename this repo accordingly once decided.

We should probably have a way to disable this behavior too.

Reference: https://twitter.com/srush_nlp/status/1361704681391685632?s=20

The notebook will have to be persisted. If the notebook is called "foo.lmd", we will save it to "foo/image.png".

We have to keep a list of deltas on the server to properly rebase the incoming ones, but we should drop the old ones. Ideally we would keep only the necessary deltas, but this would require idle clients to send additional acknowledgements of received deltas and may get overly complex. I think we should be fine limiting the list to N deltas.

Today we leave insert mode but the cell stay focused.

I am thinking we can start by using escripts but we should also consider shipping a release or using bakeware in the future.

Escripts are the most convenient for Elixir developers, so we should definitely consider it. The only downside of escripts is that it can't serve assets from the "priv" directory, so we will have to embed them like we did in the live dashboard.

Once we are done with the markdown importer/exporter, we should add file persistence to the notebook.

Each notebook may be associated to a file, to where we write to periodically (every 5s?).

Creating a new notebook can be done by clicking a button. You will be able to choose between three options:

1. New: creates a new notebook with no file attached to it - you will have to pick one in order to start auto-save
2. Open: opens up a notebook from a file and autosaves to the file
3. Import: opens up a notebook from a file but have no file attached to it

Once you select between New / Open / Import, you will select the execution mode: "elixir" (standalone), "mix" (like standalone but running mix deps.get + mix run inside a Mix project), or "distributed".

We should also handle escape codes on regular stdout messages. ANSI must be enabled on standalone mode with: elixir --erl "-elixir ansi_enabled true"

Those cells are not executed. It can be useful for when we want to only render some example code that may not yet be functional. We need a clean way to identify those cells.

The click in this case is ignored:

But it works if I do the same from an Elixir/Code cell.

It may be related to how we close the markdown editor on mousedown, which moves the button around, making it impossible for us to click it.

Many IEx functions return :"do not show this result in output" in order to now show any result on the output. We should probably respect it. Here is an example of a cell:

import IEx.Helpers
h Kernel

Today I need to explicitly press the Edit icon on the top right.

Note: we don't need this for v0.1.

Both in the UI and when importing notebooks, we should disable any HTML nodes in any markdown source. We should also disable the IAL extension as that would allow people to add random onclick events and similar. Alternatively we can keep the extension but explicitly list which attributes we allow (I guess class and not much more).

Currently when we have a cell focused and use keybindings to insert another cell of the same type (n, Esc, n) it usually breaks. This is a LV bug reported here, but opening this just to remember to update LV version once this is fixed.

If we don't want to get too fancy, we can start with using ? to show a dialog like in ExDoc.

Today we need to evaluate and then press escape. Perhaps we can support shift+command+enter as a shortcut?

Furthermore, pressing command+enter in a Markdown cell adds a newline... we should probably make it a noop?

Extend LiveBookWeb.Helpers.ansi_string_to_html to support more advanced ANSI codes like those generated with IO.ANSI.color/3 (more details here). Additionally it would be great to support multiple modifiers at once (e.g. foreground + background + italic).

@jonatanklosko, what do you think about adding ss as "show sections"? And potentially sn as "show notebook settings"?