sagemath / cloud Goto Github PK

This doesn't work

plot(cos(x)).plot3d(1)

It works with plot3d(), but not otherwise; it's probably just one more thing to implement.

there are various places, where left and middle clicks are inconsistent. e.g. usually, the left click works while the middle click doesn't.

• list of projects: in the box outside the link, middle click doesn't open in the background.
• list of files: like above
• middle-click on project tabs does not close the project.

who finds more?

Add a global file-based repository of interesting scientific and mathematical data. This could be a Ceph-based file-system share, which sits in /data. It contains directories (or sub-volumes (?)) for each set of data.

EDIT: or a private cloud, for more than one project, and based on the S3 protocol.

Inside of each of these, to do this in an orderly fashion and allow automatic processing, there should be

• the data files
• scripts, libraries or worksheets that make it easy to access them (e.g. routines to read the data into sage matrices or pandas data frames to make life easy)
• documentation: what the data is about, who contributed it, licensing (preferably CC0 or equivalent)
• to automatically create a nice index page of all data sets, there could be a dedicated manifest.yaml file, which contains structured information (name, type, author, topic, versions+updates, the script file or even the script itself, ...) which can be parsed and processed automatically.

An example could be a large collection of zeta zeros, public data (data.gov?), ...

Implementation details/ideas:

• Each contributor for data receives write access to their own sub-directory. This can be done easily, either by creating a new sub-volume for each one of them or enabling ACL on the file-system level.
• Define a global environmental variable SMC_DATA which points to /data to make moving this directory possible.

rename the "projects" tab to "home" or "overview". in it, there is (the later on improved) list of projects and next to it a general purpose sage cell. then, you can do some minor computations instantly without having to digg around in projects.

A large part of the world writes in languages that go from right-to-left. The first step to support their way of writing in a setting, where commands and other text is in English, is to selectively switch the writing direction for one single line of code. This can be accomplished by setting the dir="rtl" attribute in the div for one line.

<div class="CodeMirror" style="position: relative;" dir="rtl">

or via CSS: div { direction: rtl; }

This would make it necessary to make each line aware of its writing direction, store this, and handle surrounding line breaks and much more. Also, the user interface would need a button, which selectively toggles the writing direction and also serves as an indicator for the current line.

Also, a keyboard shortcut like Alt+d (?) could be introduced, which would also allow toggling the direction in other areas like chat, project name, etc.

(Originally, this arose in editing LaTeX documents, but it is a more general issue that covers all aspects where the CodeMirror editor is used to edit line-based documents.)

Place 1:

the "files" tab shows a big header called "Files". Next to it, there is just white space.
I suggest to get rid of that entire header line, the highlighted tab menu already indicates that we are in the "files" tab.
Same for "recent" and "log". the other tabs are ok-ish ...

Place 2:

If you are inside a subdirectory, recent, settings, log, etc. ... all of them show the sub-directory at the top-left right below the tab selection bar. Clicking on it just opens up the Files menu. I don't see the point of having the path displayed there at all and suggest to remove this entire horizontal line.

Finally, some margins should stay, everything shouldn't be glued together ;)

When I copy/paste more than one line of code, the editor could automatically indent all lines given the indentation of the line where it is inserted.

Code in buffer:

x = 1
y = 2
z = 3

Code where inserted, the "|" indicates the cursor:

def f():
    abc = 0
    |

Current result:

def f():
    abc = 0
    x = 1
y = 2
z = 3

Expected result:

def f():
    abc = 0
    x = 1
    y = 2
    z = 3

First, this requires a infrastructure on the back-end for sending emails. To avoid spamming, it should only be possible to send emails to either oneself or also all collaborators. It's also a good question which email address is used for sender? If #19 is also implemented, the name should be a unique "hash" which is not the project's ID.

Something sensible could be: <random-string>@cloud.sagemath.com.

Two use-cases:

• Like the feature "Email Collaborators" in Google Drive, add a setting in the "Log" or the "Project Settings" to send a message to all collaborators via Email.
• add a command like salvus.email(...) to allow to get notifications when a computation has finished. maybe even wrap this for convenience in a little bash script.

Here's the input:

def condor(x, y):
    e = gamma(y+1)
    a = gamma(1/2*y-1/2*x+1/2)
    b = gamma(-1/2*x+1/2*y+1)
    c = gamma(1/2*x+1/2*y+1/2)
    d = gamma(1/2*x+1/2*y+1)

    alpha = cos(pi*(y-x))
    beta  = cos(pi*(y+x))

    return log(e)+log(a)*((alpha-1)/2)+log(b)*((-alpha-1)/2) \
    +log(c)*((beta-1)/2)+log(d)*((-beta-1)/2)

%var x,y

def f(x,y):
    return float(abs(condor(x,y)))

plot3d(x*y, (x, -8, 8), (y, 0, 8), plot_points=10)

This was found by Peter Luschny.

the pdf preview in the latex editor cuts off the margins of the document. instead of this, the viewport should have a horizontal scrollbar and a horizontal offset should be set.

example: bottom of jquery's scrollLeft() docu

to reproduce: \usepackage{a4wide} is already triggering this.

The mouse-hover for revealing additional functionality is not good. Rather, I suggest to follow the common interface designs from other websites which list files, e.g. google drive, mega, etc.

• check-boxes on the left
• action buttons at the top
• "traditional" columns, sortable & save their state of sorting
• right-click context menu (which suppresses the default context menu)
• hide the git-status information for normal users (these are those who have no git repository ;-)
• show who is currently editing a file, last modification info, etc.

There is no spellchecker for LaTeX documents.

As a workaround, I also tried if the browser-based spellchecker could be enabled, but it doesn't work either (in FF).

Possible implementation:

1. run content through aspell in non-interactive mode.
   • -t is for latex files
   • -d [lang-code] can be used to select the language. (right now, no language packs are installed)
   • cat <file>.tex | aspell -a -t gives a sensible list which can be used to give suggestions, would require work on the ui, etc.
2. Somehow fix up the editor in a way, that the native spellchecker of the browser works.

There's no attempt right now for line numbers in the error messages to have anything to do with the corresponding line numbers in the editor.

1. Line numbers ignore blank lines
2. Line numbers start at 1, rather than the line number where the input to the cell began.

When implementing this, there's the question of whether the line number should refer to the line before or after the code was evaluated, since the editor line numbers can change before the error appears (e.g., inserting a new cell above). I do think this can be properly addressed.

This was reported by a few people, including Kevin Allen Martin, so it is a high priority.

Related to #18: what to do with receiving emails? A possible idea would be a daemon, which processes all incoming emails, collects them by project, and dissects the attachments for each email. All processed contents end up as text files in ~/inbox/ in a subdirectory <timestamp yyyymmdd-HHMM> and then <email-subject>.txt and attachments get the specified name or a generic one if unspecified. The reason for the timestamp directory is to avoid overwriting duplicate files and making it easy to sort by date ...

To avoid unintentional emails, maybe it will be necessary to have a unique "hash" as a name for each project ... instead of the ID.

steps to reproduce:

1. open "files", go into a subdirectory
2. open a file
3. go back into "files"

expected: i'm still in the same subdirectory.

Note: there is also a related bug: for the first fraction of a second, the path after opening a subdirectory shows root /

When showing image, make it resizable - also for salvus.file.
Maybe also add a transparent overlay button (in the top right corner) to directly download the image?

The name div next to the cursor of a collaborator has no background. A semi-transparent white background should do the trick, e.g. background-color:rgba(255,255,255,0.7);

Christophe Bal reports that the following doesn't work correctly:

oneDict = {
 'key_1': 1,
 'key_2': 2
}

Note, however, that indenting the last } one space, and then it works...

The relevant code is divide_into_blocks in parsing.py.

Like the "files" tab, add a tab for the processes run by the project. It shows a periodically updated list of running tasks.

• name, the (truncated?) command line
• start time, total of consumed user+sys time
• current CPU/MEM consumption

Things to take care of:

• the updates of the listing do not mix up the ordering too much. E.g. a good sorting criteria could be the "creation time".
• filter processes, which are project specific. that's easily done by checking the full path of the running software.

Possible actions:

• terminate & kill (a normal "X" icon for terminate, something in red for kill)
• stop & continue (toggles between "icon-pause" and "icon-play")

Bonus: it has a tree-indentation for parent processes.

A cute icon could be "cogs"

A nice approximation is running htop -u whoami`` and pressing "t" to toggle tree view.

This is just a sketch: Introducing a new concept of "groups" as a collection of independent projects owned by one or more users. The model follows roughly the model of groups on github and similar collaborative pages.

• a group is basically a collection of projects, which are not owned by one particular "real user" identity,
• a group is administered by one or more real users,
• other users can be added to be collaborators, they can have read/write access to selected or all projects of the group, or only read-only access,

Use-cases:

• research project: as such, there is a collection of projects and all collaborators can work in any one of them.
• class room: a teacher (as administrator) creates the group and all associated students can create their own projects. (needs more complex access control than the ones mentioned above, but doable)

UI/UX:

• The current project overview page lists the projects by groups as categories. The usual projects owned by the user are listed e.g. as "My Projects".
• Next to the "create project" is a "create group" button.
• The "new project" dialog also lets the user select a group where the project should be created (if allowed to do so).
• The groups themselves also need their own dialog, for adding other users, setting access rights, name, etc.

Extra:

Add a chat across all collaborators in a group, like their own communication channel. (like an extension to what the "log" is currently doing for each individual project)

The SMC Templates are now SMC Examples hosted at

https://github.com/sagemath/cloud-examples.git

I'll continue to push to both, but e.g. the help page should change the path.

Similar to #31: A global instance of cassandra, where users can get read/write access for their dedicated space and other projects have read-only access. The benefit of #31 is flexibility, while this one here might give better performance. Both approaches might also work hand-in-hand, where the files serve as the raw input data and inside cassandra sits the processed information.

After closing the last open tab inside a project, open either "files" or "recent" based on from where the last file has been opened.

1. create a project specific variable for the last tab
2. default is "files"
3. when opening from "recent", set it to "recent" - similarly for "files".
4. when last tab is closed, open based on that variable.

The file listing shows all files as if they are equal. It turns out, some of them are maybe more important, while other ones aren't.

This ticket is about introducing a ⭐ next to the file name, which let's the user bookmark a file - just like in Gmail or similar applications. These special files are always listed on top of the remaining ones, probably sorted by alphabet - below a line, the remaining files are listed as usually.

Then, it is possible to immediately have important files at hand (the .tex source files for editing, a default .term terminal, the .sagews file which also produces some images, etc.) without having an obstructed view.

The goal is to integrate "Mathics" like the IPython Notebook.

Quote from here

I think doing this would make a lot of sense. It even has (rather simple) integrated 3d graphics using three.js, etc.

The web server is Django, so the main obstruction will be adding base_url support. ... If it had another option, "--base_url", then that would be very helpful. Adding realtime synchronization support would depend a lot on their document model, which I know nothing about.

Another different thing to do would be to have a mathics pexpect interface in Sage, install mathics system-wide, and then "%default_mode mathics" would work in worksheets. We would want to make sure 2d and 3d graphics somehow work, since both are likely important.

sage: pretty_print(1+x^2)
<html><script type="math/tex">\newcommand{\Bold}[1]{\mathbf{#1}}x^{2} + 1</script></html>

It looks fine, except for the verbatim html. Has this ever worked?

The files listing is only updated when it is re-opened. New modifications by collaborators or automatically running scripts are not visible.

I suggest to either have a watch/notification system on the file-system level, or just reload the listing once a minute.

That way, when clicking through projects and looking at the files listing, you are immediately informed about new modifications.

The goal of this issue is to let users run executable files via a single click in the file browser - e.g. a play-icon next to the name.

Once #38 is implemented, the execution happens in that proposed dialog box and shows the generated output (or allows to interactively enter something)

On top of this basic functionality, the following advanced features could be realized:

• Shift-click on that icon opens a small dialog window, where users can enter command line arguments prior to executing it.

Edit: If the mentioned modal dialog above is implemented, Shift-click could actually start a new temporary terminal in the background, e.g. in .local/temp/<start-timestamp>.term and just keeps sitting there until the process ends. The additional feature for adding command-line arguments could instead be triggerd by Alt+click and explicitly show two buttons: Run vs. Run in background.

• Makefiles could also be treated like executable files. Clicking on the icon would show the list of targets and the user can click on one of them conveniently. Then, actually make -f <makefile> <target> is run.
• Similar special treatment for executable setup.py files?
• Finally, all *.sagews files could also be treated as executable files in such a sense, that all their cells are executed top-down.

core goal: There shouldn't be the need to be create a terminal file for just a few commands.

Iteration 1: Next to the wrench icon, add a small icon for a persistent project specific terminal 👎

Iteration 2:

Enhance the small single-line textbox in the file browser:

1. If the command produces some stdout/stderr output, open up a modal dialog box and display the result in a terminal. This terminal is already "dead", i.e. there is no prompt. Clicking Esc or Return just closes it and that's it. Most importantly, I can read through the text conveniently instead of that small crappy text box.
2. Additionally, add a "terminal" icon next to it, which opens up the same modal dialog with a temporary terminal in it. If there is a command in that small box, it is executed like usually in the current directory and the terminal gives you a prompt. Also, Shift+Enter instead of just Enter could open up that temporary terminal box dialog. Ctrl+d and similar close the prompt and also the terminal.

Dialog Box:

• displays the output of the command in a temporary terminal.
• a button "terminate" to kill it when the process doesn't stop. (for those who do not know about Ctrl-c.
• and also has a "close" button (x at top right?), which destroys the terminal and of course, closes the window. (additionally to e.g. Esc)

In summary, that wouldn't change much on the interface and one doesn't have the need to pile up terminal files for small commands. Also, #39 would work with that, too ... just another way to open this.

Currently, the user has to know about the Sage commands in notebook prior to starting to work with it. A collection of templates for common tasks would be tremendously helpful, because it gives a head start for accomplishing a certain task.

Here is an outline how this could work:

1. The UI contains a button "f(x)". When clicking on it, it opens up a dedicated dialog box.
2. On the left is a tree (or two columns) of topics and explicit tasks
3. The user navigates to a certain entry - e.g. Calculus > Limit Example No. 1 - sees a preview of the code and a description of what it is.
4. After pressing "insert", a new input cell below the one where the cursor is, is inserted and contains this working example.

The scope of examples could range from simple one-liners up to several lines of more complicated code. In any case, the user is then free to play around with the provided example to tailor it for the specific needs.

Extensions:

• A template system, where these snippets are parametrized by variables and functions.
• Assigning custom keyboard shortcuts to ones favourite templates.
• A "quick-search", where you just hit a keyboard shortcut that brings up this dialog with a focused search box. Type in roughly what you want and select the snippet via a single click or via keyboard cursors.
• A way to collect submissions for new entries with a review. (that could be done right here as an issue)

Related:

• Code Snippets for a Sage Cell
• Sagemath Interact

Running R has two modes. Either the full R or just the utility Rscript for running filename.R scripts.

Observe this inconsistency:

~$ which Rscript
/usr/bin/Rscript
~$ which R
/usr/local/bin/R

I think it's best to not link to Sage's R from /usr/local/bin and keep that separate from Sage's R. Power users can always do sage -R if they need to.

2d plotting uses svg=False by default because of a bug in Chrome which is hit by the matplotlib renderer. The following svg=True draws nothing, but with svg=False, draw a single vertical line:

 show(line([(10,0), (10,10)],axes=False), svg=True)

Links:

• The matplotlib issue about this
• The open Chromium issue about this

Ideas to fix this: somehow hack matplotlib to get around this, change our renderer (e.g., write a whole new renderer using something like maybe <svgjs.com> that completely avoids using matpltolib, for enhanced interactivity), or fix the bug in chrome.

I don't like using png's that much, since they look crappy on my HiDPI display...

After closing the browser tab, all the state information about which project and which files are open is lost. A possible solution could be to store this persistently in the database. When an existing session parameter is given as a parameter to the URL, the state is restored as it was before. e.g. ?session=<uuid>

Related to that, a list of existing sessions should be listed in the global settings menu. It should also be possible to delete old session (and they automatically vanish after a month?)

it is not yet in this list so...

change in laptop temperature:

• surfing to the login page, 47C to 53C, idle
• after login, constant ~52, idle
• working* with notebook up to +70C

*working=just trying out basic calculus and 2d plotting

Screenshot: after creating 3 tables á 6 cells, idle, 60C

Browser: SWare Iron (=Chrome) Version 27.0.1500.0 (201000)
Setup: Win8, Intel Core2 Duo L7500 @ 1.6GHz, 3GB Ram, GPU Intel 965 Onboard

It's cumbersome to switch between projects and files. Similarily to the web-browser, where Ctrl+Page Up/Down switches between tabs, I propose:

• Alt+PageUp/Down: switch between open file tabs, including the "files", "recent", ... tabs.
• Shift+Alt+PageUp/Down: switch between projects, including the overall user settings tab.

Keycodes: Alt = 18, Shift = 16, Page Down = 34, Page Up = 33.

... and Up = Left vs. Down = Right

proper delete files. right now, files can re-appear and so on.

• close the sync everywhere
• close all open files for all collaborators
• remove the file from the "recent" dialog
• ... possible even more

Hey!
Tried building the code. I absolutely love this, and I'd love to host this on my servers on EC2. I read a couple of posts in the mailing list - your plans to open-source the local hub, and so on. Any pointers on how I can work towards having such a setup?
I tried building the coffee modules - it throws an error, something about an unexpected 'IDENTIFIER' in misc_node.coffee - didn't find any such thing, though..

At least in FF, if I background-open more than one sagews and at the same time some autosave operation (??) happens, both are visible on top of each other.

I cannot really reproduce it, but it happens over and over again.

My feeling is, that an async-call executes a "success" function that makes it visible - but doesn't double-check if another tab is actually the one which is currently supposed to be displayed. I.e. it's a subtle race condition.

One repeatedly stated goal is to allow users to define shortcuts in the global settings.

Of course, there should be good default choices and this additional element must contain a switch to reset them back to this sensible default. Maybe, it would even make sense to have different profiles (e.g. "Windows", "XWin", "Gnome", "KDE", "OS X", ...) to accommodate for subtle differences between the operating systems. One of these could be a custom setting giving you the full power.

Exporting worksheets is not possible and downloading the raw file gives you something without the plots.

• Export to HTML: render a HTML page according to some template and transformation rules, which strip away a lot and make it "self sustained" (e.g. a small CSS file, maybe even embedded). The plots are collected in a subdirectory.
• Export to PDF: maybe the best way is to create a latex file with all plots in a new sub-directory. e.g. for basename.sagews -> basename.pdf/basename.tex.
  Then, the user can open the basename.tex document, automatically run pdflatex, and aquire the PDF file with everything embedded. The additional benefit is, that it is straight forward to make some additional modifications, tweak a bit around, etc. Unsolved: What happens when the document is exported again? Easy solution would be to always add a sortable timestamp to the new directory, e.g. basename.pdf-yyyymmdd-HHMM to indicate that it was a snapshot at a certain point in time.

the help page got very large and hard to read.

1. move the "help" tab to the top right next to the email address
2. implement the "affix" overview menu on the left, explained here

In the cool IPython notebook, I tried how the R integration works. Problem is, Sage's rpy2 is too old (I think). Just try this:

%load_ext rmagic

to trigger the error about the missing .active() attribute.

Updating via

sage -sh
$ pip install -U --user rpy2

seemed to fix this.

Maybe this is the first upstream ticket from SMC back to Sage?

When opening a new terminal, the cursor isn't visible until users types something.

... this also happens for me when re-opening an existing terminal. (I think especially when the screen size has changed from one machine to another one).

So, maybe it's always an issue after resizing it.

the project list just shows when the last modification did happen. the name of the user who did the last modification should be listed, the same maybe for each individual files.

of course, that might make it necessary to re-design some part of the interface.

also, that might be related to a general display of which collaborator is currently present and so on.

Add a dropdown list of possible backends which can be used for execution. That's much more discoverable than the default_mode and also lists explicitly all available options.

This setting should also be stored in the worksheet.

Jason:

I notice that when I have many projects open (like 6), and my window is just
small enough that the project tabs wrap to two lines (5 projects in one
line, one project tab wrapped to the next line), the restart, play, save,
and other buttons on the button bar disappear.  If I make the window
smaller, it switches to the phone layout (large buttons, no project tabs
visible), and if I make the window wider, the project tabs end up on one
line and the button bar reappears.

discussion

Right now renaming or moving a file in the UI simply does the corresponding operation on the underlying filesystem. It doesn't do anything else at all (since we simply didn't implement anything else yet!).

• Send a message to the local hub to update the sync object
• Send a broadcast message out to all attached global hubs and connected clients that the path/filename changed
• What else?

(reported by Christophe Bal, among others)

Replace the rather boring progress circles with NProgress
http://ricostacruz.com/nprogress/

This was sort of implemented at one point... then I implemented a new (vastly better) approach to file download via a raw http server, which rendered folder download no longer supported (despite button for it).

To implement this, I'll probably just have the client make a tarball (or zip?) of the folder in a temporary location, then download it using the raw file download, then delete the tmp file. It would be optimal if the temporary file were in /tmp, so it doesn't get snapshotted, and doesn't waste space in case the client doesn't properly clean up. To make this work, I should extend the raw http server to allow for serving files in /tmp in addition to files in the project itself... Maybe just a symlink from / to $SAGEMATHCLOUD/root and ensure the static http server allows following symlinks.

Reported by Christophe Bal.

In the project settings, add a text box to enter a list of patterns, which should be excluded from the file listings (if you click on the eye for the hidden files, they show up along with the .*-files). It would be best to just copy the way how .gitignore works.

Example:

temp/
*.log
*~