jupyter-widgets-contrib / ipycanvas Goto Github PK

We should find a way to cache draw commands, I suppose they should be cached in the Model, and played in the View

It can be more optimized sometimes to use layers of canvases when some layers don't need to be updated that much

On my Binder instance, in the notebook MultiCanvas.ipynb, the command from ipycanvas import MultiCanvasleads to

ImportError: cannot import name 'MultiCanvas' from 'ipycanvas' (/srv/conda/envs/notebook/lib/python3.7/site-packages/ipycanvas/__init__.py)

If I follow the Binder link and then manually switch the URL to lab

https://notebooks.gesis.org/binder/jupyter/user/martinrenou-ipycanvas-w7kg97yf/lab

the Jupyter Lab user interface starts and prompts for a rebuild. After
the rebuild and restart the canvas does not work. I see this in the image demo:

And in the Javascript console I see this

Error: "Object 'jupyter.widget' not found in registry"
    loadObject default.js:1538
    loadObject default.js:1517
    _handleCommOpen default.js:994
    _handleMessage default.js:1101

Hello,

I am really having fun learning ipycanvas. However, now that I get more serious about my projects I am beginning to wonder if there is an easy way to automatically scale a canvas to the available size. For example when I resize the browser window or when I am using a phone to display my notebook it should automatically increase/decrease the size accordingly.

Kind Regards,
Domme

The draw_image method should support MultiCanvas as input

I didn't understand the effect of caching argument in the Canvas class constructor. There are examples illustrate examples when caching is enable or not?

I ran into a problem when following this documentation on how to sync image data to python, from
https://ipycanvas.readthedocs.io/en/latest/retrieve_images.html

from ipycanvas import Canvas

canvas = Canvas(width=200, height=200, sync_image_data=True)

# Perform some drawings...

def save_to_file(*args, **kwargs):
    canvas.to_file('my_file.png')

# Listen to changes on the ``image_data`` trait and call ``save_to_file`` when it changes.
canvas.observe(save_to_file, 'image_data')

Specifically, the callback save_to_file was never called. I found this issue on the ipywidgets repo:
jupyter-widgets/ipywidgets#2148

They suggest you need to use an Output widget in some situations. I changed the code to the following, as by their suggestion:

from ipywidgets import Output
from IPython.display import display
from ipycanvas import Canvas

canvas = Canvas(width=200, height=200, sync_image_data=True)

out = Output()
display(out)

# draw something
canvas.fill_rect(0,0, 10, 10)

@out.capture()
def callback(*args, **kwargs):
    canvas.to_file('my_file.png')

canvas.observe(callback, 'image_data')

I suggest you add that to the documentation. That's all :)

I'm using Jupyter Notebook 6.0.3, not using Jupyter Lab

btw this is such an awesome library, I've been building interactive simulations, so smooth with this lib! Will share some stuff soon hopefully

https://developer.mozilla.org/en-US/docs/Web/API/Canvas_API/Tutorial/Applying_styles_and_colors#Canvas_fill_rules

After calling clear() and then drawing new vectors, all the old drawing reappears as well. How do I truly get rid of it?

Hello,

I just started research on novel teaching concepts for electrical engineering students using Jupyter Notebooks. I will need to draw some slightly customized diagrams so I am really looking forward to using your library. However, I don't get an output from canvas. I tried some of the example files and only got a "meta-description" instead of an actual canvas. Here's my input and output for the smilie face example:

Input

from math import pi

from ipycanvas import Canvas

canvas = Canvas(width=200, height=200)

# Draw smiley face
canvas.begin_path()
canvas.arc(75, 75, 50, 0, pi * 2, True) # Outer circle
canvas.move_to(110, 75)
canvas.arc(75, 75, 35, 0, pi, False) # Mouth (clockwise)
canvas.move_to(65, 65)
canvas.arc(60, 65, 5, 0, pi * 2, True) # Left eye
canvas.move_to(95, 65)
canvas.arc(90, 65, 5, 0, pi * 2, True) # Right eye
canvas.stroke()

canvas

Output

Canvas(height=200, width=200)

This happens for all the example files in a similar fashion. However, I can run the example files inside the binder instance from the link on the project website. If you need more information or do have any idea what could be the problem please let me know. Thanks in advance :)

Kind regards,
Domme

My setup
Xubuntu 20.04
Firefox 78.0.1 (64-bit)
Python 3.8.2
Jupyter Lab 2.1.5
ipycanvas 0.4.7

The setup.py should fail if npm is not available

canvas.fill_rects([100.1],[100], 5)

gives "Uncaught (in promise) Unknown dtype float64"

Hi there,

I am working on using ipycanvas for a data annotation widget. Typically, I document my widget packages by:

• writing a jupyter notebook with the widgets in
• executing them manually (once)
• saving them using the jupyter notebook Widgets > Save widget state functionality
• include them in my sphinx documentation using nbsphinx

This works well for some widgets (e.g. here, where the buttons are still interactive).

However, when I try and do this with ipycanvas widgets (I am nesting a MultiCanvas inside a VBox), the widgets are not displayed properly.

Is there an issue there that I'm missing? Do I need to do anything "special" to include ipycanvas widgets? Or is it an ipycanvas bug?

Thanks very much,
Jan

We should add a get_imate_data method, similar to https://developer.mozilla.org/en-US/docs/Web/API/CanvasRenderingContext2D/getImageData and the current put_image_data. It would return a NumPy array.

We should remove the on_client_ready event, it should be invisible to the user. The front-end should first acknowledge that it is ready to receive commands, and then the back-end should start sending them automatically.

First of all thanks for this great Jupyter extension!

For combining text and drawing functions it would be very valuable to know the size of rendered text in the current rendering context. The canvas2d API contains the method measureText which returns these properties. Is it possible to implement this into the ipycanvas widget code so that the backend in python can query this method?

Not sure if this is a voila or ipycanvas issue.

Canvas.scale() works fine in notebook or jupyterlab but not in voila rendered dashboards (the shaded image below should be larger):

Also, when the voila dashboard is opened, the canvas is not visible (further action/draw makes it visible). Maybe it has something to do with the line below in the logs?

WARNING:tornado.general:403 GET /voila/files/voila/ipycanvas.js (::1): File not whitelisted

We should provide a CanvasPattern widget

https://developer.mozilla.org/en-US/docs/Web/API/Canvas_API/Tutorial/Applying_styles_and_colors#Patterns

There is currently no way of drawing a line without making a path, which I think can be confusing for users (see https://github.com/martinRenou/ipycanvas/issues/57 for example).

We could add a stroke_line(start_x, start_y, end_x, end_y) method which draws a line directly, without making a path. Even though this is not part of the official Web Canvas API.

Hi Martin, nice job your ipycanvas.

I quickly simplified and ported this codepen to a python notebook.
The resulting animation flickers, because from the nb there is no way, afaik, to have the draw() function called by the browser's requestAnimationFrame

Any suggestion on how to implement it?
In src/widget.ts and ipycanvas/canvas.py you have the infrastructure to handle callback from canvas events. It seems a good place to start.

Anyway I'm doing some test. I'd appreciate if you could describe how to build ipycanvas in local

This is the ported code:

# -- new cell --
from ipycanvas import Canvas, hold_canvas
import time

def sec_ms():
    tm = time.time()
    t = str(tm)
    ps = t.split('.')
    s = tm % 60
    ms = ps[1][:3]
    return (int(s), int(ms))

ctx = Canvas(width=300, height=300)

def draw():
    ctx.global_composite_operation = 'destination-over'
    ctx.clear_rect(0, 0, 300, 300)

    ctx.fill_style = 'rgba(0, 0, 0, 0.4)'
    ctx.stroke_style = 'rgba(0, 153, 255, 0.4)'

    ctx.begin_path()
    ctx.arc(150, 150, 105, 0, 3.14 * 2, False) # Earth orbit
    ctx.stroke()

    ctx.save()  # save Sun
    ctx.translate(150, 150)

    # Earth
    s, ms = sec_ms()
    ctx.rotate(((2 * 3.14) / 60) * s + ((2 * 3.14) / 60000) * ms)
    ctx.translate(105, 0)
    ctx.begin_path()
    ctx.arc(0,0,10,0,2*3.14)
    ctx.stroke()

    # Moon
    ctx.save()  # save Earth
    ctx.rotate(((2 * 3.14) / 6) * s + ((2 * 3.14) / 6000) * ms)
    ctx.translate(0, 28.5)
    ctx.begin_path()
    ctx.arc(0,0,3.5,0,2*3.14)
    ctx.stroke()
    ctx.restore()  # restore Earth

    ctx.restore()  # restore Sun
  
ctx

# -- new cell --
for i in range(200):
    draw()
    time.sleep(0.03)

I got a fresh install of both jupyter notebook, conda, and ipycanvas. ipycanvas, however, doesn't work:

404 GET /static/ipycanvas.js?v=20200328182705

Related JS output:

The resource from “http://localhost:8888/static/ipycanvas.js?v=20200328182705” was blocked due to MIME type (“text/html”) mismatch (X-Content-Type-Options: nosniff).
Untitled.ipynb
Loading failed for the <script> with source “http://localhost:8888/static/ipycanvas.js?v=20200328182705”. Untitled.ipynb:1:1
Could not instantiate widget manager-base.js:273
Error: "Could not create a model."
    v utils.js:119
utils.js:119
Couldn't process kernel message Error: "Script error for "ipycanvas"
http://requirejs.org/docs/errors.html#scripterror"
    makeError http://localhost:8888/static/components/requirejs/require.js?v=951f856e81496aaeec2e71a1c2c0d51f:168
    onScriptError http://localhost:8888/static/components/requirejs/require.js?v=951f856e81496aaeec2e71a1c2c0d51f:1735
main.min.js:48686:42
Error: Script error for "ipycanvas"
http://requirejs.org/docs/errors.html#scripterror 2 require.js:168:17
Source map error: Error: request failed with status 404
Resource URL: http://localhost:8888/static/notebook/js/main.min.js?v=fd76f62c63dfcfdb33c73b0d15290316
Source Map URL: main.min.js.map

(presumably related to ipycanvas' JS file not existing)

My last runs were against examples copied straight from the docs, to eliminate coding errors on my side, before I noticed the 404 on the JS file. The python class itself imports fine though, so it looks like it's just a rendering issue.

I'm assuming there's a config/install error on my part, but I'm not sure where. I followed the docs, but it doesn't properly install it. If relevant, I also installed ipywidgets after ipycanvas (I reinstalled ipycanvas afterwards just to see if that was relevant, but after testing), but ipywidgets works as expected (and the widgets actually appear). ipycanvas is still broken, and I'm not sure why.

I'm also not sure what info is needed to debug this, so if you need anything else, just tell me and I'll grab it. Running Linux Mint 19.3 x64 with the latest versions I can get my hands on of all the packages.

https://developer.mozilla.org/en-US/docs/Web/API/Canvas_API/Tutorial/Applying_styles_and_colors#Shadows

I can't get any output. For example, this piece of code:

from ipycanvas import Canvas

canvas = Canvas(width=200, height=200)
canvas.fill_rect(25, 25, 100, 100)
canvas.clear_rect(45, 45, 60, 60)
canvas.stroke_rect(50, 50, 50, 50)
canvas

the only output I'm getting is:

Canvas(height=200, width=200)

I'm running than on MacOS, with brewed Python 3.7 and latest Jupyter. All installed according to the book, without Anaconda. No errors thrown, nothing to debug. All modules present, node v13, all are there waiting. Tested on multiple browsers too (Safari, Chrome, Firefox).

I'm totally clueless. I'm new in the game, so maybe I'm doing a trivial error, but Google can't help me.

Hey there,

I really like to work with ipycanvas and built a basic structure for my project that I would like to animate interactively using ipywidgets. However, I did not figure out yet how to link for example a slider to affect let's say an angle of rotation inside my canvas. Is this natively possible and I just didn't figure it out yet? Or would I have to go through the whole JavaScript custom widget tutorial? I don't know JS and would not be able to use Jupyter Lab which would be a shame :-/
I already tried to create my own "animation engine" but only got choppy results so far. Some things which work in native Python don't seem to work fine inside of Jupyter (in particular threads and timers).

Kind regards,
Domme

Hello again,

thanks for your great support so far, Martin. I managed to realize all my ideas and achieved satisfying results with ipycanvas. One thing I am still missing for scientific diagrams is LaTeX support, though. Are there any plans on implementing it? Or do you have an idea on how to get LaTeX-supporting objects, for example an ipywidgets.Label, inside a canvas?

Cheers,
Domme

So I wanted to run the Game of Life example with Voila, but it seems to be partially working (haven't investigated the cause yet).

To test it on Binder:

• Gist: https://gist.github.com/jtpio/ac9fa41239fd0098ede03ec042aec574
• Binder: https://mybinder.org/v2/gist/jtpio/ac9fa41239fd0098ede03ec042aec574/master?urlpath=%2Fvoila%2Frender%2Fgol.ipynb

In https://ipycanvas.readthedocs.io/en/latest/basic_usage.html#optimizing-drawings you mention "internal jupyter parameters". It might be nice to be more specific about what these settings are. To get the error you mention I used

for i in range(6000):
    canvas.fill_rect(25, 25, 100, 100)

to get:

IOPub message rate exceeded.
The notebook server will temporarily stop sending output
to the client in order to avoid crashing it.
To change this limit, set the config variable
`--NotebookApp.iopub_msg_rate_limit`.

Current values:
NotebookApp.iopub_msg_rate_limit=1000.0 (msgs/sec)
NotebookApp.rate_limit_window=3.0 (secs)

Proposed text:

This communication is limited to 1000 commands/s in any 3 second window if you did not change internal Jupyter parameters, and it can be extremely slow to send commands one after the other. To modify these start Jupyter with:

--NotebookApp.iopub_msg_rate_limt=<number of messages per second>

and/or

--NotebookApp.rate_limit_window=<window measure messaging rate (seconds)>

Do this at your own risk as these limits are in place to avoid crashing the notebook server.

I have managed to create smooth physics animations within jupyter using ipcanvas. I have also managed to successfully use ipyevents on ipycanvas to trigger events.

However I am struggling to combine events within animation loop. This would be required to run a game on ipycanvas, for example when pressing keys to change the direction of a spaceship flying across the canvas.

When the animation loop is running, it appears to block the events from being processed.

I can run my animation like this:

def run_game():
    for i in range(5000):
        with hold_canvas(space):
            space.clear()
            space.fill_style = 'black'
            space.fill_rect(0,0, width, height)
            ship.update()
            space.fill_style = 'white'
            space.fill_arc(ship.position.x, ship.position.y, ship.size, 0, math.pi * 2)

And I can specify an event changing the ship's velocity like this:

from ipyevents import Event 
d = Event(source=space, watched_events=['keydown'])
d.on_dom_event(ship.thrusting)

Each one works on their own, but the event does not fire while the run_game() is running because it is blocking.

Is there a way to run this asynchronously?

Could you perhaps provide an example, which shows how one would write a game for ipycanvas?

Clearing a rectangle is not visible in previous views of the widget. To be more precise construct a canvas widget and show it:

from ipycanvas import Canvas  
canvas = Canvas(size=(200, 200))  
canvas.fill_rect(0,0,100,100)  
canvas  

Now clear a rectangle:

canvas.clear_rect(0,0,20,20)

This does not result in a change of the visible canvas widget.
Showing another instance of the widget by putting

canvas

in a new cell shows the cleared rectangle however.

Is this intentional?

It seems to be due to the fact that the views are updated by drawing the model canvas on them, but clearing a rectangle makes it transparent. Putting this.clear() before
the call to drawImage in updateCanvas fixes this for me.

Nice! Any plans for the same for xeus-cling? It would be great for my C++ course :-)

Hello,

First of all, thank you for your awesome project. I was wondering if there is an easy way to access pixel-level image data from the canvas? Is it possible to save/export canvas into an image? I noticed there is a method named create_image_data, but it only returns a zero numpy array.

The setup you currently have for image serialization works so I don't think there's a strong reason to change it, but I figured out a simpler serialization scheme from canvas -> numpy array that I think would allow you to drop the Pillow dependency if you wanted and would reduce the complexity of toBytes in util.ts. Figured I ought to share in case its helpful given how much looking at your source code has helped me.

toBytes
To get the bytes from a canvas you currently do this
https://github.com/martinRenou/ipycanvas/blob/09ca4e04c3c43bca5df46975cad28ada3d50750e/src/widget.ts#L311
https://github.com/martinRenou/ipycanvas/blob/09ca4e04c3c43bca5df46975cad28ada3d50750e/src/utils.ts#L101-L117
I think you could instead just do

const bytes = this.ctx.getImageData(
      0,
      0,
      this.canvas.width,
      this.canvas.height
    ).data.buffer;

though I found it nice to also include hte width and height in what I pass back to python (ref)
return { width: image.width, height: image.height, data: new DataView(image.data.buffer) };

then on the python side to get the numpy array without PIL the deserializer looks like (example)

_bytes = None if json['data'] is None else json['data'].tobytes()
return  np.copy(np.frombuffer(_bytes, dtype=np.uint8).reshape(json['width'], json['height'],  4))

the first line is just the function that gets used byteserialization in ipywidgets and the copy is separate the memory from that used by javascript and thus make it a writeable array.

Though I think an even more general solution is to use the serializers and traittypes from https://github.com/vidartf/ipydatawidgets which allow image_data to just be directly set and accessed as numpy arrays. For example this widget which displays a 2D RGBA image based on numpy arrays https://github.com/vidartf/ipydatawidgets/blob/master/ipydatawidgets/ndarray/media.py

I hope this is helpful! If this isn't then no worries and feel free to close this.

Hiya,

Love the library - interactive notebooks are the future!

I managed to whip up a really simple drawing-pad program on my laptop that worked well in Jupyter Lab, however when I tried loading the same notebook on a tablet (tried Firefox and Chrome) whilst it seems like mouse_down events are triggered, dragging didn't work, so drawing didn't work. I wonder whether adding the set of Touch Events might solve this problem:
https://developer.mozilla.org/en-US/docs/Web/Events#Touch_events

Thanks,

Hugh

The same way we provide a clear method we should provide a fill method

The CI seems to fail. I can reproduce locally doing pip install . on a fresh environment. The seems to be related to npm, the npm install and npm build commands seem to run with success, but then there is no lib output.

One can fix this locally by running: npm install && npm run build prior to pip install .

We might need to track this issue in our setup.py file.

We should support the drawImage method. This one should take an Image widget as input or another Canvas.

https://developer.mozilla.org/en-US/docs/Web/API/Canvas_API/Tutorial/Using_images#Drawing_images

When you want to display 1000 rectangles on the canvas, the current best way of doing it is to hold the commands and send them all at once. Something like:

with hold_canvas(canvas):
    for _ in range(1000):
        x, y, width, height = random_rect()
        canvas.fill_style = random_color()
        canvas.fill_rect(x, y, width, height)

This will send all the commands (2000 commands) at once in a single COMM message. It is fine in terms of performances. But if you want to draw a million rectangles it is not fine anymore.

We should provide a buffered version of this loop, meaning a fill_rects(x, y, width, height, fill_style) where all the parameters are either a single value (like 'red' for the fill_style) or an array that can be serialized in a binary buffer (e.g. NumPy array). This way the message that needs to be sent to the front-end is a lot lighter, and it is also super nice in terms of usability.

The pandas_sphinx_theme was recently renamed, resulting in the docs failing to build.

We should provide a Path2D widget.

https://developer.mozilla.org/en-US/docs/Web/API/Canvas_API/Tutorial/Drawing_shapes#Path2D_objects

https://developer.mozilla.org/en-US/docs/Web/API/Path2D

Hi, if an exception is thrown during an interactive action, such as a callback invoked by Canvas.on_mouse_down, there seems to be no way of knowing this.

e.g.:

from ipycanvas import Canvas

def callback(*args):
    raise ValueError("Invisible")

c = Canvas(size=(500, 500))
c.on_mouse_down(callback)
c

by contrast, errors in ipywidgets.Button calls are reported in the jupyter lab log:

from ipywidgets import Button
b = Button(description='Hi')

def callback(*args):
    raise ValueError("Invisible")

b.on_click(callback)
b

Is it possible to match this behaviour?

We should provide a LinearGradient and a RadialGradient widget classes, inheriting from CanvasGradient. And the Canvas.create_*_gradient() methods should return such widgets.

https://developer.mozilla.org/en-US/docs/Web/API/Canvas_API/Tutorial/Applying_styles_and_colors#Gradients

I am writing a notebook using ipycanvas and ipywidgets, where one of the features I want is to scale the whole canvas with a slider from ipywidgets.

I know about scale transforms for drawing, but I mean to resize the canvas element itself, larger or smaller than the initial values given to constructor.

In javascript it is trivial to set the width and height of canvas element after creation like so:

canvasElement.width = newWidth;
canvasElement.height = newHeight;

A live example can be seen in this answer on stack overflow: https://stackoverflow.com/a/39563143/251068

But when I try to use python code to set the canvas size in a similar way, it doesn't apply changes.

Is this possible now, or if not could be added as a feature in future release?

Martin, I found a strange behavior.
It seems that the event's callback function is called all at once after the last canvas operation

ubuntu 16.04
chrome 80.0.3987.163
ipycanvas 0.4.3
ipython 7.9.0
python 3.6.7

Look at this code:

from ipycanvas import Canvas, hold_canvas
import time

ctx = Canvas(width=30, height=30)
def draw():
    ctx.clear_rect(0, 0, 30, 30)

    ctx.fill_style = 'rgba(0, 0, 0, 0.4)'

    ctx.begin_path()
    ctx.arc(15, 15, 10, 0, 3.14 * 2, False)
    ctx.stroke()

ctx

tracker = []

def mouse_up(x, y):
    tracker.append(('event', time.time()))
ctx.on_mouse_up(mouse_up)

tracker.append(('begin', time.time()))
for i in range(200):
    if i == 100:
        tracker.append(('@100', time.time()))
    with hold_canvas(ctx):
        draw()
    time.sleep(0.1)
tracker.append(('end', time.time()))

During the for I click 4 times, a second from each other
This is the content of tracker. The callback is called at the same second.

[('begin', 1586535083.212178),
 ('@100', 1586535093.3813028),
 ('end', 1586535103.5517037),
 ('event', 1586535103.5942726),
 ('event', 1586535103.6389883),
 ('event', 1586535103.697385),
 ('event', 1586535103.7318618)]

We should add a sync_image_data boolean attribute and an image_data NumPy array attribute.

sync_image_data would be False by default and image_data None by default. If sync_image_data is True, we synchronise the current Canvas pixels with the image_data attribute, allowing the user to retrieve the current image on the Canvas from the Python back-end as a NumPy array.

Instead of stacking each Canvas view on top of each other to build the MultiCanvas in the DOM, we might want to use one canvas element on which we draw all layers from the CanvasModels.

This would have some benefits:

• The DOM structure will be simpler
• The MultiCanvas sizing/layout will behave exactly like a normal Canvas
• We don't need to create multiple Canvas views, we can use offscreen canvases

We might loose some performances though. As when one canvas updates, we need to redraw all layers using drawImage.

What you are doing is very close to this API already, and I am not suggesting a dependency, merely API compatibility. But if you were to do this you would get Chaco plotting on the ipycanvas essentially for free.

See https://github.com/enthought/enable/blob/master/kiva/abstract_graphics_context.py for the ABC that defines the API.

I think MultiCanvas does not receive events in ipycanvas 0.5.0 using the built-in mouse events.

Tried with the example notebook in JupyterLab:
https://github.com/martinRenou/ipycanvas/blob/master/examples/hand_drawing.ipynb

And in binder:
https://mybinder.org/v2/gh/martinRenou/ipycanvas/stable?filepath=examples

The interaction_layer is able to receive events by displaying the individual layer.
But by displaying the actual MultiCanvas, the interaction_layer does not receive events.

For now, downgrading to 0.4.7 works
conda install ipycanvas=0.4.7 --no-channel-priority
jupyter labextension install [email protected]