jupyter-xeus / xeus-python Goto Github PK

import inspect

def foo():
    pass

inspect.getsource(foo)

does not seem to work in the xpython kernel...

Line 6:   inspect.getsource(foo)

File /home/martin/miniconda3/envs/foo/lib/python3.7/inspect.py, in getsource:
Line 973:   lines, lnum = getsourcelines(object)

File /home/martin/miniconda3/envs/foo/lib/python3.7/inspect.py, in getsourcelines:
Line 955:   lines, lnum = findsource(object)

File /home/martin/miniconda3/envs/foo/lib/python3.7/inspect.py, in findsource:
Line 786:   raise OSError('could not get source code')

OSError: could not get source code

Resulting in https://github.com/QuantStack/py2vega/ to not work (EDIT: at least not in the case Python functions, it would work with string functions)

It seems that ptvsd doe snot support interactive mode yet (see microsoft/ptvsd#201).

The result is that we have two stackframes, one from the interactive mode and one from the code that hit the breakpoints. When we reach the end of the file containing the code of the cell, it comes back to the interactive frame. This is the same behavior when the code is stoppe don the last line of a funciton and we click the next button: it commes back to the call site.

When a stop event is emitted, the debugger should check that the thread is not stopped on a "interactive call site", that is, the stackTrace result does not contain a single stackFrame whose source path is "". If so, the stop event should be discarded and a next request should be sent to ptvsd.

When having a cell with this content:

class Foo():
    pass

f = Foo()
f;

it displays the value of f, which is not the case using ipykernel

We should think about Matplotlib support. Maybe implement a custom backend, or try to reuse the notebook back-end.

Typing the following:

In [1]: x = []                                                                                                                                                                                            

In [2]: x[0].

and pressing tab for completion will raise an error:

ERROR: received bad message: IndexError: list index out of range                                                                                                                             
Message content: {"code":"x[0].","cursor_pos":5}

Unhandled exception in event loop:
  File "/home/martin/miniconda3/envs/xeus-python/lib/python3.6/site-packages/prompt_toolkit/eventloop/coroutine.py", line 90, in step_next
    new_f = coroutine.throw(exc)
  File "/home/martin/miniconda3/envs/xeus-python/lib/python3.6/site-packages/prompt_toolkit/buffer.py", line 1648, in new_coroutine
    yield From(coroutine(*a, **kw))
  File "/home/martin/miniconda3/envs/xeus-python/lib/python3.6/site-packages/prompt_toolkit/eventloop/coroutine.py", line 90, in step_next
    new_f = coroutine.throw(exc)
  File "/home/martin/miniconda3/envs/xeus-python/lib/python3.6/site-packages/prompt_toolkit/buffer.py", line 1500, in async_completer
    cancel=lambda: not proceed()))
  File "/home/martin/miniconda3/envs/xeus-python/lib/python3.6/site-packages/prompt_toolkit/eventloop/coroutine.py", line 86, in step_next
    new_f = coroutine.send(None)
  File "/home/martin/miniconda3/envs/xeus-python/lib/python3.6/site-packages/prompt_toolkit/eventloop/async_generator.py", line 114, in consume_async_generator
    item = iterator.send(send)
  File "/home/martin/miniconda3/envs/xeus-python/lib/python3.6/site-packages/prompt_toolkit/completion/base.py", line 166, in get_completions_async
    for item in self.get_completions(document, complete_event):
  File "/home/martin/miniconda3/envs/xeus-python/lib/python3.6/site-packages/jupyter_console/ptshell.py", line 119, in get_completions
    cursor_pos=document.cursor_position
  File "/home/martin/miniconda3/envs/xeus-python/lib/python3.6/site-packages/jupyter_console/completer.py", line 39, in complete_request
    msg = self.client.shell_channel.get_msg(timeout=self.timeout)
  File "/home/martin/miniconda3/envs/xeus-python/lib/python3.6/site-packages/jupyter_client/blocking/channels.py", line 57, in get_msg
    raise Empty

Exception 
Press ENTER to continue...

We should correctly catch errors when looking for completion, and just discard them.

What if we drop Python 2 support? That would remove some code

It would be nice to have support for async / await at the top-level. At the moment the kernel does not respond when using the following code snippet:

import asyncio
await asyncio.sleep(2)

Here are the logs after setting XEUS_LOG=1 and interrupting the kernel (see screencast above):

XEUS: sent message on iopub - kernel_core.f23afc045e2740f0abf2fd97d9b28e26.execute_input
msg_type: execute_input
ERROR: during execute_request
make_tuple(): unable to convert arguments to Python object (compile in debug mode for details)
XEUS: sent message on iopub - kernel_core.f23afc045e2740f0abf2fd97d9b28e26.status
msg_type: status
terminate called after throwing an instance of 'zmq::error_t'
  what():  Interrupted system call[I 21:25:59.177 LabApp] Kernel interrupted: c6315021-3d48-4b8a-936d-1d013872d051

[I 21:26:01.900 LabApp] KernelRestarter: restarting kernel (1/5), keep random ports

Tested with xeus-python 0.6.6.

Hi!
Thank you for your work!
Trying the xeus-python kernel with binder using your project badge, I noticed a cell causing kernel repeated crashes:

from ipywidgets import interact
@interact
def foo(x = ['a', 'b'], n=(1, 10)):
    print(x * n)

I'm not able to help so it's just a simple report.

The display function should have the same signature as the one from IPython. For example the IPython display function takes a list of objects to display, not a single object.

... if it doed not exist. Also this folder should be created under the default tmp folder.

to_json and from_json for py::object should be optimized a bit. Instead of parsing string representation of the object we should implement recursive functions that create the json/py::object

libc++abi.dylib: terminating with uncaught exception of type pybind11::error_already_set: AttributeError: type object 'kernel._Mock' has no attribute 'showtraceback'
At:
/Users/kevin/miniconda3/envs/ngl-hack/lib/python3.6/site-packages/ipywidgets/widgets/widget.py(223): m

hi all, our nglview user and I got above error in bash terminal when trying a simple example of nglview: macos, chrome, ipywidgets 7.5.1, latest xeus-python from QuantStack.

There's no simple code to reproduce, but you can try "xeus" branch in nglview repo

import nglview
nglview.demo() # this will crash (kernel dies) the notebook in xeus-python

Should the tmp_file_prefix, tmp_file_suffix and stopped_threads fields also follow camelCase, for consistency?

https://github.com/QuantStack/xeus-python/blob/98c46803888cc20928c3c7367418363ee7f565d9/src/xdebugger.cpp#L200-L206

It seems that loading the bunny video in the example notebook on binder causes the frontend to indicate that the kernel has died while it has not. Further code execution still works.

cc @JohanMabille

I had this error:

AttributeError: 
'xeus_python_logger.XPythonLogger' object has no attribute 'flush'

I don't know how to reproduce it. Maybe we should implement flush, if it makes sense.

the debugInfo command should be handled even if the debugger has not started, which is not the case today.

hi,

Please see very simple example

from ipywidgets import *
Text("hello")

In the JS console. I see error:

utils.js:35 Uncaught (in promise) TypeError: Cannot read property 'filter' of null
    at h (utils.js:35)
    at e.update_classes (widget.js:801)
    at widget.js:731

line 731: _this.update_classes([], _this.model.get('_dom_classes'));
In IPython, I think _this.model.get('_dom_classes') returns an empty list in this case.

When using output widgets and adding data from multiple threads, xeus-python doesn't seem to properly update the contents of the output widget. Only upon triggering the kernel from the frontend (e.g. by activating auto-completion or executing a print statement) the contents of the output widget get updated.

I am using the following output redirector:

import threading
import ipywidgets as widgets
import sys

output_registry = {}

class OutputRedirector:
    def __init__(self, original):
        self.original = original
    
    def write(self, msg):
        thread_name = threading.currentThread().name
        if thread_name != 'MainThread':
            output_registry[threading.currentThread().getName()].append_stdout(msg)
        else:
            self.original.write(msg)

    def flush(self):
        self.original.flush()

    # necessary for ipython, but **not** for xeus-python!
    def set_parent(self, parent):
        self.original.set_parent(parent)

sys.stdout = OutputRedirector(sys.stdout)

And now you can create a new thread with outptu widget, e.g.

def func():
    while True:
        print("Hello?")

def new_thread():
    output_registry['test'] = widgets.Output()
    thread = threading.Thread(target=func, name="test")
    thread.start()
    return output_registry['test']

We currently configure the embedded python interpreter by setting the PYTHONHOME to be that of the prefix at compile time. This causes some issues because of the binary relocation is not enabled on windows. (A simple fix would be to read that information from a text configuration file).

There are some discussions about getting the configuration from Python interactive here:
https://bugs.python.org/issue35706

Optionally, input request could return EOF on timeout. This would prevent locking on repeated input request.

See ipython/ipython#11841,

IPython seem to sometime return base64 str, and sometime return raw data depending on the type of the object (lib.display.Image vs matplotlib.Figure for example).

I haven't checked how that affect data on the wire when you use ipykernel, but as you have spent more time than me recently on the protocol; maybe this is something you have observed; and have thoughts on.

I'm tempted to change IPython to always return bytes, but just want to know your thoughts.

A showtraceback function should be added in xeus-python. It's used by ipywidgets https://github.com/jupyter-widgets/ipywidgets/blob/master/ipywidgets/widgets/widget.py#L191.
An error is thrown when this code is triggered.

As described in #161, running ptvsd form a interpreter in interactive mode results in two stackFrames when the code hits a breakpoint: one for the code being executed, and one from the interpreter itself. The debugger should filter the results of a stackTrace request and remove the additional stackFrame due to the interactive mode before returning the result to the frontend.

Publishing xeus-python as a package that can be installed by pip opens it up to use in a wider set of venues, namely in environments where conda is unavailable.

... from a middleware function from xeus instead of being hardcoded.

I have seen the tests hang forever in a deadlock.

See e.g.
https://ci.appveyor.com/project/QuantStack/xeus-python/builds/28997776/job/v8rmy9gixbveb3ix

We should try to run benchmarks against ipykernel. And see if statically linking to Pythonlibs can improve performance.

Because we monkey patch the display module from IPython, users cannot do something like:

from IPython.display import IFrame

because we do not provide the IFrame class in our patch module. I suppose the way to go would be "merge" the IPython.display module with our patch instead of completely overriding it, and we completely override it only if IPython is not installed.

Just what the title says :)

The debugInfo response should tell if the debugger has stopped on a breakpoint. If so, additional members should specify the file and the line number of the breakpoint.

Line magics:

• %alias
• %alias_magic
• %autoawait
• %autocall
• %automagic
• %autosave
• %bookmark
• %cat
• %cd
• %clear
• %colors
• %config
• %connect_info
• %cp
• %debug
• %dhist
• %dirs
• %doctest_mode
• %ed
• %edit
• %env
• %gui
• %hist
• %history
• %killbgscripts
• %ldir
• %less
• %lf
• %lk
• %ll
• %load
• %load_ext
• %loadpy
• %logoff
• %logon
• %logstart
• %logstate
• %logstop
• %ls
• %lsmagic
• %lx
• %macro
• %magic
• %man
• %matplotlib
• %mkdir
• %more
• %mv
• %notebook
• %page
• %pastebin
• %pdb
• %pdef
• %pdoc
• %pfile
• %pinfo
• %pinfo2
• %popd
• %pprint
• %precision
• %prun
• %psearch
• %psource
• %pushd
• %pwd
• %pycat
• %pylab
• %qtconsole
• %quickref
• %recall
• %rehashx
• %reload_ext
• %rep
• %rerun
• %reset
• %reset_selective
• %rm
• %rmdir
• %run
• %save
• %sc
• %set_env
• %store
• %sx
• %system
• %tb
• %time
• %timeit
• %unalias
• %unload_ext
• %who
• %who_ls
• %whos
• %xdel
• %xmode

Cell magics:

• %%!
• %%HTML
• %%SVG
• %%bash
• %%debug
• %%file
• %%html
• %%javascript
• %%js
• %%latex
• %%capture
• %%markdown
• %%perl
• %%prun
• %%pypy
• %%python
• %%python2
• %%python3
• %%ruby
• %%script
• %%sh
• %%svg
• %%sx
• %%system
• %%time
• %%timeit
• %%writefile

An alternative to conda-forge/xeus-python-feedstock#27 would be for xeus-python to give an error message when:

• ptvsd is missing
• xpython_debug_logs folder is missing

Temporary files should be created in a temporary folder whose name contains the pid of the process to avoid collisions when many kernels are running.

At the moment we implement the source request from the Debug Adapter Protocol directly in xeus-python instead of forwarding the event to ptvsd, because it doesn't seem to be supported in interactive mode yet.

However the DAP requires sourceReference to be sent in the source request: https://microsoft.github.io/debug-adapter-protocol/specification#Requests_Source

So if we want to follow the DAP closely we should consider making the sourceReference field mandatory (even if it's not used).

For the kernel_info_request_impl, we need to find a way to retrieve the Python version currently used. Maybe the Python C API provides a MACRO that contains this version value?

The display does not work sometimes:

In [1]: list
Execution error: TypeError: descriptor '__repr__' of 'list' object needs an argument
In [2]: BaseException
Execution error: TypeError: descriptor '__repr__' of 'BaseException' object needs an argument

Hello!

We at CLASP (https://github.com/clasp-developers/clasp) and CANDO (https://github.com/cando-developers/cando) are very excited about Xeus and looking to use Xeus-python to work with our chemistry simulations. We love ipywidgets, and we're glad to see that Xeus-python fully supports the widgets.

However, what we really need in our notebooks is the NGLview widget (As seen on Jupyter's website) (GitHub Repo), which allows us to view our molecular structures and trajectories. I tried (along with the help of the nglview developers) to get NGLview up and running in a xeus-python kernel, but we could not get it up and running properly (for more info, see: nglviewer/nglview#844). For instance, we ran into the same issue as #106, where we have code from IPython.display import HTML, JavaScript, display but we cannot import HTML or JavaScript. After hacking past those errors, we still could not get the widget working.

We're curious to know what the intentions are of Xeus-python to support 3rd party ipywidgets' extensions, such as NGLview. Any information you may have about the current status about 3rd party widget support in Xeus-python or the intended support for them would be greatly appreciated.

Thanks!

In the last release, I renamed xis_complete.hpp into xcompletion.hpp. That was a mistake. xis_complete is for checking if a piece of Python code is complete, not for code completion.

It seems that if py::exec get fixed (with respect to calling other functions defined in the same call), we should be able to

• drop the custom six-based exec function.
• drop the workaround due to the extra function in the traceback.

Also, six is used for selecting between PY3 and PY2 in is_complete_request.

At the moment xeus-python always returns "true" for the debugger key of the kernel info request.

Should we return "false" if ptvsd ia not available?

When displaying a widget, an error occurs:

Execution error: TypeError: display(): incompatible function arguments. The following argument types are supported:
    1. (obj: object, display_id: object=None, update: bool=False) -> None

Invoked with: {'text/plain': 'IntSlider(value=0)', 'application/vnd.jupyter.widget-view+json': {'version_major': 2, 'version_minor': 0, 'model_id': 'fc3a0f66c4624f5ba9fb724abe7702b5'}}; kwargs: raw=True

At:
  /home/martin/miniconda3/envs/xeus-python/lib/python3.6/site-packages/ipywidgets/widgets/widget.py(722): _ipython_display_
  <ast>(1): <module>

I guess we need to add the raw argument to display and update_display, like it's done for the display_hook

We should improve the display of errors, with a clear traceback.