Incorrect line number in exception tracebacks about pyscript HOT 18 CLOSED

@jorenham so here are the blockers for that approach:

• multiple scripts cannot land in the same file on the VFS or there will be more confusion than we try to solve
• because all pyscript tags share the same runtime, it's not clear how to signal one script, instead of another one, produced the error ... DOM insertion order? what if it's an event defined in script 2 but not script 1?
• if you do care about precise line error you can already saver your file as .py, write that file in the config.toml or config.json or <py-config> as file to store in the VFS and then you do import thing.py as __thing in your code ... this is the cleanest approach to me and it already works
• it is possible that we might want to improve bootstrap and teardown of each script but hooks can happen at distance because scripts can be appended dynamically ... right now expectations (and/or bugs based on these expectations) are consistent across all possible scenarios ... branching logic here and there might lead to more issues ...

After thinking about it a bit more, I think the multiple run approach would better reflect expectations and be actually easier to implement or reason about: your code that might fail runs within a dedicated chunk of Python that should be well defined in terms of lines.

I want to see how bad of a performance impact the multiple approach has and, if that's not the case, I think that's the best we can offer:

• it works with multiple scripts using the same pyodide interpreter (or MicroPython)
• it doesn't change too much logic around hooks before or after code
• it evaluates a part the program as main entry point without ever conflicting with the DOM position of the script or the numerically incremented __mainX all things that tell users nothing useful ...

As summary: do you think evaluating user code in between a run (or runAsync) for hooks and stdlib would solve your issue?

from pyscript.

is

RuntimeError: raised at line 1

what matters here?

we prepend our stdlib and run code all at once (performance reasons, mostly) so even if we do our best to make that a one-liner, the error would still be off by one so that I think this would fall into the wontfix category ... not sure what others think about it though

from pyscript.

@WebReflection What matters is that File "<exec>", line 45, in <module> is incorrect, because the error is raised on line 1.

from pyscript.

as explained, the error is raised after the stdlib is bootstrapped and prepended to the code ... your code is:

import sys
print(sys.version)

the running code is:

# we ensure pyscript module is available
# we ensure the stdlib is loaded
# we ensure all things that need to be patched are
# ... a few lines later ...

import sys
print(sys.version)

This impacts code hooks too, which can be prepended or post-pended (although this is not an issue in here), and so on ... so, what do you expect us doing, if not eventually run multiple times and slow down things as opposite of run everything at once? 🤔

from pyscript.

to clarify, I am not taking this issue lightly, I want to do the right thing, but it might means a lot of compromises, ugly branched code here and there, at the cost of a more precise indication of when or where your code failed ... this might be desirable but it requires some investigation and tons of changes in both polyscript and pyscript.

from pyscript.

pinging @antocuni @ntoll @fpliger ... what do you think?

from pyscript.

I understand the reasons behind this.
But perhaps it's possible to use something like sys.excepthook to either modify the traceback objects themselves, or to print the traceback with the correct file source and line numbers? Admittedly, it's a rather hacky solution. And in some (probably rare) cases, this might cause issues with libraries that also use sys.excepthook, e.g. pretty-errors.

from pyscript.

Or maybe instead of

# we ensure pyscript module is available
# we ensure the stdlib is loaded
# we ensure all things that need to be patched are
# ... a few lines later ...

# <main.py>
import sys
print(sys.version)
# </main.py>

you could do

# we ensure pyscript module is available
# we ensure the stdlib is loaded
# we ensure all things that need to be patched are
# ... a few lines later ...

import main

(or the analogue but with importlib)

from pyscript.

if I got your latest suggestion right, you are saying that any code should be stored into the virtual FS as module on its own and then be imported in the middle of our processing ... right?

If that's the case, it's an interesting solution but it still would slow down operations as the content needs to be saved in the VFS first, then made available through an import that should never conflict with any other module name ... I think that's worth exploring and thanks for the suggestion (still assuming I got it right) ... I'll play around with this ASAP as I think others might agree too ... for the unique module name I also think that main might be a good candidate, if not _main_ or, why not, __pyscript__ as reserved module name.

Hints there appreciated 👋

edit

P.S. as I wrote that, one thing we should be super sure about is that no globally scoped leak should happen or be relevant at all outside that module and that __terminal__, the only special case so far, would work with above mentioned solution too.

from pyscript.

@WebReflection Although I'm not familiar with the virtual filesystem, I think that your interpretation matches my idea 😃.

And in order to avoid contaminating the global namespace, what about doing something like import main as __main?

from pyscript.

The import trick appears to be a workaround that's already usable:
Instead of <script type="py" src="./main.py" ...>, you use src="./init.py", and in init.py you import main as __main. Raising an error at line 1 in main.py will then (correctly) show

Traceback (most recent call last):
  File "/lib/python311.zip/_pyodide/_base.py", line 501, in eval_code
    .run(globals, locals)
     ^^^^^^^^^^^^^^^^^^^^
  File "/lib/python311.zip/_pyodide/_base.py", line 339, in run
    coroutine = eval(self.code, globals, locals)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<exec>", line 45, in <module>
  File "/home/pyodide/main.py", line 1, in <module>
    raise RuntimeError('raised at main.py:1')
RuntimeError: raised at main.py:1

Note that the filename is also pretty much correct now, albeit with /home/pyodide prepended to it.

For the full code, see:
https://pyscript.com/@jorenham/bug-traceback-line-numbers-workaround/v3

from pyscript.

@WebReflection can we write inside try block and deal with it?
https://_12.pyscriptapps.com/calm-butterfly/latest/

from pyscript.

@nasrin1748 I am not sure I am following what you are saying ... you do want to know errors in your code, we're not going to shadow these and if you don't want to know, you can already put your code within a try block ... am I missing something?

from pyscript.

@nasrin1748 I am not sure I am following what you are saying ... you do want to know errors in your code, we're not going to shadow these and if you don't want to know, you can already put your code within a try block ... am I missing something?

File "/lib/python311.zip/_pyodide/_base.py", line 501, in eval_code .run(globals, locals)
^^^^^^^^^^^^^^^^^^^^
File "/lib/python311.zip/_pyodide/_base.py", line 339, in run coroutine = eval(self.code, globals, locals)

@WebReflection Just another doubt.I never read a .zip file directly.Is it possible to read the .zip file without unzipping?

from pyscript.

@nasrin1748 please use Discord for unrelated questions, otherwise it becomes very hard to follow up about this issue, thank you.

from pyscript.

@jorenham I've tackled this issue upstream in polyscript pyscript/polyscript#81

So far I couldn't notice any relevant performance degradation but I feel like the code now does the right thing, as explained in that MR.

I am planning to bring this to PyScript ASAP and eventually publish it as npm so you don't need to wait for a release, but it'd be very lovely if you could test that module once lands on npm and provide any meaningful feedback.

I think we're doing the right thing there by executing code in separate runs, so that the debugging experience would be actually 1:1 with users code, not the code we inject, nor the code hooks injects ... this also helps debugging broken hooks, specially the one appended after code execution, as the problem was obviously propagating in there too.

The final result is a slightly bigger base library but correctness and expectations should have higher priority than a couple of Kb out there ... I hope we all agree on this.

from pyscript.

@jorenham FYI if you use the module from npm things should be very different now:

• JS via https://cdn.jsdelivr.net/npm/@pyscript/[email protected]/dist/core.js
• CSS via https://cdn.jsdelivr.net/npm/@pyscript/[email protected]/dist/core.css

please let me know if you think that's OK and it works as expected, thank you.

from pyscript.

@WebReflection That seems to solve it, great!
https://pyscript.com/@jorenham/bug-traceback-line-numbers/v2

from pyscript.