Comments (18)
As a workaround, I'm using --sanitize-with=workaround.txt
, where workaround.txt
contains
regex: (stream)+
replace: stream
from nbval.
I suspect this is related to the number of stream outputs produced by the cell varying (each stdout.flush produces a message. This number can depend on timing). nbval should be merging consecutive stream outputs into one (what we call coalesce_streams
in nbconvert), so that the number of messages doesn't affect the output to be compared.
from nbval.
It looks like my workaround isn't working: the tests are flaky. Does anyone have a better workaround?
from nbval.
@minrk: Will stream outputs be coalesced in the saved notebook's output, or does nbval need to process those as well?
from nbval.
I guess we might also have to handle \r
. Should all streams with the same name be combined, or only subsequent ones? Are there any other gotchas? Is there a library call for handling these things (should there be)?
from nbval.
We should also handle backspace characters, according to this code: https://github.com/jupyter/notebook/blob/e77d7ebfb060d7ce45fb85e865a2c6d8e7635900/notebook/static/base/js/utils.js#L467. However, it seems nbconvert does not do this.
from nbval.
Is this notebook behavior going to be an issue?
from nbval.
To highlight this better, the outputs of this cell are not deterministic:
import time
import numpy as np
print('Bar!', file=sys.stderr)
print('Combo breaker!', file=sys.stdout)
time.sleep(np.random.rand())
print('Foo!', file=sys.stderr)
from nbval.
PS: The stream
vs streamstream
means that one cell had one stream, while the other cell had two. This can definitely be improved!
from nbval.
I think we can only combine consecutive stream outputs from the same stream, to make it more consistent.
I think we had discussed a kernel change so that printing on one stream would effectively flush the other, to make output order more deterministic.
from nbval.
@vidartf to answer your question, yes consecutive outputs from the same stream will be merged into one output in the live notebook and in nbconvert during nbconvert --execute
. The nbconvert implementation is here.
from nbval.
Fix suggested in #57.
from nbval.
Sorry, I thought #57 was the one we already merged.
from nbval.
Thanks for fixing this! Do you have an ETA on publishing the fix? (I'm still getting flaky errors from nbval 0.5
on pypi.)
from nbval.
@vidartf is there anything else you want to work on before we cut 0.6?
from nbval.
I think we should be OK. I actually meant to get a release during PyData, but forgot about it.
from nbval.
I have released 0.6. :-)
from nbval.
Thank you! 👍
from nbval.
Related Issues (20)
- Is it possible to mock an object globally for all notebooks, eg with conftest.py? HOT 1
- Test failure with matplotlib 3.4: to_png deprecated HOT 2
- Documentation about skipping certain output types do not work
- Pytest fails to run on a seemingly innocent unit test
- Cell numbers are off by 1 compared to what you see in the notebook
- New release please HOT 2
- Testing structural similarity rather than string identity HOT 2
- When using Sanitizer, Special Chars on notebook are not escaped
- new pytest version throws deprecation warning HOT 3
- 0.9.6: sphinx warnings
- 0.9.6: pytest is failing in few units HOT 3
- CI tests are failing HOT 1
- Many tests fail on FreeBSD HOT 1
- New release? HOT 1
- 0.10.0 release captures output even when ';' is present HOT 3
- javascript not supported? HOT 2
- 0.10.0: documentation build fails
- pytest 8.1.0 breaks nbval 0.10.0 HOT 1
- `--sanitize-with` option seems to be behaving weirdly with "newly computed (test) output"
- Parallel --dist loadscope issue HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from nbval.