Comments (7)
I like the pattern. It's flexible and clean to read. We'll want to document it well when we get to that step of the project so it's clear how to execute in either mode with isolated or shared kernels.
from testbook.
The following pattern seems promising and is fairly flexible 🎉
import pytest
from testbook import testbook
@pytest.fixture(scope='module')
def notebook():
# prerun cells here that need to be run just once across all tests in the scope
with testbook('/path/to/notebook.ipynb', prerun=['some', 'cells']) as tb:
yield tb
def test_1(notebook):
assert notebook.value("bar") == 'hello world'
notebook.inject("bar = 'something else'", run=True)
def test_2(notebook):
assert notebook.value("bar") == 'something else'
We will want to document this well, perhaps provide tutorials/examples as to how it can be used.
from testbook.
#19 brings this functionality in place as well.
We can let the user create a global instance of notebook_loader
, which can then be used across multiple tests - hence sharing the execution context.
Take a look at the following code snippet:
from testbook import notebook_loader
foo = notebook_loader('/path/to/notebook.ipynb', prerun=['tag1', tag2', tag3'])
@foo
def test_notebook_1(notebook):
assert notebook.cell_output_text('execute_foo') == 'foo\n'
@foo
def test_notebook_1(notebook):
assert notebook.cell_output_text('some_other_tag') == 'some other output\n'
or..
def test_notebook():
with foo():
assert notebook.cell_output_text('execute_foo') == 'foo\n'
def test_notebook_1():
with foo():
assert notebook.cell_output_text('some_other_tag') == 'some other output\n'
Of course, it can be named something much clearer than "foo"
UPDATE: The above implementation does not work, it does not preserve the kernel context across multiple tests. More info at #28
from testbook.
Is it possible to replicate the kernel context across multiple tests? For example, let us say that there is a specific set of cells which need to be executed prior to all the tests in a module. Is it possible to always start with the same fresh initial context in every test?
I would imagine that it would involve storing the kernel context somehow and "injecting" it into future kernels? Not sure.
cc @MSeal
from testbook.
Is it possible to always start with the same fresh initial context in every test?
Aren't we making that an option to the testbook decorator via the prerun
argument? It does require you decorate each test, so another option would be to follow the top answer from https://stackoverflow.com/questions/22627659/run-code-before-and-after-each-test-in-py-test and document (or provide?) a function for wrapping all tests in a scope with the notebook prep, maybe by generating a fixture within that scope, or more likely, by passing the tb object to each test?
Related: in ipython there's also %reset -f
which isn't a perfect solution but it would save a lot of time on kernel reboot times as an optimization for tests against that kernel.
from testbook.
I would imagine that it would involve storing the kernel context somehow and "injecting" it into future kernels? Not sure.
In process state transfers... not a reliable pattern. In some controlled contexts it's technically possible but I would not pursue that path for implementations.
from testbook.
Related: in ipython there's also %reset -f which isn't a perfect solution but it would save a lot of time on kernel reboot times as an optimization for tests against that kernel.
This is actually perfect for speeding up the tests that run on the same notebook. 💯
from testbook.
Related Issues (20)
- Getting an InterpreterNotFoundError on running the command tox -e py36 during setup HOT 1
- Document `nb` attribute and how it represents the nbformat object
- Entire notebook executed before the specific method HOT 3
- Testbook fails when using pytest test-classes HOT 2
- Support for objects as return_value when patching HOT 1
- Testbook giving timeout error because of nbclient HOT 1
- testbook loads wrong signature for function HOT 1
- Use pickle instead of JSON for serialization HOT 1
- support `async` function calls
- Asyncio errors and intermittent test failure
- Add support for SKIPPING cells' execution HOT 3
- Return value as None from testbook function HOT 4
- Mention in the contributing docs that an `ipykernel` with name `python3` must be present for tests to run locally
- Incompatibility between `execute=False` and `tb.patch()`
- testbook.ref cannot recognize complex variable
- `AttributeError: data` when repeatedly calling same function
- Because of project structure, relative paths in Notebook are no longer valid when testbook executes Notebook before tests HOT 1
- ModuleNotFoundError while using @testbook('....ipynb', execute=True) HOT 1
- Add type annotations
- Remove codecov dependency
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from testbook.