Add the testing from openpathsampling (opentis) and integrate into travis testi

This was added by <a class="issue-link js-issue-link" data-error-text="Failed to load

Ah. It works similar to nosetest. Just put the line <div class="snippet-clipboard-

Add ipython notebook testing about assaytools HOT 2 OPEN

choderalab commented on June 12, 2024

Add ipython notebook testing

from assaytools.

Comments (2)

jchodera commented on June 12, 2024

This was added by #4, but how do I add this to travis for testing?

from assaytools.

jhprinz commented on June 12, 2024

Ah. It works similar to nosetest. Just put the line

- python ipnbdoctest.py [options] xyz_notebook.ipynb

This will then start a kernel, run all cells and check if they pass. If you put the option --strict then also the output has to match. It takes care of changing memory addresses and UUIDs automatically.
If it passes the tool exits with 0 signaling travis that the test passed. Otherwise it exits with 1 and travis assumes it failed.

It's a standalone script that requires 'ipython-notebook' and

import os,sys
import base64
import re
import argparse

This is the usage syntax:

>>> ipython jan-hendrikprinz$ python ipnbdoctest.py -h
usage: ipnbdoctest.py [-h] [--timeout TIMEOUT] [--strict] [--fail-if-timeout]
                      file.ipynb

Run all cell in an ipython notebook as a test and check whether these
successfully execute and compares their output to the one inside the notebook

positional arguments:
  file.ipynb         the notebook to be checked

optional arguments:
  -h, --help         show this help message and exit
  --timeout TIMEOUT  the default timeout time in seconds for a cell
                     evaluation. Default is 300s.
  --strict           if set to true then the default test is that cell have to
                     match otherwise a diff will not be considered a failed
                     test
  --fail-if-timeout  if set to true then a timeout is considered a failed test

The output looks like this: