joblib / pyreport Goto Github PK

View Code? Open in Web Editor NEW

57.0 57.0 19.0 224 KB

Generate reports out of python scripts

Home Page: http://gael-varoquaux.info/programming/pyreport-literate-programming-in-python.html

Python 96.70% Makefile 3.30%

pyreport's Introduction

The homepage of joblib with user documentation is located on:

https://joblib.readthedocs.io

Getting the latest code

To get the latest code using git, simply type:

git clone https://github.com/joblib/joblib.git

If you don't have git installed, you can download a zip of the latest code: https://github.com/joblib/joblib/archive/refs/heads/main.zip

Installing

You can use pip to install joblib:

pip install joblib

from any directory or:

python setup.py install

from the source directory.

Dependencies

Joblib has no mandatory dependencies besides Python (supported versions are 3.8+).
Joblib has an optional dependency on Numpy (at least version 1.6.1) for array manipulation.
Joblib includes its own vendored copy of loky for process management.
Joblib can efficiently dump and load numpy arrays but does not require numpy to be installed.
Joblib has an optional dependency on python-lz4 as a faster alternative to zlib and gzip for compressed serialization.
Joblib has an optional dependency on psutil to mitigate memory leaks in parallel worker processes.
Some examples require external dependencies such as pandas. See the instructions in the Building the docs section for details.

Workflow to contribute

To contribute to joblib, first create an account on github. Once this is done, fork the joblib repository to have your own repository, clone it using 'git clone' on the computers where you want to work. Make your changes in your clone, push them to your github account, test them on several computers, and when you are happy with them, send a pull request to the main repository.

Running the test suite

To run the test suite, you need the pytest (version >= 3) and coverage modules. Run the test suite using:

pytest joblib

from the root of the project.

Building the docs

To build the docs you need to have sphinx (>=1.4) and some dependencies installed:

pip install -U -r .readthedocs-requirements.txt

The docs can then be built with the following command:

make doc

The html docs are located in the doc/_build/html directory.

Making a source tarball

To create a source tarball, eg for packaging or distributing, run the following command:

python setup.py sdist

The tarball will be created in the dist directory. This command will compile the docs, and the resulting tarball can be installed with no extra dependencies than the Python standard library. You will need setuptool and sphinx.

Making a release and uploading it to PyPI

This command is only run by project manager, to make a release, and upload in to PyPI:

python setup.py sdist bdist_wheel
twine upload dist/*

Note that the documentation should automatically get updated at each git push. If that is not the case, try building th doc locally and resolve any doc build error (in particular when running the examples).

Updating the changelog

Changes are listed in the CHANGES.rst file. They must be manually updated but, the following git command may be used to generate the lines:

git log --abbrev-commit --date=short --no-merges --sparse

pyreport's People

Contributors

Stargazers

Watchers

Forkers

gaelvaroquaux ptitjano stenri jkitchin pombredanne thorstenkranz svenk jgillis cj-wright winterflower hoffmann rlagoy farhangithub27 afcarl fagan2888 naterich2

pyreport's Issues

I can't find the pyreport executable

I am on OSX, and installed pyreport 3 ways. I can see the module installed, but I can't find the pyreport executable script.

From github:
```
$ git clone git://github.com/joblib/pyreport.git
$ cd pyreport
$ sudo python install pyreport
```
My python is in /opt/local/bin, but there is no pyreport in /opt/local/bin or /opt/local/Library/Frameworks/Python.framework/Versions/2.7/bin/
Using MacPorts' easy_install
$ sudo /opt/local/bin/easy_install-2.7 pyreport
The pyreport script is nowhere to be found.
Using the system easy_install
$ /usr/bin/easy_install pyreport
$ ls /usr/bin/pyre*
ls: /usr/bin/pyre*: No such file or directory

How can I get the executable script?

Latex compilation fails when using utf8

With some UTF8 files, even when is specified as an utf8 file in the first two lines of code, LaTeX compilation may failed.

An ugly fix of this problem, is to use:
overrides = {'output_encoding': 'utf8', 'initial_header_level': 0} line 514 of main.py

PyReport Broken with Docutils 0.11

PyReport crashes with any python script. I think the PyReport import function cannot handle newer calls from Docutils. Using it with a pyreport symlink from $PATH (c.f. #6):

$ pyreport --version
pyreport 0.3.3
$ python -c "import docutils; print docutils.__version__"
0.11
$ cat > hello.py
print "Hello World" 
$ pyreport hello.py 
Running python script /.../hello.py:

Hello World

Outputing report to /.../hello.pdf
Traceback (most recent call last):
  File "/home/sven/bin/pyreport", line 1260, in <module>
    commandline_call()
  File "/home/sven/bin/pyreport", line 1200, in commandline_call
    main(pyfile, overrides=options)
  File "/home/sven/bin/pyreport", line 1252, in main
    output_list = shape_output_list(output_list, options)
  File "/home/sven/bin/pyreport", line 364, in shape_output_list
    output_list = map(check_rst_block, output_list)
  File "/home/sven/bin/pyreport", line 668, in check_rst_block
    publisher.set_components('standalone', 'restructuredtext', 'pseudoxml')
  File "/usr/lib/python2.7/dist-packages/docutils/core.py", line 93, in set_components
    self.set_reader(reader_name, self.parser, parser_name)
  File "/usr/lib/python2.7/dist-packages/docutils/core.py", line 82, in set_reader
    reader_class = readers.get_reader_class(reader_name)
  File "/usr/lib/python2.7/dist-packages/docutils/readers/__init__.py", line 110, in get_reader_class
    module = __import__(reader_name, globals(), locals(), level=1)
TypeError: __call__() got an unexpected keyword argument 'level'

No output is generated. The only way pyreport doesn't crash is using the -x option.

local modules not found when script is run with pyreport?

Hi,

great tool! But a major drawback for me is that it doesn't find local modules: I have a script named "exp123.py" and a file named "progress.py" that I import from "exp123.py". Now, pyreport complains:

ImportError: No module named progress

Is there a workaround or is that a limitation that isn't mentioned on the project page?

Feature request (with patch): recognize latex blocks in output

Hi Gael,

I often generate lots of output inside inner loops, and I would like to be able to format that nicely. The following patch recognizes blocks wrapped in \begin{latex} \end{latex} brackets in output chunks and passes that directly into the latex output. I've used it successfully for adding section headings, include TOC entries, and generated latex tables etc into the output.

Please consider this (or something like it) for a future version of pyreport.

Regards,
Bradley

*** pyreport-0.3.4c/pyreport/main.py    Sat Nov 14 08:49:10 2009
--- pyreport/main.py    Sun Nov  6 15:04:12 2011
***************
*** 668,674 ****
          elif block[0] == "textBlock":
              rst_text = self.textBlocktpl % (self.add_indent(block[1])) 
          elif block[0] == "outputBlock":
!             rst_text = self.outputBlocktpl % ((block[1]).replace("\n","\n    "))
              for figure_name in block[2]:
                  rst_text = re.sub("Here goes figure " + re.escape(figure_name),
                          self.figuretpl % (os.path.splitext(figure_name)[0]),
--- 668,685 ----
          elif block[0] == "textBlock":
              rst_text = self.textBlocktpl % (self.add_indent(block[1])) 
          elif block[0] == "outputBlock":
!             rst_text = ""
!             startescape = "\\begin{latex}"
!             endescape = "\\end{latex}"
!             while startescape in block[1]:
!                 start = block[1].index(startescape)
!                 end = block[1].index(endescape)
!                 if start > 0:
!                     rst_text += self.outputBlocktpl % ((block[1][0:start]).replace("\n","\n    "))
!                 rst_text += self.latexBlocktpl % (self.add_indent(block[1][(start+len(startescape)):end]))
!                 block[1] = block[1][(end+len(endescape)):]
!             if len(block[1]) > 0:
!                 rst_text += self.outputBlocktpl % ((block[1]).replace("\n","\n    "))
              for figure_name in block[2]:
                  rst_text = re.sub("Here goes figure " + re.escape(figure_name),
                          self.figuretpl % (os.path.splitext(figure_name)[0]),

Dependencies missing / pip installation broken

The dependency to docutils (https://pypi.python.org/pypi/docutils) is missing. It must be installed manually. This should be done by marking the it as dependency.

Furthermore, #6 marks that the pip installation procedure misses creating a binary/softlink to the executable script.

plots created in inner function

Hi Gael,

this seems like an insanely useful tool for the scipy community, so many thanks for publishing this. I've started to use it for some of my experiments and I have noticed that it does not seem to be possible to include plots in the report that were created in inner functions. Is this possible somehow, or can pyreport be extended to do this?

Thanks,
Chris

How to get the header like on the screenshots from the project page?

The results that pyreport generates differ from the screenshots on the project page, e.g.:

I do not get that header. I also want the name of the input file and the date of the report to be added. How can we achieve this?

Error in "image" directive when using procedures

When I run the snippet, I get the following error. Calling a print statement after pylab.show seems to cause an issue. I would appreciate any workarounds. Thanks!

#!/usr/bin/env python
from pylab import *

def foo1():
    print "hello"
    figure()
    plot([1,2,3])
    show()

def foo2():
    print "hello2"
    figure()
    plot([2,3,4])
    show()

def bar():
    foo1()
    foo2()

bar()

andrewchou:rnd andrewchou$ pyenv/bin/pyreport -t html src-scripts/factor-mom/test.py
Running python script /Users/andrewchou/dev/rnd/src-scripts/factor-mom/test.py:

hello
Here goes figure test_pyreport_0.png
hello2
Here goes figure test_pyreport_1.png

Outputing report to /Users/andrewchou/dev/rnd/src-scripts/factor-mom/test.html
<string>:50: (ERROR/3) Error in "image" directive:
no content permitted.

.. image:: test_pyreport_0.png

    hello2


Ran script in 0.46s