Code Monkey home page Code Monkey logo

pydeps's Introduction

pydeps

Documentation Status

image

image

Downloads

Python module dependency visualization.

This package is primarily intended to be used from the command line through the pydeps command.

Feature requests and bug reports:

Please report bugs and feature requests on GitHub at https://github.com/thebjorn/pydeps/issues

How to install

pip install pydeps

To create graphs with pydeps you also need to install Graphviz. Please follow the installation instructions provided in the Graphviz link (and make sure the dot command is on your path).

Usage

usage: pydeps [-h] [--debug] [--config FILE] [--no-config] [--version] [-L LOG]
              [--find-package] [-v] [-o file] [-T FORMAT] [--display PROGRAM]
              [--noshow] [--show-deps] [--show-raw-deps] [--deps-output DEPS_OUT]
              [--show-dot] [--dot-output DOT_OUT] [--nodot] [--no-output]
              [--show-cycles] [--debug-mf INT] [--noise-level INT]
              [--max-bacon INT] [--max-module-depth INT] [--pylib] [--pylib-all]
              [--include-missing] [-x PATTERN [PATTERN ...]]
              [-xx MODULE [MODULE ...]] [--only MODULE_PATH [MODULE_PATH ...]]
              [--externals] [--reverse] [--rankdir {TB,BT,LR,RL}] [--cluster]
              [--min-cluster-size INT] [--max-cluster-size INT]
              [--keep-target-cluster] [--collapse-target-cluster]
              [--rmprefix PREFIX [PREFIX ...]] [--start-color INT]
              fname
positional arguments:

fname filename

optional arguments:

-h, --help show this help message and exit --debug turn on all the show and verbose options (mainly for debugging pydeps itself) --config FILE specify config file --no-config disable processing of config files --version print pydeps version -L LOG, --log LOG set log-level to one of CRITICAL, ERROR, WARNING, INFO, DEBUG, NOTSET. --find-package tries to automatically find the name of the current package. -v, --verbose be more verbose (-vv, -vvv for more verbosity) -o file write output to 'file' -T FORMAT output format (svg|png) --display PROGRAM program to use to display the graph (png or svg file depending on the T parameter) --noshow, --no-show don't call external program to display graph --show-deps show output of dependency analysis --show-raw-deps show output of dependency analysis before removing skips --deps-output write output of dependency analysis to file (instead of screen) --show-dot show output of dot conversion --dot-output write dot code to file (instead of screen) --nodot, --no-dot skip dot conversion --no-output don't create .svg/.png file, implies --no-show (-t/-o will be ignored) --show-cycles show only import cycles --debug-mf INT set the ModuleFinder.debug flag to this value --noise-level INT exclude sources or sinks with degree greater than noise-level --max-bacon INT exclude nodes that are more than n hops away (default=2, 0 -> infinite) --max-module-depth INT coalesce deep modules to at most n levels --pylib include python std lib modules --pylib-all include python all std lib modules (incl. C modules) --include-missing include modules that are not installed (or can't be found on sys.path) --only MODULE_PATH only include modules that start with MODULE_PATH, multiple paths can be provided --externals create list of direct external dependencies --reverse draw arrows to (instead of from) imported modules --rankdir set the direction of the graph, legal values are TB (default, imported modules above importing modules), BT (opposite direction of TB), LR (left-to-right), and RL (right-to-left) --cluster draw external dependencies as separate clusters --min-cluster-size INT the minimum number of nodes a dependency must have before being clustered (default=0) --max-cluster-size INT the maximum number of nodes a dependency can have before the cluster is collapsed to a single node (default=0) --keep-target-cluster draw target module as a cluster --collapse-target-cluster collapse target module (this implies --cluster) --rmprefix PREFIX remove PREFIX from the displayed name of the nodes (multiple prefixes can be provided) -x PATTERN, --exclude PATTERN input files to skip (e.g. foo.*), multiple patterns can be provided --exclude-exact MODULE (shorthand -xx MODULE) same as --exclude, except requires the full match. -xx foo.bar will exclude foo.bar, but not foo.bar.blob

Note: if an option with a variable number of arguments (like -x) is provided before fname, separate the arguments from the filename with -- otherwise fname will be parsed as an argument of the option. Example: $ pydeps -x os sys -- pydeps.

You can of course also import pydeps from Python and use it as a library, look in tests/test_relative_imports.py for examples.

Example

This is the result of running pydeps on itself (pydeps pydeps):

image

(full disclosure: this is for an early version of pydeps)

Notes

pydeps finds imports by looking for import-opcodes in python bytecodes (think .pyc files). Therefore, only imported files will be found (ie. pydeps will not look at files in your directory that are not imported). Additionally, only files that can be found using the Python import machinery will be considered (ie. if a module is missing or not installed, it will not be included regardless if it is being imported). This can be modified by using the --include-missing flag.

Displaying the graph:

To display the resulting .svg or .png files, pydeps by default calls an appropriate opener for the platform, like xdg-open foo.svg.

This can be overridden with the --display PROGRAM option, where PROGRAM is an executable that can display the image file of the graph.

You can also export the name of such a viewer in either the PYDEPS_DISPLAY or BROWSER environment variable, which changes the default behaviour when --display is not used.

Configuration files

All options can also be set in a .pydeps file using .ini file syntax (parsable by ConfigParser). Command line options override options in the .pydeps file in the current directory, which again overrides options in the user's home directory (%USERPROFILE%\.pydeps on Windows and ${HOME}/.pydeps otherwise).

An example .pydeps file:

[pydeps]
max_bacon = 2
no_show = True
verbose = 0
pylib = False
exclude =
    os
    re
    sys
    collections
    __future__

pydeps will also look for configuration data in pyproject.toml (under [tool.pydeps]) and setup.cfg (under [pydeps]).

Bacon (Scoring)

pydeps also contains an ErdΕ‘s-like scoring function (a.k.a. Bacon number, from Six degrees of Kevin Bacon (http://en.wikipedia.org/wiki/Six_Degrees_of_Kevin_Bacon) that lets you filter out modules that are more than a given number of 'hops' away from the module you're interested in. This is useful for finding the interface a module has to the rest of the world.

To find pydeps' interface to the Python stdlib (less some very common modules).

shell> pydeps pydeps --show --max-bacon 2 --pylib -x os re types _* enum

image

--max-bacon 2 (the default) gives the modules that are at most 2 hops away, and modules that belong together have similar colors. Compare that to the output with the --max-bacon=0 (infinite) filter:

image

Import cycles

pydeps can detect and display cycles with the --show-cycles parameter. This will _only display the cycles, and for big libraries it is not a particularly fast operation. Given a folder with the following contents (this uses yaml to define a directory structure, like in the tests):

relimp:
    - __init__.py
    - a.py: |
        from . import b
    - b.py: |
        from . import a

pydeps relimp --show-cycles displays:

image

Clustering

Running pydeps pydeps --max-bacon=4 on version 1.8.0 of pydeps gives the following graph:

image

If you are not interested in the internal structure of external modules, you can add the --cluster flag, which will collapse external modules into folder-shaped objects:

shell> pydeps pydeps --max-bacon=4 --cluster

image

To see the internal structure _and delineate external modules, use the --max-cluster-size flag, which controls how many nodes can be in a cluster before it is collapsed to a folder icon:

shell> pydeps pydeps --max-bacon=4 --cluster --max-cluster-size=1000

image

or, using a smaller max-cluster-size:

shell> pydeps pydeps --max-bacon=4 --cluster --max-cluster-size=3

image

To remove clusters with too few nodes, use the --min-cluster-size flag:

shell> pydeps pydeps --max-bacon=4 --cluster --max-cluster-size=3 --min-cluster-size=2

image

In some situations it can be useful to draw the target module as a cluster:

shell> pydeps pydeps --max-bacon=4 --cluster --max-cluster-size=3 --min-cluster-size=2 --keep-target-cluster

image

..and since the cluster boxes include the module name, we can remove those prefixes:

shell> pydeps pydeps --max-bacon=4 --cluster --max-cluster-size=3 --min-cluster-size=2 --keep-target-cluster --rmprefix pydeps. stdlib_list.

image

Maximum module depth

For Python packages that have a module structure more than two levels deep, the graph can easily become overwhelmingly complex. Use the --max-module-depth=n flag to examine the internal dependencies of a package while limiting the module depth (private and testing-related modules are removed to further simplify the graph using -x ...):

shell> pydeps pandas --only pandas --max-module-depth=2 -x pandas._* pandas.test* pandas.conftest

image

Graph direction

The direction of the graph can be specified using the --rankdir flag.

Top to bottom (default):

shell> pydeps pydeps --rankdir TB

image

Bottom to top:

shell> pydeps pydeps --rankdir BT

image

Left to right:

shell> pydeps pydeps --rankdir LR

image

Right to left:

shell> pydeps pydeps --rankdir RL

image

Collapsing target package

When internal target package dependencies are unimportant, they can be collapsed using the --collapse-target-cluster flag. This option also implies --cluster:

shell> pydeps pydeps --collapse-target-cluster

image

Intermediate format

An attempt has been made to keep the intermediate formats readable, eg. the output from pydeps --show-deps .. looks like this:

...
"pydeps.mf27": {
    "imported_by": [
        "__main__",
        "pydeps.py2depgraph"
    ],
    "kind": "imp.PY_SOURCE",
    "name": "pydeps.mf27",
    "path": "pydeps\\mf27.py"
},
"pydeps.py2depgraph": {
    "imported_by": [
        "__main__",
        "pydeps.pydeps"
    ],
    "imports": [
        "pydeps.depgraph",
        "pydeps.mf27"
    ],
    "kind": "imp.PY_SOURCE",
    "name": "pydeps.py2depgraph",
    "path": "pydeps\\py2depgraph.py"
}, ...

Version history

Version 1.12.19 Thanks to wiguwbe for a PR that fixes an inconsistency with the --no-dot flag.

Version 1.12.13 Better docs for larger packages. See maximum_module_depth for an example. Thanks to sheromon for the PR.

Version 1.12.5 Pydeps can now read configuration data from pyproject.toml. Thanks to septatrix for pushing the idea and for countering my toml-rant with an informative argument.

Version 1.11.0 drop support for Python 3.6. Thanks to pawamoy for removing imports of the deprecated imp module. (Parts of it has been vendorized due to a Python bug, see the code for details.)

Version 1.10.1 Thanks to vector400 for a new option --rankdir which renders the graph in different directions.

Version 1.10.0 supports Python 3.10.

Version 1.9.15 Thanks to Pipeline Foundation for a very much improved CI pipeline, and a CD pipeline as well.

Version 1.9.14 Thanks to poneill for fixing a cryptic error message when run in a directory without an __init__.py file.

Version 1.9.13 Thanks to glumia and SimonBiggs for improving the documentation.

Version 1.9.10 no_show is now honored when placed in .pydeps file. Thanks to romain-dartigues for the PR.

Version 1.9.8 Fix for maximum recursion depth exceeded when using large frameworks (like sympy). Thanks to tanujkhattar for finding the fix and to balopat for reporting it.

Version 1.9.7 Check PYDEPS_DISPLAY and BROWSER for a program to open the graph, PR by jhermann

Version 1.9.1 graphs are now stable on Python 3.x as well -this was already the case for Py2.7 (thanks to pawamoy for reporting and testing the issue and to kinow for helping with testing).

Version 1.9.0 supports Python 3.8.

Version 1.8.7 includes a new flag --rmprefix which lets you remove prefixes from the node-labels in the graph. The _name of the nodes are not effected so this does not cause merging of nodes, nor does it change coloring - but it can lead to multiple nodes with the same label (hovering over the node will give the full name). Thanks to aroberge for the enhancement request.

Version 1.8.5 With svg as the output format (which is the default), paths are now hilighted on mouse hover (thanks to tomasito665 for the enhancement request).

Version 1.8.2 incldes a new flag --only that causes pydeps to only report on the paths specified:

shell> pydeps mypackage --only mypackage.a mypackage.b

Version 1.8.0 includes 4 new flags for drawing external dependencies as clusters. See clustering for examples. Additionally, the arrowheads now have the color of the source node.

Version 1.7.3 includes a new flag -xx or --exclude-exact which matches the functionality of the --exclude flag, except it requires an exact match, i.e. -xx foo.bar will exclude foo.bar, but not foo.bar.blob (thanks to AvenzaOleg for the PR).

Version 1.7.2 includes a new flag, --no-output, which prevents creation of the .svg/.png file.

Version 1.7.1 fixes excludes in .pydeps files (thanks to eqvis for the bug report).

Version 1.7.0 The new --reverse flag reverses the direction of the arrows in the dependency graph, so they point _to the imported module instead of _from the imported module (thanks to goetzk for the bug report and tobiasmaier for the PR!).

Version 1.5.0 Python 3 support (thanks to eight04 for the PR).

Version 1.3.4 --externals will now include modules that haven't been installed (what modulefinder calls badmodules).

Version 1.2.8 A shortcut for finding the direct external dependencies of a package was added:

pydeps --externals mypackage

which will print a json formatted list of module names to the screen, e.g.:

(dev) go|c:\srv\lib\dk-tasklib> pydeps --externals dktasklib
[
    "dkfileutils"
]

which means that the dktasklib package only depends on the dkfileutils package.

This functionality is also available programmatically:

import os
from pydeps.pydeps import externals
# the directory that contains setup.py (one level up from actual package):
os.chdir('package-directory')
print externals('mypackage')

Version 1.2.5: The defaults are now sensible, such that:

shell> pydeps mypackage

will likely do what you want. It is the same as pydeps --show --max-bacon=2 mypackage which means display the dependency graph in your browser, but limit it to two hops (which includes only the modules that your module imports -- not continuing down the import chain). The old default behavior is available with pydeps --noshow --max-bacon=0 mypackage.

Contributing

  1. Fork it
  2. It is appreciated (but not required) if you raise an issue first: https://github.com/thebjorn/pydeps/issues
  3. Create your feature branch (git checkout -b my-new-feature)
  4. Commit your changes (git commit -am 'Add some feature')
  5. Push to the branch (git push origin my-new-feature)
  6. Create new Pull Request

pydeps's People

Contributors

aleks-ivanov avatar aroberge avatar avenzaoleg avatar dependabot[bot] avatar eight04 avatar everwinter23 avatar ewen-lbh avatar jhermann avatar joshkel avatar mchlnix avatar miketheman avatar mlga avatar pawamoy avatar romain-dartigues avatar septatrix avatar sheromon avatar spagh-eddie avatar thebjorn avatar theotherbjorn avatar wiguwbe avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

pydeps's Issues

Pydeps mydir not detecting imports

Hello,

So I have the following directory:

Directory
L__init__.py
L database.py
L controller.py
L exceptions.py
L models.py
L mainclass.py

If I run pydeps directory, it only seems to detect the database.py file, and then traces the sqlalchemy dependencies in that file. I'm importing all of these files in each other, so there should be a graph of those imports. I saw your example in your documentation, how were you able to trace imports of multiple .py files in the same directory? for example, pydeps.colors and pydeps.depgraph?

RecurrsionError for pydeps on sympy

Hi, I was trying to give pydeps a try on a project which has sympy as a dependency and hit an error: RecursionError: maximum recursion depth exceeded while calling a Python object. I don't know if this is a bug in pydeps or sympy, but figured I'd report it here in case its something pydeps can handle. The error occurs with the default args, and with --show-cycles.

If there's any more debug info I can provide, let me know. Thanks!

$ mkdir pdeptest
$ python3.7 -m venv pdeptest/
$ cd pdeptest/ && source bin/activate
(pdeptest) $ pip install pydeps
Collecting pydeps
... <snip> ...
Successfully installed pydeps-1.7.3 stdlib-list-0.5.0
(pdeptest) $ pip install sympy
Collecting sympy
... <snip> ...
Successfully installed mpmath-1.1.0 sympy-1.4
(pdeptest) $ echo 'import sympy' > test.py
(pdeptest) $ pydeps test.py
Traceback (most recent call last):
  File "/Users/kevin.krsulichibm.com/pdeptest/bin/pydeps", line 11, in <module>
    load_entry_point('pydeps==1.7.3', 'console_scripts', 'pydeps')()
  File "/Users/kevin.krsulichibm.com/pdeptest/lib/python3.7/site-packages/pydeps/pydeps.py", line 131, in pydeps
    return _pydeps(inp, **_args)
  File "/Users/kevin.krsulichibm.com/pdeptest/lib/python3.7/site-packages/pydeps/pydeps.py", line 30, in _pydeps
    dep_graph = py2depgraph.py2dep(trgt, **kw)
  File "/Users/kevin.krsulichibm.com/pdeptest/lib/python3.7/site-packages/pydeps/py2depgraph.py", line 214, in py2dep
    mf.run_script(dummy.fname)
  File "/Users/kevin.krsulichibm.com/pdeptest/lib/python3.7/site-packages/pydeps/mf27.py", line 117, in run_script
    self.load_module('__main__', fp, pathname, stuff)
  File "/Users/kevin.krsulichibm.com/pdeptest/lib/python3.7/site-packages/pydeps/py2depgraph.py", line 146, in load_module
    self, fqname, fp, pathname, (suffix, mode, kind)
  File "/Users/kevin.krsulichibm.com/pdeptest/lib/python3.7/site-packages/pydeps/mf27.py", line 335, in load_module
    self.scan_code(co, module)
  File "/Users/kevin.krsulichibm.com/pdeptest/lib/python3.7/site-packages/pydeps/mf27.py", line 472, in scan_code
    self._safe_import_hook(name, module, fromlist, level=level)
  File "/Users/kevin.krsulichibm.com/pdeptest/lib/python3.7/site-packages/pydeps/mf27.py", line 353, in _safe_import_hook
    self.import_hook(name, caller, level=level)
  File "/Users/kevin.krsulichibm.com/pdeptest/lib/python3.7/site-packages/pydeps/py2depgraph.py", line 116, in import_hook
    return mf27.ModuleFinder.import_hook(self, name, caller, fromlist, level)
  File "/Users/kevin.krsulichibm.com/pdeptest/lib/python3.7/site-packages/pydeps/mf27.py", line 130, in import_hook
    q, tail = self.find_head_package(parent, name)

... <snip> ...

  File "/Users/kevin.krsulichibm.com/pdeptest/lib/python3.7/site-packages/pydeps/py2depgraph.py", line 146, in load_module
    self, fqname, fp, pathname, (suffix, mode, kind)
  File "/Users/kevin.krsulichibm.com/pdeptest/lib/python3.7/site-packages/pydeps/mf27.py", line 335, in load_module
    self.scan_code(co, module)
  File "/Users/kevin.krsulichibm.com/pdeptest/lib/python3.7/site-packages/pydeps/mf27.py", line 456, in scan_code
    for what, args in scanner(co):
  File "/Users/kevin.krsulichibm.com/pdeptest/lib/python3.7/site-packages/pydeps/mf27.py", line 431, in scan_opcodes_34
    bytecode = list(dis.Bytecode(co))
  File "/usr/local/Cellar/python/3.7.4/Frameworks/Python.framework/Versions/3.7/lib/python3.7/dis.py", line 478, in __init__
    self._linestarts = dict(findlinestarts(co))
  File "/usr/local/Cellar/python/3.7.4/Frameworks/Python.framework/Versions/3.7/lib/python3.7/dis.py", line 448, in findlinestarts
    for byte_incr, line_incr in zip(byte_increments, line_increments):
RecursionError: maximum recursion depth exceeded while calling a Python object

Make connection lines highlightable

Hi there! First of all, thanks for creating this package. It's awesome.

For large projects, though, the graph can get a bit chaotic. It would be very helpful to be able to click on a line in the browser and make it highlight so that we can see where it connects to (e.g., as PyCharm does with DB UML visualisations). To illustrate the problem, bellow follows a screenshot of the pydeps output for a project I'm working on.

image

Thanks!

Draw module internal dependencies

Hi,

my question somewhat relates to issue #25.

First, I encounter the same issue as outlined in that ticket. It got closed without addressing the issue unfortunately. The workaround mentioned there correctly works in the way that the python files become part of the output.

But the workaround does not work in case a modules file imports another file from the same module.

Given

mydir
|   `-- __init__.py
|   `-- controller.py
|   `-- database.py
|   `-- exceptions.py
|   `-- mainclass.py
    `-- models.py

and mydir.controller imports and uses mydir.database, then this dependency is not available in the graph.

Is there any way to make this dependency visible? I would like to use that information as it would ease refactoring / organizing code as it would greatly make internal oddities visible.

Kind regards
alex

test failed.

Hi @thebjorn

Can you help to check this issue?

Thanks!

============================= test session starts ==============================
platform linux -- Python 3.8.2, pytest-4.6.9, py-1.8.0, pluggy-0.13.0
rootdir: /builddir/build/BUILD/pydeps-1.9.0
collected 37 items
tests/test_cli.py F [ 2%]
tests/test_colors.py ..... [ 16%]
tests/test_cycles.py . [ 18%]
tests/test_dep2dot.py F [ 21%]
tests/test_dot.py FFFF. [ 35%]
tests/test_externals.py . [ 37%]
tests/test_file.py ... [ 45%]
tests/test_funny_names.py . [ 48%]
tests/test_json.py . [ 51%]
tests/test_py2dep.py . [ 54%]
tests/test_relative_imports.py ....... [ 72%]
tests/test_render_context.py ... [ 81%]
tests/test_skinny_package.py . [ 83%]
tests/test_skip.py ...... [100%]
=================================== FAILURES ===================================
_________________________________ test_output __________________________________
src = '\ndigraph G {\n concentrate = true;\n\n rankdir = TB;\n node [style=filled,fillcolor="#ffffff",fontcolor="#0...o_a [weight="4",minlen="2",fillcolor="#b65353"];\n bar_b -> foo_a [weight="4",minlen="2",fillcolor="#b65353"];\n}\n'
fmt = 'svg'
def call_graphviz_dot(src, fmt):
"""Call dot command, and provide helpful error message if we
cannot find it.
"""
try:

      svg = dot(src, T=fmt)

pydeps/dot.py:75:


src = '\ndigraph G {\n concentrate = true;\n\n rankdir = TB;\n node [style=filled,fillcolor="#ffffff",fontcolor="#0...o_a [weight="4",minlen="2",fillcolor="#b65353"];\n bar_b -> foo_a [weight="4",minlen="2",fillcolor="#b65353"];\n}\n'
kw = {}, cmd = 'dot -Tsvg'
def dot(src, **kw):
"""Execute the dot command to create an svg output.
"""
cmd = "dot -T%s" % kw.pop('T', 'svg')
for k, v in list(kw.items()):
if v is True:
cmd += " -%s" % k
else:
cmd += " -%s%s" % (k, v)

  return pipe(cmd, to_bytes(src))

pydeps/dot.py:67:


cmd = 'dot -Tsvg'
txt = b'\ndigraph G {\n concentrate = true;\n\n rankdir = TB;\n node [style=filled,fillcolor="#ffffff",fontcolor="#...o_a [weight="4",minlen="2",fillcolor="#b65353"];\n bar_b -> foo_a [weight="4",minlen="2",fillcolor="#b65353"];\n}\n'
def pipe(cmd, txt):
"""Pipe txt into the command cmd and return the output.
"""

  return Popen(
        cmd2args(cmd),
        stdout=subprocess.PIPE,
        stdin=subprocess.PIPE,
        shell=win32
    ).communicate(txt)[0]

pydeps/dot.py:49:


self = <subprocess.Popen object at 0xb56d7fb8>, args = ['dot', '-Tsvg']
bufsize = -1, executable = None, stdin = -1, stdout = -1, stderr = None
preexec_fn = None, close_fds = True, shell = False, cwd = None, env = None
universal_newlines = None, startupinfo = None, creationflags = 0
restore_signals = True, start_new_session = False, pass_fds = ()
def init(self, args, bufsize=-1, executable=None,
stdin=None, stdout=None, stderr=None,
preexec_fn=None, close_fds=True,
shell=False, cwd=None, env=None, universal_newlines=None,
startupinfo=None, creationflags=0,
restore_signals=True, start_new_session=False,
pass_fds=(), *, encoding=None, errors=None, text=None):
"""Create new Popen instance."""
_cleanup()
# Held while anything is calling waitpid before returncode has been
# updated to prevent clobbering returncode if wait() or poll() are
# called from multiple threads at once. After acquiring the lock,
# code must re-check self.returncode to see if another thread just
# finished a waitpid() call.
self._waitpid_lock = threading.Lock()

    self._input = None
    self._communication_started = False
    if bufsize is None:
        bufsize = -1  # Restore default
    if not isinstance(bufsize, int):
        raise TypeError("bufsize must be an integer")

    if _mswindows:
        if preexec_fn is not None:
            raise ValueError("preexec_fn is not supported on Windows "
                             "platforms")
    else:
        # POSIX
        if pass_fds and not close_fds:
            warnings.warn("pass_fds overriding close_fds.", RuntimeWarning)
            close_fds = True
        if startupinfo is not None:
            raise ValueError("startupinfo is only supported on Windows "
                             "platforms")
        if creationflags != 0:
            raise ValueError("creationflags is only supported on Windows "
                             "platforms")

    self.args = args
    self.stdin = None
    self.stdout = None
    self.stderr = None
    self.pid = None
    self.returncode = None
    self.encoding = encoding
    self.errors = errors

    # Validate the combinations of text and universal_newlines
    if (text is not None and universal_newlines is not None
        and bool(universal_newlines) != bool(text)):
        raise SubprocessError('Cannot disambiguate when both text '
                              'and universal_newlines are supplied but '
                              'different. Pass one or the other.')

    # Input and output objects. The general principle is like
    # this:
    #
    # Parent                   Child
    # ------                   -----
    # p2cwrite   ---stdin--->  p2cread
    # c2pread    <--stdout---  c2pwrite
    # errread    <--stderr---  errwrite
    #
    # On POSIX, the child objects are file descriptors.  On
    # Windows, these are Windows file handles.  The parent objects
    # are file descriptors on both platforms.  The parent objects
    # are -1 when not using PIPEs. The child objects are -1
    # when not redirecting.

    (p2cread, p2cwrite,
     c2pread, c2pwrite,
     errread, errwrite) = self._get_handles(stdin, stdout, stderr)

    # We wrap OS handles *before* launching the child, otherwise a
    # quickly terminating child could make our fds unwrappable
    # (see #8458).

    if _mswindows:
        if p2cwrite != -1:
            p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0)
        if c2pread != -1:
            c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0)
        if errread != -1:
            errread = msvcrt.open_osfhandle(errread.Detach(), 0)

    self.text_mode = encoding or errors or text or universal_newlines

    # How long to resume waiting on a child after the first ^C.
    # There is no right value for this.  The purpose is to be polite
    # yet remain good for interactive users trying to exit a tool.
    self._sigint_wait_secs = 0.25  # 1/xkcd221.getRandomNumber()

    self._closed_child_pipe_fds = False

    if self.text_mode:
        if bufsize == 1:
            line_buffering = True
            # Use the default buffer size for the underlying binary streams
            # since they don't support line buffering.
            bufsize = -1
        else:
            line_buffering = False

    try:
        if p2cwrite != -1:
            self.stdin = io.open(p2cwrite, 'wb', bufsize)
            if self.text_mode:
                self.stdin = io.TextIOWrapper(self.stdin, write_through=True,
                        line_buffering=line_buffering,
                        encoding=encoding, errors=errors)
        if c2pread != -1:
            self.stdout = io.open(c2pread, 'rb', bufsize)
            if self.text_mode:
                self.stdout = io.TextIOWrapper(self.stdout,
                        encoding=encoding, errors=errors)
        if errread != -1:
            self.stderr = io.open(errread, 'rb', bufsize)
            if self.text_mode:
                self.stderr = io.TextIOWrapper(self.stderr,
                        encoding=encoding, errors=errors)
      self._execute_child(args, executable, preexec_fn, close_fds,
                            pass_fds, cwd, env,
                            startupinfo, creationflags, shell,
                            p2cread, p2cwrite,
                            c2pread, c2pwrite,
                            errread, errwrite,
                            restore_signals, start_new_session)

/usr/lib/python3.8/subprocess.py:854:


self = <subprocess.Popen object at 0xb56d7fb8>, args = ['dot', '-Tsvg']
executable = b'dot', preexec_fn = None, close_fds = True, pass_fds = ()
cwd = None, env = None, startupinfo = None, creationflags = 0, shell = False
p2cread = 8, p2cwrite = 10, c2pread = 11, c2pwrite = 12, errread = -1
errwrite = -1, restore_signals = True, start_new_session = False
def _execute_child(self, args, executable, preexec_fn, close_fds,
pass_fds, cwd, env,
startupinfo, creationflags, shell,
p2cread, p2cwrite,
c2pread, c2pwrite,
errread, errwrite,
restore_signals, start_new_session):
"""Execute program (POSIX version)"""

    if isinstance(args, (str, bytes)):
        args = [args]
    elif isinstance(args, os.PathLike):
        if shell:
            raise TypeError('path-like args is not allowed when '
                            'shell is true')
        args = [args]
    else:
        args = list(args)

    if shell:
        # On Android the default shell is at '/system/bin/sh'.
        unix_shell = ('/system/bin/sh' if
                  hasattr(sys, 'getandroidapilevel') else '/bin/sh')
        args = [unix_shell, "-c"] + args
        if executable:
            args[0] = executable

    if executable is None:
        executable = args[0]

    sys.audit("subprocess.Popen", executable, args, cwd, env)

    if (_USE_POSIX_SPAWN
            and os.path.dirname(executable)
            and preexec_fn is None
            and not close_fds
            and not pass_fds
            and cwd is None
            and (p2cread == -1 or p2cread > 2)
            and (c2pwrite == -1 or c2pwrite > 2)
            and (errwrite == -1 or errwrite > 2)
            and not start_new_session):
        self._posix_spawn(args, executable, env, restore_signals,
                          p2cread, p2cwrite,
                          c2pread, c2pwrite,
                          errread, errwrite)
        return

    orig_executable = executable

    # For transferring possible exec failure from child to parent.
    # Data format: "exception name:hex errno:description"
    # Pickle is not used; it is complex and involves memory allocation.
    errpipe_read, errpipe_write = os.pipe()
    # errpipe_write must not be in the standard io 0, 1, or 2 fd range.
    low_fds_to_close = []
    while errpipe_write < 3:
        low_fds_to_close.append(errpipe_write)
        errpipe_write = os.dup(errpipe_write)
    for low_fd in low_fds_to_close:
        os.close(low_fd)
    try:
        try:
            # We must avoid complex work that could involve
            # malloc or free in the child process to avoid
            # potential deadlocks, thus we do all this here.
            # and pass it to fork_exec()

            if env is not None:
                env_list = []
                for k, v in env.items():
                    k = os.fsencode(k)
                    if b'=' in k:
                        raise ValueError("illegal environment variable name")
                    env_list.append(k + b'=' + os.fsencode(v))
            else:
                env_list = None  # Use execv instead of execve.
            executable = os.fsencode(executable)
            if os.path.dirname(executable):
                executable_list = (executable,)
            else:
                # This matches the behavior of os._execvpe().
                executable_list = tuple(
                    os.path.join(os.fsencode(dir), executable)
                    for dir in os.get_exec_path(env))
            fds_to_keep = set(pass_fds)
            fds_to_keep.add(errpipe_write)
            self.pid = _posixsubprocess.fork_exec(
                    args, executable_list,
                    close_fds, tuple(sorted(map(int, fds_to_keep))),
                    cwd, env_list,
                    p2cread, p2cwrite, c2pread, c2pwrite,
                    errread, errwrite,
                    errpipe_read, errpipe_write,
                    restore_signals, start_new_session, preexec_fn)
            self._child_created = True
        finally:
            # be sure the FD is closed no matter what
            os.close(errpipe_write)

        self._close_pipe_fds(p2cread, p2cwrite,
                             c2pread, c2pwrite,
                             errread, errwrite)

        # Wait for exec to fail or succeed; possibly raising an
        # exception (limited in size)
        errpipe_data = bytearray()
        while True:
            part = os.read(errpipe_read, 50000)
            errpipe_data += part
            if not part or len(errpipe_data) > 50000:
                break
    finally:
        # be sure the FD is closed no matter what
        os.close(errpipe_read)

    if errpipe_data:
        try:
            pid, sts = os.waitpid(self.pid, 0)
            if pid == self.pid:
                self._handle_exitstatus(sts)
            else:
                self.returncode = sys.maxsize
        except ChildProcessError:
            pass

        try:
            exception_name, hex_errno, err_msg = (
                    errpipe_data.split(b':', 2))
            # The encoding here should match the encoding
            # written in by the subprocess implementations
            # like _posixsubprocess
            err_msg = err_msg.decode()
        except ValueError:
            exception_name = b'SubprocessError'
            hex_errno = b'0'
            err_msg = 'Bad exception data from child: {!r}'.format(
                          bytes(errpipe_data))
        child_exception_type = getattr(
                builtins, exception_name.decode('ascii'),
                SubprocessError)
        if issubclass(child_exception_type, OSError) and hex_errno:
            errno_num = int(hex_errno, 16)
            child_exec_never_called = (err_msg == "noexec")
            if child_exec_never_called:
                err_msg = ""
                # The error must be from chdir(cwd).
                err_filename = cwd
            else:
                err_filename = orig_executable
            if errno_num != 0:
                err_msg = os.strerror(errno_num)
          raise child_exception_type(errno_num, err_msg, err_filename)

E FileNotFoundError: [Errno 2] No such file or directory: 'dot'
/usr/lib/python3.8/subprocess.py:1702: FileNotFoundError
During handling of the above exception, another exception occurred:
tmpdir = local('/tmp/pytest-of-mockbuild/pytest-0/test_output0')
def test_output(tmpdir):
files = """
unrelated: []
foo:
- init.py
- a.py: |
from bar import b
bar:
- init.py
- b.py
"""
with create_files(files) as workdir:
assert os.getcwd() == workdir

        outname = os.path.join('unrelated', 'foo.svg')
        assert not os.path.exists(outname)
      pydeps(fname='foo', **empty('--noshow', output=outname))

tests/test_cli.py:25:


pydeps/pydeps.py:136: in pydeps
return _pydeps(inp, **_args)
pydeps/pydeps.py:46: in _pydeps
svg = dot.call_graphviz_dot(dotsrc, fmt)
pydeps/dot.py:78: in call_graphviz_dot
cli.error("""


args = ("\n cannot find 'dot'\n\n pydeps calls dot (from graphviz) to create svg diagrams,\n please make sure that the dot executable is available\n on your path.\n ",)
kwargs = {'file': <_pytest.capture.EncodedFile object at 0xb56fffd0>}
def error(*args, **kwargs): # pragma: nocover
"""Print an error message and exit.
"""
kwargs['file'] = sys.stderr
print("\n\tERROR:", *args, **kwargs)

  sys.exit(1)

E SystemExit: 1
pydeps/cli.py:23: SystemExit
----------------------------- Captured stderr call -----------------------------
ERROR:
cannot find 'dot'
pydeps calls dot (from graphviz) to create svg diagrams,
please make sure that the dot executable is available
on your path.

_________________________________ test_dep2dot _________________________________
src = '\ndigraph G {\n concentrate = true;\n\n rankdir = TB;\n node [style=filled,fillcolor="#ffffff",fontcolor="#0..._b [label="foo.b",fillcolor="#c24747",fontcolor="#ffffff"];\n foo_b -> foo_a [weight="5",fillcolor="#c24747"];\n}\n'
fmt = 'svg'
def call_graphviz_dot(src, fmt):
"""Call dot command, and provide helpful error message if we
cannot find it.
"""
try:

      svg = dot(src, T=fmt)

pydeps/dot.py:75:


src = '\ndigraph G {\n concentrate = true;\n\n rankdir = TB;\n node [style=filled,fillcolor="#ffffff",fontcolor="#0..._b [label="foo.b",fillcolor="#c24747",fontcolor="#ffffff"];\n foo_b -> foo_a [weight="5",fillcolor="#c24747"];\n}\n'
kw = {}, cmd = 'dot -Tsvg'
def dot(src, **kw):
"""Execute the dot command to create an svg output.
"""
cmd = "dot -T%s" % kw.pop('T', 'svg')
for k, v in list(kw.items()):
if v is True:
cmd += " -%s" % k
else:
cmd += " -%s%s" % (k, v)

  return pipe(cmd, to_bytes(src))

pydeps/dot.py:67:


cmd = 'dot -Tsvg'
txt = b'\ndigraph G {\n concentrate = true;\n\n rankdir = TB;\n node [style=filled,fillcolor="#ffffff",fontcolor="#..._b [label="foo.b",fillcolor="#c24747",fontcolor="#ffffff"];\n foo_b -> foo_a [weight="5",fillcolor="#c24747"];\n}\n'
def pipe(cmd, txt):
"""Pipe txt into the command cmd and return the output.
"""

  return Popen(
        cmd2args(cmd),
        stdout=subprocess.PIPE,
        stdin=subprocess.PIPE,
        shell=win32
    ).communicate(txt)[0]

pydeps/dot.py:49:


self = <subprocess.Popen object at 0xb5ca73d0>, args = ['dot', '-Tsvg']
bufsize = -1, executable = None, stdin = -1, stdout = -1, stderr = None
preexec_fn = None, close_fds = True, shell = False, cwd = None, env = None
universal_newlines = None, startupinfo = None, creationflags = 0
restore_signals = True, start_new_session = False, pass_fds = ()
def init(self, args, bufsize=-1, executable=None,
stdin=None, stdout=None, stderr=None,
preexec_fn=None, close_fds=True,
shell=False, cwd=None, env=None, universal_newlines=None,
startupinfo=None, creationflags=0,
restore_signals=True, start_new_session=False,
pass_fds=(), *, encoding=None, errors=None, text=None):
"""Create new Popen instance."""
_cleanup()
# Held while anything is calling waitpid before returncode has been
# updated to prevent clobbering returncode if wait() or poll() are
# called from multiple threads at once. After acquiring the lock,
# code must re-check self.returncode to see if another thread just
# finished a waitpid() call.
self._waitpid_lock = threading.Lock()

    self._input = None
    self._communication_started = False
    if bufsize is None:
        bufsize = -1  # Restore default
    if not isinstance(bufsize, int):
        raise TypeError("bufsize must be an integer")

    if _mswindows:
        if preexec_fn is not None:
            raise ValueError("preexec_fn is not supported on Windows "
                             "platforms")
    else:
        # POSIX
        if pass_fds and not close_fds:
            warnings.warn("pass_fds overriding close_fds.", RuntimeWarning)
            close_fds = True
        if startupinfo is not None:
            raise ValueError("startupinfo is only supported on Windows "
                             "platforms")
        if creationflags != 0:
            raise ValueError("creationflags is only supported on Windows "
                             "platforms")

    self.args = args
    self.stdin = None
    self.stdout = None
    self.stderr = None
    self.pid = None
    self.returncode = None
    self.encoding = encoding
    self.errors = errors

    # Validate the combinations of text and universal_newlines
    if (text is not None and universal_newlines is not None
        and bool(universal_newlines) != bool(text)):
        raise SubprocessError('Cannot disambiguate when both text '
                              'and universal_newlines are supplied but '
                              'different. Pass one or the other.')

    # Input and output objects. The general principle is like
    # this:
    #
    # Parent                   Child
    # ------                   -----
    # p2cwrite   ---stdin--->  p2cread
    # c2pread    <--stdout---  c2pwrite
    # errread    <--stderr---  errwrite
    #
    # On POSIX, the child objects are file descriptors.  On
    # Windows, these are Windows file handles.  The parent objects
    # are file descriptors on both platforms.  The parent objects
    # are -1 when not using PIPEs. The child objects are -1
    # when not redirecting.

    (p2cread, p2cwrite,
     c2pread, c2pwrite,
     errread, errwrite) = self._get_handles(stdin, stdout, stderr)

    # We wrap OS handles *before* launching the child, otherwise a
    # quickly terminating child could make our fds unwrappable
    # (see #8458).

    if _mswindows:
        if p2cwrite != -1:
            p2cwrite = msvcrt.open_osfhandle(p2cwrite.Detach(), 0)
        if c2pread != -1:
            c2pread = msvcrt.open_osfhandle(c2pread.Detach(), 0)
        if errread != -1:
            errread = msvcrt.open_osfhandle(errread.Detach(), 0)

    self.text_mode = encoding or errors or text or universal_newlines

    # How long to resume waiting on a child after the first ^C.
    # There is no right value for this.  The purpose is to be polite
    # yet remain good for interactive users trying to exit a tool.
    self._sigint_wait_secs = 0.25  # 1/xkcd221.getRandomNumber()

    self._closed_child_pipe_fds = False

    if self.text_mode:
        if bufsize == 1:
            line_buffering = True
            # Use the default buffer size for the underlying binary streams
            # since they don't support line buffering.
            bufsize = -1
        else:
            line_buffering = False

    try:
        if p2cwrite != -1:
            self.stdin = io.open(p2cwrite, 'wb', bufsize)
            if self.text_mode:
                self.stdin = io.TextIOWrapper(self.stdin, write_through=True,
                        line_buffering=line_buffering,
                        encoding=encoding, errors=errors)
        if c2pread != -1:
            self.stdout = io.open(c2pread, 'rb', bufsize)
            if self.text_mode:
                self.stdout = io.TextIOWrapper(self.stdout,
                        encoding=encoding, errors=errors)
        if errread != -1:
            self.stderr = io.open(errread, 'rb', bufsize)
            if self.text_mode:
                self.stderr = io.TextIOWrapper(self.stderr,
                        encoding=encoding, errors=errors)
      self._execute_child(args, executable, preexec_fn, close_fds,
                            pass_fds, cwd, env,
                            startupinfo, creationflags, shell,
                            p2cread, p2cwrite,
                            c2pread, c2pwrite,
                            errread, errwrite,
                            restore_signals, start_new_session)

/usr/lib/python3.8/subprocess.py:854:


self = <subprocess.Popen object at 0xb5ca73d0>, args = ['dot', '-Tsvg']
executable = b'dot', preexec_fn = None, close_fds = True, pass_fds = ()
cwd = None, env = None, startupinfo = None, creationflags = 0, shell = False
p2cread = 8, p2cwrite = 10, c2pread = 11, c2pwrite = 12, errread = -1
errwrite = -1, restore_signals = True, start_new_session = False
def _execute_child(self, args, executable, preexec_fn, close_fds,
pass_fds, cwd, env,
startupinfo, creationflags, shell,
p2cread, p2cwrite,
c2pread, c2pwrite,
errread, errwrite,
restore_signals, start_new_session):
"""Execute program (POSIX version)"""

    if isinstance(args, (str, bytes)):
        args = [args]
    elif isinstance(args, os.PathLike):
        if shell:
            raise TypeError('path-like args is not allowed when '
                            'shell is true')
        args = [args]
    else:
        args = list(args)

    if shell:
        # On Android the default shell is at '/system/bin/sh'.
        unix_shell = ('/system/bin/sh' if
                  hasattr(sys, 'getandroidapilevel') else '/bin/sh')
        args = [unix_shell, "-c"] + args
        if executable:
            args[0] = executable

    if executable is None:
        executable = args[0]

    sys.audit("subprocess.Popen", executable, args, cwd, env)

    if (_USE_POSIX_SPAWN
            and os.path.dirname(executable)
            and preexec_fn is None
            and not close_fds
            and not pass_fds
            and cwd is None
            and (p2cread == -1 or p2cread > 2)
            and (c2pwrite == -1 or c2pwrite > 2)
            and (errwrite == -1 or errwrite > 2)
            and not start_new_session):
        self._posix_spawn(args, executable, env, restore_signals,
                          p2cread, p2cwrite,
                          c2pread, c2pwrite,
                          errread, errwrite)
        return

    orig_executable = executable

    # For transferring possible exec failure from child to parent.
    # Data format: "exception name:hex errno:description"
    # Pickle is not used; it is complex and involves memory allocation.
    errpipe_read, errpipe_write = os.pipe()
    # errpipe_write must not be in the standard io 0, 1, or 2 fd range.
    low_fds_to_close = []
    while errpipe_write < 3:
        low_fds_to_close.append(errpipe_write)
        errpipe_write = os.dup(errpipe_write)
    for low_fd in low_fds_to_close:
        os.close(low_fd)
    try:
        try:
            # We must avoid complex work that could involve
            # malloc or free in the child process to avoid
            # potential deadlocks, thus we do all this here.
            # and pass it to fork_exec()

            if env is not None:
                env_list = []
                for k, v in env.items():
                    k = os.fsencode(k)
                    if b'=' in k:
                        raise ValueError("illegal environment variable name")
                    env_list.append(k + b'=' + os.fsencode(v))
            else:
                env_list = None  # Use execv instead of execve.
            executable = os.fsencode(executable)
            if os.path.dirname(executable):
                executable_list = (executable,)
            else:
                # This matches the behavior of os._execvpe().
                executable_list = tuple(
                    os.path.join(os.fsencode(dir), executable)
                    for dir in os.get_exec_path(env))
            fds_to_keep = set(pass_fds)
            fds_to_keep.add(errpipe_write)
            self.pid = _posixsubprocess.fork_exec(
                    args, executable_list,
                    close_fds, tuple(sorted(map(int, fds_to_keep))),
                    cwd, env_list,
                    p2cread, p2cwrite, c2pread, c2pwrite,
                    errread, errwrite,
                    errpipe_read, errpipe_write,
                    restore_signals, start_new_session, preexec_fn)
            self._child_created = True
        finally:
            # be sure the FD is closed no matter what
            os.close(errpipe_write)

        self._close_pipe_fds(p2cread, p2cwrite,
                             c2pread, c2pwrite,
                             errread, errwrite)

        # Wait for exec to fail or succeed; possibly raising an
        # exception (limited in size)
        errpipe_data = bytearray()
        while True:
            part = os.read(errpipe_read, 50000)
            errpipe_data += part
            if not part or len(errpipe_data) > 50000:
                break
    finally:
        # be sure the FD is closed no matter what
        os.close(errpipe_read)

    if errpipe_data:
        try:
            pid, sts = os.waitpid(self.pid, 0)
            if pid == self.pid:
                self._handle_exitstatus(sts)
            else:
                self.returncode = sys.maxsize
        except ChildProcessError:
            pass

        try:
            exception_name, hex_errno, err_msg = (
                    errpipe_data.split(b':', 2))
            # The encoding here should match the encoding
            # written in by the subprocess implementations
            # like _posixsubprocess
            err_msg = err_msg.decode()
        except ValueError:
            exception_name = b'SubprocessError'
            hex_errno = b'0'
            err_msg = 'Bad exception data from child: {!r}'.format(
                          bytes(errpipe_data))
        child_exception_type = getattr(
                builtins, exception_name.decode('ascii'),
                SubprocessError)
        if issubclass(child_exception_type, OSError) and hex_errno:
            errno_num = int(hex_errno, 16)
            child_exec_never_called = (err_msg == "noexec")
            if child_exec_never_called:
                err_msg = ""
                # The error must be from chdir(cwd).
                err_filename = cwd
            else:
                err_filename = orig_executable
            if errno_num != 0:
                err_msg = os.strerror(errno_num)
          raise child_exception_type(errno_num, err_msg, err_filename)

E FileNotFoundError: [Errno 2] No such file or directory: 'dot'
/usr/lib/python3.8/subprocess.py:1702: FileNotFoundError
During handling of the above exception, another exception occurred:
def test_dep2dot():
files = """
foo:
- init.py
- a.py: |
from . import b
- b.py
"""
with create_files(files) as workdir:
assert simpledeps('foo', '-LDEBUG -vv') == {
'foo.b -> foo.a'
}

        args = pydeps.cli.parse_args(["foo", "--noshow"])
      pydeps.pydeps(**args)

tests/test_dep2dot.py:26:


pydeps/pydeps.py:136: in pydeps
return _pydeps(inp, **_args)
pydeps/pydeps.py:46: in _pydeps
svg = dot.call_graphviz_dot(dotsrc, fmt)
pydeps/dot.py:78: in call_graphviz_dot
cli.error("""


args = ("\n cannot find 'dot'\n\n pydeps calls dot (from graphviz) to create svg diagrams,\n please make sure that the dot executable is available\n on your path.\n ",)
kwargs = {'file': <_pytest.capture.EncodedFile object at 0xb56fffd0>}
def error(*args, **kwargs): # pragma: nocover
"""Print an error message and exit.
"""
kwargs['file'] = sys.stderr
print("\n\tERROR:", *args, **kwargs)

  sys.exit(1)

E SystemExit: 1
pydeps/cli.py:23: SystemExit
----------------------------- Captured stdout call -----------------------------
Namespace(cluster=False, debug=False, debug_mf=0, display=None, exclude=[], exclude_exact=[], externals=False, fname='foo', format='svg', include_missing=False, keep_target_cluster=False, log=None, max_bacon=2, max_cluster_size=0, min_cluster_size=0, no_config=False, no_output=False, nodot=False, noise_level=200, noshow=False, only=[], output=None, pylib=False, pylib_all=False, reverse=False, rmprefix=[], show=True, show_cycles=False, show_deps=False, show_dot=False, show_raw_deps=False, start_color=0, verbose=2, version=False)
----------------------------- Captured stderr call -----------------------------
ERROR:
cannot find 'dot'
pydeps calls dot (from graphviz) to create svg diagrams,
please make sure that the dot executable is available
on your path.

___________________________________ test_svg ___________________________________
tmpdir = local('/tmp/pytest-of-mockbuild/pytest-0/test_svg0')
def test_svg(tmpdir):
tmpdir.chdir()
ab = tmpdir.join('ab.svg')

  dot(u"""
    digraph G {
        a -> b
    }
    """, o=ab.basename)

/builddir/build/BUILD/pydeps-1.9.0/tests/test_dot.py:7:


/builddir/build/BUILD/pydeps-1.9.0/pydeps/dot.py:67: in dot
return pipe(cmd, to_bytes(src))
/builddir/build/BUILD/pydeps-1.9.0/pydeps/dot.py:49: in pipe
return Popen(
/usr/lib/python3.8/subprocess.py:854: in init
self._execute_child(args, executable, preexec_fn, close_fds,


self = <subprocess.Popen object at 0xb5beab08>
args = ['dot', '-Tsvg', '-oab.svg'], executable = b'dot', preexec_fn = None
close_fds = True, pass_fds = (), cwd = None, env = None, startupinfo = None
creationflags = 0, shell = False, p2cread = 8, p2cwrite = 10, c2pread = 11
c2pwrite = 12, errread = -1, errwrite = -1, restore_signals = True
start_new_session = False
def _execute_child(self, args, executable, preexec_fn, close_fds,
pass_fds, cwd, env,
startupinfo, creationflags, shell,
p2cread, p2cwrite,
c2pread, c2pwrite,
errread, errwrite,
restore_signals, start_new_session):
"""Execute program (POSIX version)"""

    if isinstance(args, (str, bytes)):
        args = [args]
    elif isinstance(args, os.PathLike):
        if shell:
            raise TypeError('path-like args is not allowed when '
                            'shell is true')
        args = [args]
    else:
        args = list(args)

    if shell:
        # On Android the default shell is at '/system/bin/sh'.
        unix_shell = ('/system/bin/sh' if
                  hasattr(sys, 'getandroidapilevel') else '/bin/sh')
        args = [unix_shell, "-c"] + args
        if executable:
            args[0] = executable

    if executable is None:
        executable = args[0]

    sys.audit("subprocess.Popen", executable, args, cwd, env)

    if (_USE_POSIX_SPAWN
            and os.path.dirname(executable)
            and preexec_fn is None
            and not close_fds
            and not pass_fds
            and cwd is None
            and (p2cread == -1 or p2cread > 2)
            and (c2pwrite == -1 or c2pwrite > 2)
            and (errwrite == -1 or errwrite > 2)
            and not start_new_session):
        self._posix_spawn(args, executable, env, restore_signals,
                          p2cread, p2cwrite,
                          c2pread, c2pwrite,
                          errread, errwrite)
        return

    orig_executable = executable

    # For transferring possible exec failure from child to parent.
    # Data format: "exception name:hex errno:description"
    # Pickle is not used; it is complex and involves memory allocation.
    errpipe_read, errpipe_write = os.pipe()
    # errpipe_write must not be in the standard io 0, 1, or 2 fd range.
    low_fds_to_close = []
    while errpipe_write < 3:
        low_fds_to_close.append(errpipe_write)
        errpipe_write = os.dup(errpipe_write)
    for low_fd in low_fds_to_close:
        os.close(low_fd)
    try:
        try:
            # We must avoid complex work that could involve
            # malloc or free in the child process to avoid
            # potential deadlocks, thus we do all this here.
            # and pass it to fork_exec()

            if env is not None:
                env_list = []
                for k, v in env.items():
                    k = os.fsencode(k)
                    if b'=' in k:
                        raise ValueError("illegal environment variable name")
                    env_list.append(k + b'=' + os.fsencode(v))
            else:
                env_list = None  # Use execv instead of execve.
            executable = os.fsencode(executable)
            if os.path.dirname(executable):
                executable_list = (executable,)
            else:
                # This matches the behavior of os._execvpe().
                executable_list = tuple(
                    os.path.join(os.fsencode(dir), executable)
                    for dir in os.get_exec_path(env))
            fds_to_keep = set(pass_fds)
            fds_to_keep.add(errpipe_write)
            self.pid = _posixsubprocess.fork_exec(
                    args, executable_list,
                    close_fds, tuple(sorted(map(int, fds_to_keep))),
                    cwd, env_list,
                    p2cread, p2cwrite, c2pread, c2pwrite,
                    errread, errwrite,
                    errpipe_read, errpipe_write,
                    restore_signals, start_new_session, preexec_fn)
            self._child_created = True
        finally:
            # be sure the FD is closed no matter what
            os.close(errpipe_write)

        self._close_pipe_fds(p2cread, p2cwrite,
                             c2pread, c2pwrite,
                             errread, errwrite)

        # Wait for exec to fail or succeed; possibly raising an
        # exception (limited in size)
        errpipe_data = bytearray()
        while True:
            part = os.read(errpipe_read, 50000)
            errpipe_data += part
            if not part or len(errpipe_data) > 50000:
                break
    finally:
        # be sure the FD is closed no matter what
        os.close(errpipe_read)

    if errpipe_data:
        try:
            pid, sts = os.waitpid(self.pid, 0)
            if pid == self.pid:
                self._handle_exitstatus(sts)
            else:
                self.returncode = sys.maxsize
        except ChildProcessError:
            pass

        try:
            exception_name, hex_errno, err_msg = (
                    errpipe_data.split(b':', 2))
            # The encoding here should match the encoding
            # written in by the subprocess implementations
            # like _posixsubprocess
            err_msg = err_msg.decode()
        except ValueError:
            exception_name = b'SubprocessError'
            hex_errno = b'0'
            err_msg = 'Bad exception data from child: {!r}'.format(
                          bytes(errpipe_data))
        child_exception_type = getattr(
                builtins, exception_name.decode('ascii'),
                SubprocessError)
        if issubclass(child_exception_type, OSError) and hex_errno:
            errno_num = int(hex_errno, 16)
            child_exec_never_called = (err_msg == "noexec")
            if child_exec_never_called:
                err_msg = ""
                # The error must be from chdir(cwd).
                err_filename = cwd
            else:
                err_filename = orig_executable
            if errno_num != 0:
                err_msg = os.strerror(errno_num)
          raise child_exception_type(errno_num, err_msg, err_filename)

E FileNotFoundError: [Errno 2] No such file or directory: 'dot'
/usr/lib/python3.8/subprocess.py:1702: FileNotFoundError
_________________________________ test_svg_str _________________________________
tmpdir = local('/tmp/pytest-of-mockbuild/pytest-0/test_svg_str0')
def test_svg_str(tmpdir):
tmpdir.chdir()
ab = tmpdir.join('ab.svg')

  dot("""
    digraph G {
        a -> b
    }
    """, o=ab.basename)

/builddir/build/BUILD/pydeps-1.9.0/tests/test_dot.py:18:


/builddir/build/BUILD/pydeps-1.9.0/pydeps/dot.py:67: in dot
return pipe(cmd, to_bytes(src))
/builddir/build/BUILD/pydeps-1.9.0/pydeps/dot.py:49: in pipe
return Popen(
/usr/lib/python3.8/subprocess.py:854: in init
self._execute_child(args, executable, preexec_fn, close_fds,


self = <subprocess.Popen object at 0xb5619400>
args = ['dot', '-Tsvg', '-oab.svg'], executable = b'dot', preexec_fn = None
close_fds = True, pass_fds = (), cwd = None, env = None, startupinfo = None
creationflags = 0, shell = False, p2cread = 8, p2cwrite = 10, c2pread = 11
c2pwrite = 12, errread = -1, errwrite = -1, restore_signals = True
start_new_session = False
def _execute_child(self, args, executable, preexec_fn, close_fds,
pass_fds, cwd, env,
startupinfo, creationflags, shell,
p2cread, p2cwrite,
c2pread, c2pwrite,
errread, errwrite,
restore_signals, start_new_session):
"""Execute program (POSIX version)"""

    if isinstance(args, (str, bytes)):
        args = [args]
    elif isinstance(args, os.PathLike):
        if shell:
            raise TypeError('path-like args is not allowed when '
                            'shell is true')
        args = [args]
    else:
        args = list(args)

    if shell:
        # On Android the default shell is at '/system/bin/sh'.
        unix_shell = ('/system/bin/sh' if
                  hasattr(sys, 'getandroidapilevel') else '/bin/sh')
        args = [unix_shell, "-c"] + args
        if executable:
            args[0] = executable

    if executable is None:
        executable = args[0]

    sys.audit("subprocess.Popen", executable, args, cwd, env)

    if (_USE_POSIX_SPAWN
            and os.path.dirname(executable)
            and preexec_fn is None
            and not close_fds
            and not pass_fds
            and cwd is None
            and (p2cread == -1 or p2cread > 2)
            and (c2pwrite == -1 or c2pwrite > 2)
            and (errwrite == -1 or errwrite > 2)
            and not start_new_session):
        self._posix_spawn(args, executable, env, restore_signals,
                          p2cread, p2cwrite,
                          c2pread, c2pwrite,
                          errread, errwrite)
        return

    orig_executable = executable

    # For transferring possible exec failure from child to parent.
    # Data format: "exception name:hex errno:description"
    # Pickle is not used; it is complex and involves memory allocation.
    errpipe_read, errpipe_write = os.pipe()
    # errpipe_write must not be in the standard io 0, 1, or 2 fd range.
    low_fds_to_close = []
    while errpipe_write < 3:
        low_fds_to_close.append(errpipe_write)
        errpipe_write = os.dup(errpipe_write)
    for low_fd in low_fds_to_close:
        os.close(low_fd)
    try:
        try:
            # We must avoid complex work that could involve
            # malloc or free in the child process to avoid
            # potential deadlocks, thus we do all this here.
            # and pass it to fork_exec()

            if env is not None:
                env_list = []
                for k, v in env.items():
                    k = os.fsencode(k)
                    if b'=' in k:
                        raise ValueError("illegal environment variable name")
                    env_list.append(k + b'=' + os.fsencode(v))
            else:
                env_list = None  # Use execv instead of execve.
            executable = os.fsencode(executable)
            if os.path.dirname(executable):
                executable_list = (executable,)
            else:
                # This matches the behavior of os._execvpe().
                executable_list = tuple(
                    os.path.join(os.fsencode(dir), executable)
                    for dir in os.get_exec_path(env))
            fds_to_keep = set(pass_fds)
            fds_to_keep.add(errpipe_write)
            self.pid = _posixsubprocess.fork_exec(
                    args, executable_list,
                    close_fds, tuple(sorted(map(int, fds_to_keep))),
                    cwd, env_list,
                    p2cread, p2cwrite, c2pread, c2pwrite,
                    errread, errwrite,
                    errpipe_read, errpipe_write,
                    restore_signals, start_new_session, preexec_fn)
            self._child_created = True
        finally:
            # be sure the FD is closed no matter what
            os.close(errpipe_write)

        self._close_pipe_fds(p2cread, p2cwrite,
                             c2pread, c2pwrite,
                             errread, errwrite)

        # Wait for exec to fail or succeed; possibly raising an
        # exception (limited in size)
        errpipe_data = bytearray()
        while True:
            part = os.read(errpipe_read, 50000)
            errpipe_data += part
            if not part or len(errpipe_data) > 50000:
                break
    finally:
        # be sure the FD is closed no matter what
        os.close(errpipe_read)

    if errpipe_data:
        try:
            pid, sts = os.waitpid(self.pid, 0)
            if pid == self.pid:
                self._handle_exitstatus(sts)
            else:
                self.returncode = sys.maxsize
        except ChildProcessError:
            pass

        try:
            exception_name, hex_errno, err_msg = (
                    errpipe_data.split(b':', 2))
            # The encoding here should match the encoding
            # written in by the subprocess implementations
            # like _posixsubprocess
            err_msg = err_msg.decode()
        except ValueError:
            exception_name = b'SubprocessError'
            hex_errno = b'0'
            err_msg = 'Bad exception data from child: {!r}'.format(
                          bytes(errpipe_data))
        child_exception_type = getattr(
                builtins, exception_name.decode('ascii'),
                SubprocessError)
        if issubclass(child_exception_type, OSError) and hex_errno:
            errno_num = int(hex_errno, 16)
            child_exec_never_called = (err_msg == "noexec")
            if child_exec_never_called:
                err_msg = ""
                # The error must be from chdir(cwd).
                err_filename = cwd
            else:
                err_filename = orig_executable
            if errno_num != 0:
                err_msg = os.strerror(errno_num)
          raise child_exception_type(errno_num, err_msg, err_filename)

E FileNotFoundError: [Errno 2] No such file or directory: 'dot'
/usr/lib/python3.8/subprocess.py:1702: FileNotFoundError
_________________________________ test_boolopt _________________________________
tmpdir = local('/tmp/pytest-of-mockbuild/pytest-0/test_boolopt0')
def test_boolopt(tmpdir):
tmpdir.chdir()
ab = tmpdir.join('ab.svg')

  dot("""
    digraph G {
        a -> b
    }
    """, x=True, o=ab.basename)

/builddir/build/BUILD/pydeps-1.9.0/tests/test_dot.py:29:


/builddir/build/BUILD/pydeps-1.9.0/pydeps/dot.py:67: in dot
return pipe(cmd, to_bytes(src))
/builddir/build/BUILD/pydeps-1.9.0/pydeps/dot.py:49: in pipe
return Popen(
/usr/lib/python3.8/subprocess.py:854: in init
self._execute_child(args, executable, preexec_fn, close_fds,


self = <subprocess.Popen object at 0xb5b9e208>
args = ['dot', '-Tsvg', '-x', '-oab.svg'], executable = b'dot'
preexec_fn = None, close_fds = True, pass_fds = (), cwd = None, env = None
startupinfo = None, creationflags = 0, shell = False, p2cread = 8, p2cwrite = 10
c2pread = 11, c2pwrite = 12, errread = -1, errwrite = -1, restore_signals = True
start_new_session = False
def _execute_child(self, args, executable, preexec_fn, close_fds,
pass_fds, cwd, env,
startupinfo, creationflags, shell,
p2cread, p2cwrite,
c2pread, c2pwrite,
errread, errwrite,
restore_signals, start_new_session):
"""Execute program (POSIX version)"""

    if isinstance(args, (str, bytes)):
        args = [args]
    elif isinstance(args, os.PathLike):
        if shell:
            raise TypeError('path-like args is not allowed when '
                            'shell is true')
        args = [args]
    else:
        args = list(args)

    if shell:
        # On Android the default shell is at '/system/bin/sh'.
        unix_shell = ('/system/bin/sh' if
                  hasattr(sys, 'getandroidapilevel') else '/bin/sh')
        args = [unix_shell, "-c"] + args
        if executable:
            args[0] = executable

    if executable is None:
        executable = args[0]

    sys.audit("subprocess.Popen", executable, args, cwd, env)

    if (_USE_POSIX_SPAWN
            and os.path.dirname(executable)
            and preexec_fn is None
            and not close_fds
            and not pass_fds
            and cwd is None
            and (p2cread == -1 or p2cread > 2)
            and (c2pwrite == -1 or c2pwrite > 2)
            and (errwrite == -1 or errwrite > 2)
            and not start_new_session):
        self._posix_spawn(args, executable, env, restore_signals,
                          p2cread, p2cwrite,
                          c2pread, c2pwrite,
                          errread, errwrite)
        return

    orig_executable = executable

    # For transferring possible exec failure from child to parent.
    # Data format: "exception name:hex errno:description"
    # Pickle is not used; it is complex and involves memory allocation.
    errpipe_read, errpipe_write = os.pipe()
    # errpipe_write must not be in the standard io 0, 1, or 2 fd range.
    low_fds_to_close = []
    while errpipe_write < 3:
        low_fds_to_close.append(errpipe_write)
        errpipe_write = os.dup(errpipe_write)
    for low_fd in low_fds_to_close:
        os.close(low_fd)
    try:
        try:
            # We must avoid complex work that could involve
            # malloc or free in the child process to avoid
            # potential deadlocks, thus we do all this here.
            # and pass it to fork_exec()

            if env is not None:
                env_list = []
                for k, v in env.items():
                    k = os.fsencode(k)
                    if b'=' in k:
                        raise ValueError("illegal environment variable name")
                    env_list.append(k + b'=' + os.fsencode(v))
            else:
                env_list = None  # Use execv instead of execve.
            executable = os.fsencode(executable)
            if os.path.dirname(executable):
                executable_list = (executable,)
            else:
                # This matches the behavior of os._execvpe().
                executable_list = tuple(
                    os.path.join(os.fsencode(dir), executable)
                    for dir in os.get_exec_path(env))
            fds_to_keep = set(pass_fds)
            fds_to_keep.add(errpipe_write)
            self.pid = _posixsubprocess.fork_exec(
                    args, executable_list,
                    close_fds, tuple(sorted(map(int, fds_to_keep))),
                    cwd, env_list,
                    p2cread, p2cwrite, c2pread, c2pwrite,
                    errread, errwrite,
                    errpipe_read, errpipe_write,
                    restore_signals, start_new_session, preexec_fn)
            self._child_created = True
        finally:
            # be sure the FD is closed no matter what
            os.close(errpipe_write)

        self._close_pipe_fds(p2cread, p2cwrite,
                             c2pread, c2pwrite,
                             errread, errwrite)

        # Wait for exec to fail or succeed; possibly raising an
        # exception (limited in size)
        errpipe_data = bytearray()
        while True:
            part = os.read(errpipe_read, 50000)
            errpipe_data += part
            if not part or len(errpipe_data) > 50000:
                break
    finally:
        # be sure the FD is closed no matter what
        os.close(errpipe_read)

    if errpipe_data:
        try:
            pid, sts = os.waitpid(self.pid, 0)
            if pid == self.pid:
                self._handle_exitstatus(sts)
            else:
                self.returncode = sys.maxsize
        except ChildProcessError:
            pass

        try:
            exception_name, hex_errno, err_msg = (
                    errpipe_data.split(b':', 2))
            # The encoding here should match the encoding
            # written in by the subprocess implementations
            # like _posixsubprocess
            err_msg = err_msg.decode()
        except ValueError:
            exception_name = b'SubprocessError'
            hex_errno = b'0'
            err_msg = 'Bad exception data from child: {!r}'.format(
                          bytes(errpipe_data))
        child_exception_type = getattr(
                builtins, exception_name.decode('ascii'),
                SubprocessError)
        if issubclass(child_exception_type, OSError) and hex_errno:
            errno_num = int(hex_errno, 16)
            child_exec_never_called = (err_msg == "noexec")
            if child_exec_never_called:
                err_msg = ""
                # The error must be from chdir(cwd).
                err_filename = cwd
            else:
                err_filename = orig_executable
            if errno_num != 0:
                err_msg = os.strerror(errno_num)
          raise child_exception_type(errno_num, err_msg, err_filename)

E FileNotFoundError: [Errno 2] No such file or directory: 'dot'
/usr/lib/python3.8/subprocess.py:1702: FileNotFoundError
___________________________________ test_obj ___________________________________
tmpdir = local('/tmp/pytest-of-mockbuild/pytest-0/test_obj0')
def test_obj(tmpdir): # pragma: nocover
GRAPH = u"""
digraph G {
a -> b
}
"""
import sys
if sys.version_info >= (3,):
class MyClass:
def str(self):
return GRAPH
else:
class MyClass(object):
def unicode(self):
return GRAPH

    tmpdir.chdir()
    ab = tmpdir.join('ab.svg')
  dot(MyClass(), x=True, o=ab.basename)

/builddir/build/BUILD/pydeps-1.9.0/tests/test_dot.py:56:


/builddir/build/BUILD/pydeps-1.9.0/pydeps/dot.py:67: in dot
return pipe(cmd, to_bytes(src))
/builddir/build/BUILD/pydeps-1.9.0/pydeps/dot.py:49: in pipe
return Popen(
/usr/lib/python3.8/subprocess.py:854: in init
self._execute_child(args, executable, preexec_fn, close_fds,


self = <subprocess.Popen object at 0xb5284d00>
args = ['dot', '-Tsvg', '-x', '-oab.svg'], executable = b'dot'
preexec_fn = None, close_fds = True, pass_fds = (), cwd = None, env = None
startupinfo = None, creationflags = 0, shell = False, p2cread = 8, p2cwrite = 10
c2pread = 11, c2pwrite = 12, errread = -1, errwrite = -1, restore_signals = True
start_new_session = False
def _execute_child(self, args, executable, preexec_fn, close_fds,
pass_fds, cwd, env,
startupinfo, creationflags, shell,
p2cread, p2cwrite,
c2pread, c2pwrite,
errread, errwrite,
restore_signals, start_new_session):
"""Execute program (POSIX version)"""

    if isinstance(args, (str, bytes)):
        args = [args]
    elif isinstance(args, os.PathLike):
        if shell:
            raise TypeError('path-like args is not allowed when '
                            'shell is true')
        args = [args]
    else:
        args = list(args)

    if shell:
        # On Android the default shell is at '/system/bin/sh'.
        unix_shell = ('/system/bin/sh' if
                  hasattr(sys, 'getandroidapilevel') else '/bin/sh')
        args = [unix_shell, "-c"] + args
        if executable:
            args[0] = executable

    if executable is None:
        executable = args[0]

    sys.audit("subprocess.Popen", executable, args, cwd, env)

    if (_USE_POSIX_SPAWN
            and os.path.dirname(executable)
            and preexec_fn is None
            and not close_fds
            and not pass_fds
            and cwd is None
            and (p2cread == -1 or p2cread > 2)
            and (c2pwrite == -1 or c2pwrite > 2)
            and (errwrite == -1 or errwrite > 2)
            and not start_new_session):
        self._posix_spawn(args, executable, env, restore_signals,
                          p2cread, p2cwrite,
                          c2pread, c2pwrite,
                          errread, errwrite)
        return

    orig_executable = executable

    # For transferring possible exec failure from child to parent.
    # Data format: "exception name:hex errno:description"
    # Pickle is not used; it is complex and involves memory allocation.
    errpipe_read, errpipe_write = os.pipe()
    # errpipe_write must not be in the standard io 0, 1, or 2 fd range.
    low_fds_to_close = []
    while errpipe_write < 3:
        low_fds_to_close.append(errpipe_write)
        errpipe_write = os.dup(errpipe_write)
    for low_fd in low_fds_to_close:
        os.close(low_fd)
    try:
        try:
            # We must avoid complex work that could involve
            # malloc or free in the child process to avoid
            # potential deadlocks, thus we do all this here.
            # and pass it to fork_exec()

            if env is not None:
                env_list = []
                for k, v in env.items():
                    k = os.fsencode(k)
                    if b'=' in k:
                        raise ValueError("illegal environment variable name")
                    env_list.append(k + b'=' + os.fsencode(v))
            else:
                env_list = None  # Use execv instead of execve.
            executable = os.fsencode(executable)
            if os.path.dirname(executable):
                executable_list = (executable,)
            else:
                # This matches the behavior of os._execvpe().
                executable_list = tuple(
                    os.path.join(os.fsencode(dir), executable)
                    for dir in os.get_exec_path(env))
            fds_to_keep = set(pass_fds)
            fds_to_keep.add(errpipe_write)
            self.pid = _posixsubprocess.fork_exec(
                    args, executable_list,
                    close_fds, tuple(sorted(map(int, fds_to_keep))),
                    cwd, env_list,
                    p2cread, p2cwrite, c2pread, c2pwrite,
                    errread, errwrite,
                    errpipe_read, errpipe_write,
                    restore_signals, start_new_session, preexec_fn)
            self._child_created = True
        finally:
            # be sure the FD is closed no matter what
            os.close(errpipe_write)

        self._close_pipe_fds(p2cread, p2cwrite,
                             c2pread, c2pwrite,
                             errread, errwrite)

        # Wait for exec to fail or succeed; possibly raising an
        # exception (limited in size)
        errpipe_data = bytearray()
        while True:
            part = os.read(errpipe_read, 50000)
            errpipe_data += part
            if not part or len(errpipe_data) > 50000:
                break
    finally:
        # be sure the FD is closed no matter what
        os.close(errpipe_read)

    if errpipe_data:
        try:
            pid, sts = os.waitpid(self.pid, 0)
            if pid == self.pid:
                self._handle_exitstatus(sts)
            else:
                self.returncode = sys.maxsize
        except ChildProcessError:
            pass

        try:
            exception_name, hex_errno, err_msg = (
                    errpipe_data.split(b':', 2))
            # The encoding here should match the encoding
            # written in by the subprocess implementations
            # like _posixsubprocess
            err_msg = err_msg.decode()
        except ValueError:
            exception_name = b'SubprocessError'
            hex_errno = b'0'
            err_msg = 'Bad exception data from child: {!r}'.format(
                          bytes(errpipe_data))
        child_exception_type = getattr(
                builtins, exception_name.decode('ascii'),
                SubprocessError)
        if issubclass(child_exception_type, OSError) and hex_errno:
            errno_num = int(hex_errno, 16)
            child_exec_never_called = (err_msg == "noexec")
            if child_exec_never_called:
                err_msg = ""
                # The error must be from chdir(cwd).
                err_filename = cwd
            else:
                err_filename = orig_executable
            if errno_num != 0:
                err_msg = os.strerror(errno_num)
          raise child_exception_type(errno_num, err_msg, err_filename)

E FileNotFoundError: [Errno 2] No such file or directory: 'dot'
/usr/lib/python3.8/subprocess.py:1702: FileNotFoundError
=============================== warnings summary ===============================
tests/test_cli.py::test_output
tests/test_cycles.py::test_cycle
tests/test_dep2dot.py::test_dep2dot
tests/test_dep2dot.py::test_dep2dot
tests/test_externals.py::test_relative_imports
tests/test_externals.py::test_relative_imports
tests/test_file.py::test_file
tests/test_file.py::test_file_pylib
tests/test_file.py::test_file_pyliball
tests/test_funny_names.py::test_from_html5lib
tests/test_json.py::test_dep2dot
tests/test_py2dep.py::test_py2depgraph
tests/test_relative_imports.py::test_relative_imports
tests/test_relative_imports.py::test_relative_imports2
tests/test_relative_imports.py::test_relative_imports3
tests/test_relative_imports.py::test_relative_imports_same_name_with_std
tests/test_relative_imports.py::test_relative_imports_same_name_with_std_future
tests/test_relative_imports.py::test_pydeps_colors
tests/test_relative_imports.py::test_hierarchy
tests/test_skinny_package.py::test_from_html5lib
tests/test_skip.py::test_no_skip
tests/test_skip.py::test_no_skip
tests/test_skip.py::test_skip_module_pattern
tests/test_skip.py::test_skip_module_pattern
tests/test_skip.py::test_skip_exact_pattern
tests/test_skip.py::test_skip_exact_pattern
tests/test_skip.py::test_skip_exact_pattern
tests/test_skip.py::test_skip_exact_pattern
tests/test_skip.py::test_skip_exact
tests/test_skip.py::test_skip_exact
tests/test_skip.py::test_skip_modules
tests/test_skip.py::test_rawdeps
/builddir/build/BUILD/pydeps-1.9.0/pydeps/mf27.py:115: DeprecationWarning: 'U' mode is deprecated
fp = open(pathname, READ_MODE)
-- Docs: https://docs.pytest.org/en/latest/warnings.html
============== 6 failed, 31 passed, 32 warnings in 21.93 seconds ===============

Support for namespace packages (PEP 420)

PEP 420 introduced implicit namespace packages, which are packages without __init__.py. Those packages, and the modules under them, don't seem to appear in the dependency graph generated by pydeps.

Here's an example. Given the following directory structure:

.
β”œβ”€β”€ main.py
β”œβ”€β”€ pkg1
β”‚Β Β  └── foo.py
└── pkg2
    β”œβ”€β”€ bar.py
    └── __init__.py

And main.py:

import pkg1.foo
import pkg2.bar

pydeps main.py generates the following graph:
image

I would have expected pkg1.foo to be included in the dependency graph. (On the other hand, I'm not sure there's a point in having pkg1 there since it's going to be empty.)

Support Clusters for Internal Structure?

I don't know if this is the same as #19 but I thought it might be useful to explain my use case.

Given a large codebase, some 350k LOC, (dis)organized into piles of packages, sometimes preventing a graph from being generated, since the dot file is too complex, I guess?
I ended up trying to use max-bacon=1 and tried excluding some paths in the codebase, however that ends up removing a significant amount of nodes imported into the excluded files.

I saw the new cluster feature, and wondered if this might be a viable option to use clustering for internal packages as well - so that I can "collapse" significant chunks of the codebase if there are more than N nodes inside that package.

Does that make sense?

"pydeps -o some/file.svg other/file.py" fails

Since pydeps does a chdir to the directory containing the source file, any relative path passed to -o will fail to open. One solution would be to call abspath on the output variable before the chdir.

Test failure on macOS

As part of starting to see how to contribute, I checked out the code, installed the requirements, and ran the tests, which produced this test failure.

________________________________________________________________________ test_output _________________________________________________________________________

tmpdir = local('/private/var/folders/rj/671kk3fx4cj_q5qdpp4kcmmm0000gp/T/pytest-of-miketheman/pytest-1/test_output0')

    def test_output(tmpdir):
        files = """
            unrelated: []
            foo:
                - __init__.py
                - a.py: |
                    from bar import b
            bar:
                - __init__.py
                - b.py
        """
        with create_files(files) as workdir:
>           assert os.getcwd() == workdir
E           AssertionError: assert '/private/var...T/tmpdq01lrqw' == '/var/folders/...T/tmpdq01lrqw'
E             - /private/var/folders/rj/671kk3fx4cj_q5qdpp4kcmmm0000gp/T/tmpdq01lrqw
E             ? --------
E             + /var/folders/rj/671kk3fx4cj_q5qdpp4kcmmm0000gp/T/tmpdq01lrqw

tests/test_cli.py:21: AssertionError

It looks like the os.getcwd call is returning the full path including /private/ preface.

Could you please add a simple configuration file too!? People without ConfigParser-knowledge would have difficulties!

Could you please add a simple configuration file too!? People without ConfigParser-knowledge would have difficulties!

[pydeps]
# See pydeps --help

# Console output
log=NOTSET
verbose=0
debug=0
debug-mf=0

# Graph output
format=svg
display=C:\Program Files (x86)\Inkscape\inkview.exe
output=C:\Users\my_name\tmp\pydeps-ouput.svg
noshow=1
show_deps=0
show__raw_deps=0
show_dot=0
nodot=0
show_cycles=0

# What and how to show
max_bacon=3
noise_level=200
exclude=pywin* win32* pythoncom winerror ntsecuritycon
externals=1
include_missing=0
pylib=0
pylib_all=0
reverse=0

Originally posted by @eqvis in #20 (comment)

Provide example of code

I need to do a call graph using pydeps, but on an executable. So, I think a way to do it is to add code that uses pydeps to make a call graph. The things is, I didn't find any example of how to use it programmatically. Could you, please, provide some. Sorry, if this is similar to other issues.

Support Python 3.8

Travis is failing without much info, but running the tests locally I get:

pydeps\pystdlib.py:11: in pystdlib
    return (set(stdlib_list.stdlib_list(curver)) | {
c:\srv\venv\pydeps38\lib\site-packages\stdlib_list\base.py:39: in stdlib_list
    version = get_canonical_version(version) if version is not None else '.'.join(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

version = '3.8'

    def get_canonical_version(version):

        if version in long_versions:
            version = ".".join(version.split(".")[:2])
        elif version not in short_versions:
>           raise ValueError("No such version: {}".format(version))
E           ValueError: No such version: 3.8

c:\srv\venv\pydeps38\lib\site-packages\stdlib_list\base.py:20: ValueError

There is already an upstream ticket: pypi/stdlib-list#24

Question: has anybody used pydeps for selective testing?

Hey all,

I'm considering using pydeps for selective testing on pull requests. The idea would be to compute the dependency graph, and only run tests for the modules and their dependencies which have changed compared to master.

A few false negatives are OK, it's fine for it just to work pretty well, most of the time.

Has anybody tried this? I searched around but couldn't find anything...

Selecting depth for graph

I'm trying to analyze a very large Django app. The repo looks like something like this:

my_app
β”œβ”€β”€ __init__.py
β”œβ”€β”€ client
β”‚Β Β  β”œβ”€β”€ __init__.py
β”‚Β Β  β”œβ”€β”€ client.py
β”‚Β Β  β”œβ”€β”€ middleware.py
β”‚Β Β  β”œβ”€β”€ models.py
β”‚Β Β  β”œβ”€β”€ settings.py
β”‚Β Β  └── tasks.py
β”œβ”€β”€ api
β”‚Β Β  β”œβ”€β”€ __init__.py
β”‚Β Β  β”œβ”€β”€ admin.py
β”‚Β Β  β”œβ”€β”€ apps.py
β”‚Β Β  β”œβ”€β”€ authentication.py
β”‚Β Β  β”œβ”€β”€ managers.py
β”‚Β Β  β”œβ”€β”€ migrations
β”‚Β Β  β”œβ”€β”€ models.py
β”‚Β Β  β”œβ”€β”€ permissions.py
β”‚Β Β  β”œβ”€β”€ routers.py
β”‚Β Β  β”œβ”€β”€ serializers.py
β”‚Β Β  β”œβ”€β”€ templates
β”‚Β Β  β”œβ”€β”€ tests
β”‚Β Β  β”œβ”€β”€ urls
β”‚Β Β  β”œβ”€β”€ views.py
β”‚Β Β  └── viewsets.py
β”œβ”€β”€ users
β”‚Β Β  β”œβ”€β”€ __init__.py
β”‚Β Β  β”œβ”€β”€ apps.py
β”‚Β Β  β”œβ”€β”€ blocks
β”‚Β Β  β”œβ”€β”€ documents.py

I'd like to be able to specify to what depth to graph the dependencies. I would like to figure out which modules (client, api, users, etc.) are depending on which, and which way the dependency goes.

As far as I've figured out, I can only generate a graph of everything, but that becomes huge and hard to navigate in. Or I can specify which modules I want with --only but then I'm missing the dependencies I already don't know about.

Is there any way I can graph everything, but create clusters of each module?

Not detecting imports in python file and questions

Hi,
I run pydeps on my file with this following command line (with different parameters) :
pydeps JobScriptTemplate.py --max-bacon 6 --noise-level 2 --pylib

There is some imports in my file but it detects nothing, the graph is empty.
This file comes from a package and it imports a class from another directory in this package.
It imports :

import os
import sys
import time
import shutil
from commons.core.checker.RepetException import RepetException
from commons.core.sql.TableJobAdaptator import TableJobAdaptator
from commons.core.sql.DbFactory import DbFactory
from commons.core.sql.Job import Job

The directory 'commons' is like :

.
β”œβ”€β”€ core
β”‚Β Β  β”œβ”€β”€ checker
β”‚Β Β  β”œβ”€β”€ launcher
β”‚Β Β  └── sql
β”œβ”€β”€ launcher
└── tools

and my file is on '/core/launcher' (each directory has a init.py file).

Morever I don't really understand the difference between --max-bacon and --noise-level. What is the effect of noise-level on max-bacon ?
I'm wondering if I can attribute a color for a folder level. How can I do this ? Because I want to run pydeps on different folders and I want to keep the same color for each folder.
And can I import and use pydeps in a python script ? I don't find any constructor in pydeps sources.

I thank you in advance for your help.
Mariene

Reverse direction of arrows

Hi,
pydeps has already helped me find a few ... strange ... imports and dependencies but I'm struggling with the direction of the arrows - I feel that if they show a dependency they should show the item doing the depending pointing back to the item they it depends on. At the moment, my graphs have the depended on item with an arrow to its dependency.

Importing a single object from a module results in duplicate imports

Hello,
I love the project. It is the only one that works well out of the box and with minimal configuration.

Currently, importing a single object from a module results in a duplicate import in the dependency graph:

  • the parent module
  • the child module

E.g.:

# in test_services.py
from telereddit.services.services_wrapper import ServicesWrapper

results in the following dependency graph:

I would expect the graph to not present the services node, because I'm not importing it.

In general, all modules (even if not imported directly) are present and generate duplicate imports arrows.
Am I using it wrong, is this intended behavior or is this a bug?

Read data from file or stdin

I'm wondering if this is possible to feed data to pydeps instead of letting it read it from the source files. I developed a dependency analyzer tool myself (shameless plug) and I would like to feed its output to pydeps to generate the graphs πŸ™‚

I didn't see such options in the README, but there is an "intermediate" format description. I think I can easily add a pydeps format output to my tool. How would I then feed it to pydeps?

Follow Imports: absolute, relative, and directories (not just modules)

How can pydeps display dependencies between files in a directory nested with/without modules?
Previous issues #25 or #28 don't quite answer this question.

For example, I'd like to display the dependencies of my project under the following "directory_root":

directory_root
β”œβ”€β”€ module_a
β”‚   β”œβ”€β”€ __init__.py
β”‚   β”œβ”€β”€ a.py
β”‚   └── b.py
β”œβ”€β”€ directory_a
β”‚   β”œβ”€β”€ c.py
β”‚   └── d.py
└── module_b
     β”œβ”€β”€ __init__.py
     β”œβ”€β”€ e.py
     └── f.py

where the dependencies include python libraries, modules inside this directory_root, and even files in this directory_root not organized as modules (like c.py or d.py)

Deterministic output

The generated file changes each time I run pydeps again. Would it be possible to make the resulting SVG the same each time, as long as the source is the same as well?

Additional options: remove package name, and add --version flag

Two suggestions:

  1. add a --version flag; not critical, but useful to have when reading the documentation.
  2. Add an option to remove the original package name. For example, using pydeps on pydeps, instead of showing a node as pydeps.colors, show it as colors. I believe that this addition, especially when using the --keep-target-cluster option, would make everything much easier to read.

failure with no __main__.py in package

If you have a package that contains only an __init__.py file, then pydeps <pkg> fails when it tries to calculate_bacon() because self.sources['__main__'] raises a KeyError. The solution is to add a check that verifies '__main__' in self.sources before the bacon() call.

Possible typo in depgraph2dot.py

When attempting to run pydeps on a file using --show-cycles, a warning is issued by graphviz:

Warning: using box for unknown shape octacon

This appears to be a typo in depgraph2dot.py at lines 70 and 114, as I assume the shape name is meant to be octagon

I am using

pydeps v1.8.7

Catch missing graphviz

Hi,
I'm aware the docs mention that graphviz is required, but it would be much nicer to see a reminder of that fact instead of a backtrace :)

thanks,
kk

RecursionError when importing from SciPy

EDIT:
Error is caused from an import of scipy.interpolate and was misattributed to a self import previously.

Original:

It appears that a RecursionError is invoked when a module imports itself. max-bacon and cluster limits dont seem to limit the recursion level and no other sanity check seems to be performed if the module is already in the hierarchy, meaning the maximum recursion depth is always exceeded. Specifically, the chain I encountered is the following:

  File "...\anaconda3\lib\site-packages\pydeps\py2depgraph.py", line 116, in import_hook
    return mf27.ModuleFinder.import_hook(self, name, caller, fromlist, level)
  File "...\anaconda3\lib\site-packages\pydeps\mf27.py", line 134, in import_hook
    m = self.load_tail(q, tail)
  File "...\anaconda3\lib\site-packages\pydeps\mf27.py", line 213, in load_tail
    m = self.import_module(head, mname, m)
  File "...\anaconda3\lib\site-packages\pydeps\py2depgraph.py", line 138, in import_module
    module = mf27.ModuleFinder.import_module(self, partnam, fqname, parent)
  File "...\anaconda3\lib\site-packages\pydeps\mf27.py", line 283, in import_module
    m = self.load_module(fqname, fp, pathname, stuff)
  File "...\anaconda3\lib\site-packages\pydeps\py2depgraph.py", line 146, in load_module
    self, fqname, fp, pathname, (suffix, mode, kind)
  File "\anaconda3\lib\site-packages\pydeps\mf27.py", line 335, in load_module
    self.scan_code(co, module)
  File "...\anaconda3\lib\site-packages\pydeps\mf27.py", line 511, in scan_code
    self.scan_code(literal, module)
  File "...\anaconda3\lib\site-packages\pydeps\mf27.py", line 511, in scan_code
    self.scan_code(literal, module)
  File "...\anaconda3\lib\site-packages\pydeps\mf27.py", line 472, in scan_code
    self._safe_import_hook(name, module, fromlist, level=level)
  File "...\anaconda3\lib\site-packages\pydeps\mf27.py", line 353, in _safe_import_hook
    self.import_hook(name, caller, level=level)
  File "...\anaconda3\lib\site-packages\pydeps\py2depgraph.py", line 116, in import_hook
    return mf27.ModuleFinder.import_hook(self, name, caller, fromlist, level)
  File "...\anaconda3\lib\site-packages\pydeps\mf27.py", line 130, in import_hook
    q, tail = self.find_head_package(parent, name)
  File "...\anaconda3\lib\site-packages\pydeps\mf27.py", line 191, in find_head_package
    q = self.import_module(head, qname, parent)
  File "...\anaconda3\lib\site-packages\pydeps\py2depgraph.py", line 139, in import_module
    self._add_import(module)
  File "...\anaconda3\lib\site-packages\pydeps\py2depgraph.py", line 129, in _add_import
    rpath = os.path.split(module.__file__)[0].lower()
  File "...\anaconda3\lib\ntpath.py", line 184, in split
    seps = _get_bothseps(p)
  File "...\anaconda3\lib\ntpath.py", line 35, in _get_bothseps
    if isinstance(path, bytes):
RecursionError: maximum recursion depth exceeded while calling a Python object

Import problem in Ubuntu with os.startfile()

I have an import problem when I start pydeps with the following command:

$ pydeps src/
Traceback (most recent call last):
  File "/usr/local/bin/pydeps", line 11, in <module>
    sys.exit(pydeps())
  File "/usr/local/lib/python2.7/dist-packages/pydeps/pydeps.py", line 144, in pydeps
    _pydeps(**_args)
  File "/usr/local/lib/python2.7/dist-packages/pydeps/pydeps.py", line 50, in _pydeps
    os.startfile(output)
AttributeError: 'module' object has no attribute 'startfile'

As I read at https://docs.python.org/2/library/os.html#os.startfile os.startfile() works only on Windows. Is it mean pydeps supports only Windows platorms?
Thanks.

Host: Ubuntu Xenial amd64
pydeps installed through pip

pip list | grep pydeps
pydeps (1.2.8)

Dashes in input filename lead to empty output

$ pydeps.exe -o out.svg my-script.py
dummymodule.py:113: WARNING: SKIPPING ILLEGAL MODULE_NAME: my-script

Leads to an empty file (no image content) and shows a warning. However this works fine:

$ pydeps.exe -o out-new.svg myscript.py

Output can contain dashes. Input files not. Seems like a bug

Feature Request - Output gv dot file only, skip running graphviz

I want/need to add additional options to the gv file before it is parsed by Graphviz with dot, so I'd like to just output --show-dot the file and not render it. I thought --nodot would do it, but no. I'm not sure what the best configuration option is; open to suggestions. I'm currently using /dev/null as a workaround, but it's a waste of CPU:

pydeps --max-bacon 2 --noshow --show-dot -o /dev/null udacityu/src > depgraph.gv
sed -i 's/digraph G {/digraph G {\n    nodesep=0.1\n    ranksep=0.2/g' depgraph.gv
dot -T png -x -O depgraph.gv

fails to build with python 3.9.0a6

pydeps fails to build with Python 3.9.0a6.

version = '3.9'

def get_canonical_version(version):

    if version in long_versions:
        version = ".".join(version.split(".")[:2])
    elif version not in short_versions:
      raise ValueError("No such version: {}".format(version))

E ValueError: No such version: 3.9

excludes in .pydeps ignored!

Hi,

after some tries I get something now, but I tried to configure excludes (pywin...) into my config-file...
I looked into the code and in the docs for argparser:

Note that parser-level defaults always override argument-level defaults

The same ocurrs for the other defaults! Perhaps you want to remove all the default=... in add_argument() and add your defaults into defaults before:

C:\temp>diff -u c:\Python27\Lib\site-packages\pydeps\cli.py c:\Python27\Lib\site-packages\pydeps\cli.ORG.py
--- c:\Python27\Lib\site-packages\pydeps\cli.py Thu Feb 07 13:18:36 2019
+++ c:\Python27\Lib\site-packages\pydeps\cli.ORG.py     Thu Feb 07 12:02:52 2019
@@ -75,11 +75,9 @@
     defaults = dict(
         T='svg',
         noise_level=200,
-        max_bacon=2,
+        max_bacon=200,
         exclude=[],
         display=None,
-        verbose=0,
-        debug_mf=0,
     )

     if not _args.no_config:
@@ -109,7 +107,7 @@
     # -vv   decision data (computed/converted/replaced values)
     # -vvv  status data (program state at fixed intervals, ie. not in loops)
     # -vvvv execution trace
-    p.add_argument('-v', '--verbose', action='count', help="be more verbose (-vv, -vvv for more verbosity)")
+    p.add_argument('-v', '--verbose', action='count', help="be more verbose (-vv, -vvv for more verbosity)", default=0)
     p.add_argument('-o', dest='output', metavar="file", help="write output to 'file'")
     p.add_argument('-T', dest='format', help="output format (svg|png)")
     p.add_argument('--display', help="program to use to display the graph (png or svg file depending on the T parameter)", metavar="PROGRAM")
@@ -121,13 +119,13 @@
     p.add_argument('--nodot', action='store_true', help="skip dot conversion")
     p.add_argument('--show-cycles', action='store_true', help="show only import cycles")
     p.add_argument('--debug', action='store_true', help="turn on all the show and verbose options")
-    p.add_argument('--debug-mf', type=int, metavar="INT", help="set the ModuleFinder.debug flag to this value")
+    p.add_argument('--debug-mf', default=0, type=int, metavar="INT", help="set the ModuleFinder.debug flag to this value")
     p.add_argument('--noise-level', type=int, metavar="INT", help="exclude sources or sinks with degree greater than noise-level")
-    p.add_argument('--max-bacon', type=int, metavar="INT", help="exclude nodes that are more than n hops away (default=2, 0 -> infinite)")
+    p.add_argument('--max-bacon', type=int, default=2, metavar="INT", help="exclude nodes that are more than n hops away (default=2, 0 -> infinite)")
     p.add_argument('--pylib', action='store_true', help="include python std lib modules")
     p.add_argument('--pylib-all', action='store_true', help="include python all std lib modules (incl. C modules)")
     p.add_argument('--include-missing', action='store_true', help="include modules that are not installed (or can't be found on sys.path)")
-    p.add_argument('-x', '--exclude', nargs="+", metavar="FNAME", help="input files to skip")
+    p.add_argument('-x', '--exclude', nargs="+", default=[], metavar="FNAME", help="input files to skip")
     p.add_argument('--externals', action='store_true', help='create list of direct external dependencies')
     p.add_argument('--reverse', action='store_true', help="draw arrows to (instead of from) imported modules")

Anyway to create artificial dependencies?

This library is awesome! I was wondering if there was a create artificial dependencies so that we can use this library for other things rather than just python dependencies. For example, rather than dependency graph for python, one for C, or NodeJS. Basically making this library universal. Thanks!

Programmatic use

I'm trying to use pydeps as a library, to get the SVG as text, and not output it in a file.

The examples in #50 do not allow to do that.

What I'm doing instead is this:

from pydeps import py2depgraph, dot
from pydeps.pydeps import depgraph_to_dotsrc
from pydeps.target import Target

target = Target("src/mkdocstrings")
with target.chdir_work():
    dep_graph = py2depgraph.py2dep(target)
dot_src = depgraph_to_dotsrc(target, dep_graph)
svg = dot.call_graphviz_dot(dot_src, "svg")

The problem is that this code reaches cli.verbose(...) while cli.verbose is not initialized, and is still None, resulting in a not callable exception. This forces me to do this:

from pydeps import cli

cli.verbose = cli._not_verbose

...which is a bit cumbersome πŸ˜…

Then I ran into other errors because of missing arguments, and it's very hard to know which one I need since they are hidden along a good part of the call chain in **kwargs dicts, and otherwise are not set to default values. I had to run back and forth debugging inspection to get the first options:

options = {"exclude_exact": [], "exclude": [], "only": ["mkdocstrings"], "show_cycles": False, "show_raw_deps": False}

...then stopped and decided to actually use the argument parser:

options = cli.parse_args(["src/mkdocstrings", "--noshow", "--only", "mkdocstrings"])

target = Target(options["fname"])
with target.chdir_work():
    dep_graph = py2depgraph.py2dep(target, **options)
dot_src = depgraph_to_dotsrc(target, dep_graph, **options)

Anyway, I got it to work, so it's fine, this is just some feedback on using pydeps as a library πŸ˜…

Hope this can help others πŸ™‚

Support for Cython

Support for Cython with fiel formats .pxd and .pyx
.pxd contains declarations from C++ which can be ignored.
.pyx contains import and cimport from other .pyx and .pxd files. This should be processed for dependency.

false detection of cyclic dependencies when using mypy

Hi,
first of all - thanks for the awesome tool, I've been looking for something like that for a while!

Now - a suggestion: we use this tool mostly when debugging circular dependencies in a huge project. However we also use mypy on the project, and the mypy way of typing looks like this:



# special system variable that evaluates to True only when typechecking
from typing import TYPE_CHECKING  
if TYPE_CHECKING: 
    from some_module import SomeModel

def function(model: 'SomeModel'):
    print(model.whatever)
  

So there you have it, a dependency used only for type checking that can be detected as a circular dependency (hence the if TYPE_CHECKING condition to make this work in regular Python).

So, is there a way to ignore such dependencies while building the dependency graph?

So far my only idea is to manually remove these import strings using a custom script, feed the code to pydeps and then bring the imports back :)

Release 1.9.0 its ready?

Hi @thebjorn

Release 1.9.0 its ready?

I see that the release notes says that version 1.9.0 was released but I do not see it in the tags or in the release to download it.

And in pypi.org it lacks folders like tests

Cheers,

Just installed. Get error "No such file or directory"

I just installed pydeps and it seems to have installed properly but I can't run it on any package, including pydeps itself. Here's the output I get:

(base) MacBook-Pro-5:~ Johannes$ pydeps
usage: pydeps [-h] [--debug] [--config FILE] [--no-config] [--version]
              [-L LOG] [-v] [-o file] [-T FORMAT] [--display PROGRAM]
              [--noshow] [--show-deps] [--show-raw-deps] [--show-dot]
              [--nodot] [--no-output] [--show-cycles] [--debug-mf INT]
              [--noise-level INT] [--max-bacon INT] [--pylib] [--pylib-all]
              [--include-missing] [-x PATTERN [PATTERN ...]]
              [-xx MODULE [MODULE ...]] [--only MODULE_PATH [MODULE_PATH ...]]
              [--externals] [--reverse] [--cluster] [--min-cluster-size INT]
              [--max-cluster-size INT] [--keep-target-cluster]
              [--rmprefix PREFIX [PREFIX ...]]
              fname
pydeps: error: the following arguments are required: fname
(base) MacBook-Pro-5:~ Johannes$ pydeps pydeps
No such file or directory: 'pydeps'
(base) MacBook-Pro-5:~ Johannes$ 

I am using anaconda if that makes a difference so pydeps is installed at /opt/anaconda3/bin/pydeps

Colour palette

First of all, Great Module!

Is there a way to select different colour palettes? For example, if I would like the colours to be more like Google's material design instead of the defaults from graphviz?

Reading directory names

Hi,

I'm having an issue running pydeps on directories. When I run 'pydeps file.py' I get a perfectly good svg file. Directory names, on the other hand, don't seem to work and I get the following:

  File "/opt/anaconda3/lib/python3.7/site-packages/pydeps/dummymodule.py", line 70, in __init__
    for fname in os.listdir(target.dirname):
FileNotFoundError: [Errno 2] No such file or directory

What have I done wrong?

Thanks,
Alex

Update docs with need of installing project dependencies before running pydeps

Hi, it's me again :)

I've wasted a massive amount of time today because I was trying to set up a Github Action to automatically generate the dependency graph of my project and commit it to the repository, but every time I tried, the graph generated from the CI was different from what I had on my machine. In particular, external dependencies were not shown. Instead, only my project's modules were shown, despite having max_bacon=3.

After a couple of hours, it turns out that pydeps needs to have the project's dependencies installed in the environment before running, otherwise it will not catch those imports.

Doing a pip install -r requirements.txt before running pydeps fixed the issue.

I have not seen this required in the documentation, and since the only thing my Github Workflow is doing is generate the graph, I did not install the requirements. Can we add this to the docs for future reference?

Failed on py.py

html5lib has a file called html5lib/trie/py.py, so the full module reference is html5lib._trie.py. If I run pydeps on this or on a project that uses this, it fails with an error similar to the following:

Traceback (most recent call last):
  File "C:\src\html5lib-python\env\Scripts\pydeps-script.py", line 11, in <module>
    load_entry_point('pydeps==1.5.1', 'console_scripts', 'pydeps')()
  File "c:\src\html5lib-python\env\lib\site-packages\pydeps\pydeps.py", line 263, in pydeps
    _pydeps(**_args)
  File "c:\src\html5lib-python\env\lib\site-packages\pydeps\pydeps.py", line 38, in _pydeps
    g = py2dep(fname, **kw)
  File "c:\src\html5lib-python\env\lib\site-packages\pydeps\py2depgraph.py", line 351, in py2dep
    return depgraph.DepGraph(mf_depgraph, mf._types, **kw)
  File "c:\src\html5lib-python\env\lib\site-packages\pydeps\depgraph.py", line 275, in __init__
    self.calculate_bacon()
  File "c:\src\html5lib-python\env\lib\site-packages\pydeps\depgraph.py", line 385, in calculate_bacon
    bacon(self.sources['__main__'], 0)
  File "c:\src\html5lib-python\env\lib\site-packages\pydeps\depgraph.py", line 382, in bacon
    bacon(self.sources[imp], n + 1)
  File "c:\src\html5lib-python\env\lib\site-packages\pydeps\depgraph.py", line 382, in bacon
    bacon(self.sources[imp], n + 1)
  File "c:\src\html5lib-python\env\lib\site-packages\pydeps\depgraph.py", line 382, in bacon
    bacon(self.sources[imp], n + 1)
  [Previous line repeated 2 more times]
KeyError: 'html5lib._trie.py'

I believe that the problem is this line and this line within depgraph.py. The names (depgraph keys) at this point are module names (e.g., html5lib._trie.py), not file paths (e.g., html5lib/_trie/py.py), so removing .py isn't necessary, and it breaks processing of this module.

Therefore, I think the fix is to update depgraph.py to not remove .py from names. However, I could be missing something. Does this sound right to you? If so, I can submit a PR.

Thanks!

package doesn't declare dependency on pyyaml

I tried to run pydeps, and it complained that it couldn't import yaml, which should have been installed when I got pydeps. Installing pyyaml with pip resolved the issue, but shouldn't have been necessary.

AttributeError

I get an AttributeError when I run pydeps.

$ pydeps --externals cv_prioritizer_wrapper.py
Traceback (most recent call last):
File "/usr/local/bin/pydeps", line 11, in
sys.exit(pydeps())
File "/usr/local/lib/python2.7/dist-packages/pydeps/pydeps.py", line 226, in pydeps
exts = externals(fname)
File "/usr/local/lib/python2.7/dist-packages/pydeps/pydeps.py", line 197, in externals
depgraph = py2dep(pkgname, **kw)
File "/usr/local/lib/python2.7/dist-packages/pydeps/py2depgraph.py", line 302, in py2dep
mf.run_script(fname)
File "/usr/local/lib/python2.7/dist-packages/pydeps/mf27.py", line 116, in run_script
self.load_module('main', fp, pathname, stuff)
File "/usr/local/lib/python2.7/dist-packages/pydeps/py2depgraph.py", line 144, in load_module
self, fqname, fp, pathname, (suffix, mode, kind)
File "/usr/local/lib/python2.7/dist-packages/pydeps/mf27.py", line 311, in load_module
self.scan_code(co, module)
File "/usr/local/lib/python2.7/dist-packages/pydeps/mf27.py", line 427, in scan_code
self._safe_import_hook(name, module, fromlist, level=level)
File "/usr/local/lib/python2.7/dist-packages/pydeps/mf27.py", line 329, in _safe_import_hook
self.import_hook(name, caller, level=level)
File "/usr/local/lib/python2.7/dist-packages/pydeps/py2depgraph.py", line 115, in import_hook
return mf27.ModuleFinder.import_hook(self, name, caller, fromlist, level)
File "/usr/local/lib/python2.7/dist-packages/pydeps/mf27.py", line 129, in import_hook
q, tail = self.find_head_package(parent, name)
File "/usr/local/lib/python2.7/dist-packages/pydeps/mf27.py", line 188, in find_head_package
q = self.import_module(head, qname, parent)
File "/usr/local/lib/python2.7/dist-packages/pydeps/py2depgraph.py", line 137, in import_module
module = mf27.ModuleFinder.import_module(self, partnam, fqname, parent)
File "/usr/local/lib/python2.7/dist-packages/pydeps/mf27.py", line 280, in import_module
m = self.load_module(fqname, fp, pathname, stuff)
File "/usr/local/lib/python2.7/dist-packages/pydeps/py2depgraph.py", line 144, in load_module
self, fqname, fp, pathname, (suffix, mode, kind)
File "/usr/local/lib/python2.7/dist-packages/pydeps/mf27.py", line 311, in load_module
self.scan_code(co, module)
File "/usr/local/lib/python2.7/dist-packages/pydeps/mf27.py", line 427, in scan_code
self._safe_import_hook(name, module, fromlist, level=level)
File "/usr/local/lib/python2.7/dist-packages/pydeps/mf27.py", line 329, in _safe_import_hook
self.import_hook(name, caller, level=level)
File "/usr/local/lib/python2.7/dist-packages/pydeps/py2depgraph.py", line 115, in import_hook
return mf27.ModuleFinder.import_hook(self, name, caller, fromlist, level)
File "/usr/local/lib/python2.7/dist-packages/pydeps/mf27.py", line 131, in import_hook
m = self.load_tail(q, tail)
File "/usr/local/lib/python2.7/dist-packages/pydeps/mf27.py", line 210, in load_tail
m = self.import_module(head, mname, m)
File "/usr/local/lib/python2.7/dist-packages/pydeps/py2depgraph.py", line 137, in import_module
module = mf27.ModuleFinder.import_module(self, partnam, fqname, parent)
File "/usr/local/lib/python2.7/dist-packages/pydeps/mf27.py", line 280, in import_module
m = self.load_module(fqname, fp, pathname, stuff)
File "/usr/local/lib/python2.7/dist-packages/pydeps/py2depgraph.py", line 144, in load_module
self, fqname, fp, pathname, (suffix, mode, kind)
File "/usr/local/lib/python2.7/dist-packages/pydeps/mf27.py", line 311, in load_module
self.scan_code(co, module)
File "/usr/local/lib/python2.7/dist-packages/pydeps/mf27.py", line 427, in scan_code
self._safe_import_hook(name, module, fromlist, level=level)
File "/usr/local/lib/python2.7/dist-packages/pydeps/mf27.py", line 329, in _safe_import_hook
self.import_hook(name, caller, level=level)
File "/usr/local/lib/python2.7/dist-packages/pydeps/py2depgraph.py", line 115, in import_hook
return mf27.ModuleFinder.import_hook(self, name, caller, fromlist, level)
File "/usr/local/lib/python2.7/dist-packages/pydeps/mf27.py", line 131, in import_hook
m = self.load_tail(q, tail)
File "/usr/local/lib/python2.7/dist-packages/pydeps/mf27.py", line 210, in load_tail
m = self.import_module(head, mname, m)
File "/usr/local/lib/python2.7/dist-packages/pydeps/py2depgraph.py", line 137, in import_module
module = mf27.ModuleFinder.import_module(self, partnam, fqname, parent)
File "/usr/local/lib/python2.7/dist-packages/pydeps/mf27.py", line 280, in import_module
m = self.load_module(fqname, fp, pathname, stuff)
File "/usr/local/lib/python2.7/dist-packages/pydeps/py2depgraph.py", line 144, in load_module
self, fqname, fp, pathname, (suffix, mode, kind)
File "/usr/local/lib/python2.7/dist-packages/pydeps/mf27.py", line 311, in load_module
self.scan_code(co, module)
File "/usr/local/lib/python2.7/dist-packages/pydeps/mf27.py", line 427, in scan_code
self._safe_import_hook(name, module, fromlist, level=level)
File "/usr/local/lib/python2.7/dist-packages/pydeps/mf27.py", line 329, in _safe_import_hook
self.import_hook(name, caller, level=level)
File "/usr/local/lib/python2.7/dist-packages/pydeps/py2depgraph.py", line 115, in import_hook
return mf27.ModuleFinder.import_hook(self, name, caller, fromlist, level)
File "/usr/local/lib/python2.7/dist-packages/pydeps/mf27.py", line 129, in import_hook
q, tail = self.find_head_package(parent, name)
File "/usr/local/lib/python2.7/dist-packages/pydeps/mf27.py", line 195, in find_head_package
q = self.import_module(head, qname, parent)
File "/usr/local/lib/python2.7/dist-packages/pydeps/py2depgraph.py", line 137, in import_module
module = mf27.ModuleFinder.import_module(self, partnam, fqname, parent)
File "/usr/local/lib/python2.7/dist-packages/pydeps/mf27.py", line 280, in import_module
m = self.load_module(fqname, fp, pathname, stuff)
File "/usr/local/lib/python2.7/dist-packages/pydeps/py2depgraph.py", line 144, in load_module
self, fqname, fp, pathname, (suffix, mode, kind)
File "/usr/local/lib/python2.7/dist-packages/pydeps/mf27.py", line 292, in load_module
module = self.load_package(fqname, pathname)
File "/usr/local/lib/python2.7/dist-packages/pydeps/mf27.py", line 481, in load_package
self.load_module(fqname, fp, buf, stuff)
File "/usr/local/lib/python2.7/dist-packages/pydeps/py2depgraph.py", line 144, in load_module
self, fqname, fp, pathname, (suffix, mode, kind)
File "/usr/local/lib/python2.7/dist-packages/pydeps/mf27.py", line 311, in load_module
self.scan_code(co, module)
File "/usr/local/lib/python2.7/dist-packages/pydeps/mf27.py", line 427, in scan_code
self._safe_import_hook(name, module, fromlist, level=level)
File "/usr/local/lib/python2.7/dist-packages/pydeps/mf27.py", line 329, in _safe_import_hook
self.import_hook(name, caller, level=level)
File "/usr/local/lib/python2.7/dist-packages/pydeps/py2depgraph.py", line 115, in import_hook
return mf27.ModuleFinder.import_hook(self, name, caller, fromlist, level)
File "/usr/local/lib/python2.7/dist-packages/pydeps/mf27.py", line 131, in import_hook
m = self.load_tail(q, tail)
File "/usr/local/lib/python2.7/dist-packages/pydeps/mf27.py", line 210, in load_tail
m = self.import_module(head, mname, m)
File "/usr/local/lib/python2.7/dist-packages/pydeps/py2depgraph.py", line 137, in import_module
module = mf27.ModuleFinder.import_module(self, partnam, fqname, parent)
File "/usr/local/lib/python2.7/dist-packages/pydeps/mf27.py", line 280, in import_module
m = self.load_module(fqname, fp, pathname, stuff)
File "/usr/local/lib/python2.7/dist-packages/pydeps/py2depgraph.py", line 144, in load_module
self, fqname, fp, pathname, (suffix, mode, kind)
File "/usr/local/lib/python2.7/dist-packages/pydeps/mf27.py", line 311, in load_module
self.scan_code(co, module)
File "/usr/local/lib/python2.7/dist-packages/pydeps/mf27.py", line 427, in scan_code
self._safe_import_hook(name, module, fromlist, level=level)
File "/usr/local/lib/python2.7/dist-packages/pydeps/mf27.py", line 341, in _safe_import_hook
self.import_hook(name, caller, [sub], level=level)
File "/usr/local/lib/python2.7/dist-packages/pydeps/py2depgraph.py", line 115, in import_hook
return mf27.ModuleFinder.import_hook(self, name, caller, fromlist, level)
File "/usr/local/lib/python2.7/dist-packages/pydeps/mf27.py", line 137, in import_hook
self.ensure_fromlist(m, fromlist)
File "/usr/local/lib/python2.7/dist-packages/pydeps/py2depgraph.py", line 173, in ensure_fromlist
self._add_import(getattr(module, sub))
File "/usr/local/lib/python2.7/dist-packages/pydeps/py2depgraph.py", line 123, in _add_import
if module.file or self.include_pylib_all:
AttributeError: 'str' object has no attribute 'file'

Config Options

So I have my package, lets call it colors, which I have installed inside a virtualenv. It depends on external packages like numpy, scipy, and_others. Is there a way in which for colors it explores
all there is to explore, but once it hits external dependencies like numpy it stops, but plots them,
without going inside numpy.

So I can do this when I install my package in a virtualenv in which I am yet to install any external dependency. And I have a file say:

file: foo.py

from colors import purple
from colors.palette import something_pretentious
file: colors.palette

imports numpy

And then when I run pydeps --max-bacon=0 foo.py --include-missing, can I get the same output
when I have numpy installed inside the virtualenv? So for colors it just goes and gets all deps
inside that package, but when it arrives to other packages, it just plots a node for them and then stops.

PS: Great module.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.