Plugins specified with -h are not loaded when using multiprocessing with xdist about pytest HOT 5 CLOSED

Original comment by Yoann Roman (BitBucket: silentsound, GitHub: silentsound):

Alright, here's a sample project to reproduce this. Assuming a clean virtualenv is activated:

{{{
(env)$ tar xvf Temp.tar.gz
(env)$ cd Temp
(env)$ python setup.py develop
}}}

Running the tests without multi-processing:
{{{
(env)$ py.test -p temp.plugin
======================================= test session starts ========================================
platform darwin -- Python 2.6.1 -- pytest-2.0.2
collected 1 items

temp/test_sample.py .

===================================== 1 passed in 0.02 seconds =====================================
}}}

And now with multi-processing:

{{{
(env)$ py.test -p temp.plugin -n 2
======================================= test session starts ========================================
platform darwin -- Python 2.6.1 -- pytest-2.0.2
gw0 [1] / gw1 [1]
scheduling tests via LoadScheduling
F
============================================= FAILURES =============================================
____________________________________ TestController.test_sample ____________________________________
[gw0] darwin -- Python 2.6.1 /sandbox/env/bin/python
self = <temp.test_sample.TestController testMethod=test_sample>

def test_sample(self):

  assert py.test.sample_variable == 'testing'

../Temp/temp/test_sample.py:9:

self = <AliasModule 'py.test' for 'pytest'>, name = 'sample_variable'

def __getattribute__(self, name):

  return getattr(getmod(), name)

E AttributeError: 'module' object has no attribute 'sample_variable'

/env/lib/python2.6/site-packages/py-1.4.2-py2.6.egg/py/_apipkg.py:159: AttributeError
===================================== 1 failed in 0.46 seconds =====================================

}}}

from pytest.

Original comment by Yoann Roman (BitBucket: silentsound, GitHub: silentsound):

If it helps at all, I've narrowed this issue down to line 115 of {{{xdist.remote}}}:

{{{
#!python
config._preparse(args, addopts=False)
}}}

This uses the args passed in from line 217 of {{{xdist.slavemanage}}}, which are pulled on line 207:

{{{
#!python
args = self.config.args
}}}

The problem is that {{{config.args}}} is set to the remaining args, not the full {{{sys.argv[1:]}}}, after the host session has parsed out the options. From line 317 of {{{_pytest.config}}}:

{{{
#!python
def parse(self, args):
# parse given cmdline arguments into this config object.
# Note that this can only be called once per testing process.
assert not hasattr(self, 'args'), (
"can only parse cmdline args at most once per Config object")
self._preparse(args)
self._parser.hints.extend(self.pluginmanager._hints)
args = self._parser.parse_setoption(args, self.option)
if not args:
args.append(py.std.os.getcwd())
self.args = args
}}}

And {{{Config._preparse}}}, which in turn calls {{{self.pluginmanager.consider_preparse(args)}}}, needs the full args to parse out that {{{-p}}} option. Even though the {{{-p}}} option is parsed by {{{OptParse}}}, the {{{PluginManager}}} redoes the parsing manually for some reason.

I can think of a couple of ways to fix this, but I don't know if there was a reason the raw command line options weren't passed on to the slaves.

Hope this helps!

from pytest.

Original comment by holger krekel (BitBucket: hpk42, GitHub: hpk42):

offer a semi-internal method to create a config object in subprocesses
(helps pytest-xdist plugin to fix issue34)

from pytest.

Original comment by holger krekel (BitBucket: hpk42, GitHub: hpk42):

please try to install the dev version with

pip install -i http://pypi.testrun.org -U pytest-xdist

py.test --version should show you that you use the pytest-xdist-1.6.dev3 at least.

and thanks for the precise and fitting analysis!
holger

from pytest.

Original comment by Yoann Roman (BitBucket: silentsound, GitHub: silentsound):

That fixed it!

When will pytest/pytest-xdist be released next? I'm using a pytest plugin for an open-source project and would like multiprocessing working before we release if possible.

Thanks for the quick fix!

from pytest.