pytest-dev / pytest-twisted Goto Github PK

View Code? Open in Web Editor NEW

45.0 30.0 24.0 470 KB

test twisted code with pytest

License: BSD 3-Clause "New" or "Revised" License

Python 100.00%

pytest-twisted's People

Contributors

Stargazers

Watchers

pytest-twisted's Issues

Unhandled exception somewhere within use of gatherResults causes test to hang

Please try this:

import pytest
import pytest_twisted
from twisted.internet.defer import inlineCallbacks, Deferred, gatherResults


@inlineCallbacks
def thingy():
    d1 = Deferred()
    d2 = Deferred()

    yield gatherResults([d1, d2], consumeErrors=True)


@pytest_twisted.inlineCallbacks
def test_thingy():
    result = yield thingy()

This hangs indefinitely for me when I run pytest against it. I was having the problem with python=3.7, pytest=4.3.0 and pytest-twisted=1.9, although I initially encountered it in a context where I was running python=2.7, pytest=3.1.2, pytest-twisted=1.5.

It does not appear to make a difference what consumerErrors is set to. Of course, yielding on the deferreds sequentially and not using gatherResults works fine.

The test hangs until you CTRL-C, where you see something like this:

/usr/local/lib/python2.7/dist-packages/pytest_twisted/plugin.py:75: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

d = <Deferred at 0x7f895b2ab0e0 waiting on Deferred at 0x7f895b2ab710>

def blockon(d):
    current = greenlet.getcurrent()
    assert current is not gr_twisted, "blockon cannot be called from the twisted greenlet"
    result = []

    def cb(r):
        result.append(r)
        if greenlet.getcurrent() is not current:
            current.switch(result)

    d.addCallbacks(cb, cb)
    if not result:
        _result = gr_twisted.switch()
>           assert _result is result, "illegal switch in blockon"
E           AssertionError: illegal switch in blockon

I learned that this error message probably stemmed from an unhandled exception at this post: #4

I tried the fixture suggested recently at that post, but it did not change the problem I was having.

py.test silence assert exceptions when using `unittest.TestCase`

Context

# python --version
Python 2.7.9

# py.test-2.7 --version
This is pytest version 2.6.3, imported from /usr/lib/python2.7/dist-packages/pytest.pyc
setuptools registered plugins:
  pytest-cov-1.8.0 at /usr/lib/python2.7/dist-packages/pytest_cov.pyc
  pytest-twisted-1.5 at <pytest_twisted._loader_plugin object at 0x7fe371017e50>

And pytest-twisted 1.5.

How to reproduce

Create a simple test with defer.inlineCallbacks something like:

import unittest
from twisted.internet import defer

class TestCase(unittest.TestCase):

    @defer.inlineCallbacks
    def test_silent_error(self):
        yield 1
        self.assertTrue(False)

The test will be marked as passed no matter what.

To a workaround this, you have avoid the use of unittest.TestCase.

For instance:

class TestCase:

    @defer.inlineCallbacks
    def test_silent_error(self):
        yield 1
        assert False

How do I use the pytest_twisted.inlineCallbacks decorator in conjunction with fixtures?

How can I pass the necessary fixtures to the decorator or to the function to be decorated?

With this code:

class TestScrapingSite(TestCase):
    @pytest_twisted.inlineCallbacks
    def test_crawler_stats(self, caplog):
        yield 1

raised error

TypeError: test_crawler_stats() missing 1 required positional argument: 'caplog'

Add badges to README.rst

Unless you would specifically prefer not to have them I'll add badges to the README.rst.

@vtitor, I would need you to get these for me. I generally use the SVG badges.
https://www.appveyor.com/docs/status-badges/

Publicly accessible ones are:
Travis:
Python Versions:

I'll add the Python versions to the setup.py so that the last badge will actually show something useful after the next PyPi release.

start twisted greenlet earlier

Hi! At the moment twisted greenlet is being started in the plugin in autoused session-scoped fixture, which means after test generation. This doesn't allow us to await deffereds on earlier phases like pytest_generate_tests or pytest_configure. I wonder if you mind moving greenlet initialization a bit earlier in plugin registration process, e.g. to pytest_configure hook or which one is more appropriate (pytest_addhooks?), so we wouldn't need to do it manually by direct imports from the plugin. Thank you!

Transfer maintenance/ownership of this repository?

As discussed in #10, it has been claimed that @christianmlong is no longer actively maintaining this plugin, and @schmir also does not maintain it anymore.

@vtitor has stepped in and would like to become a maintainer, so I've sent @christianmlong an email asking him if it was OK to make @vtitor the new admin of this repository. We will have to wait at least a month to move this forward though, as stated by our policy:

Repository owners can rest assured that no pytest-dev administrator will ever make releases of your repository or take ownership in any way, except in rare cases where someone becomes unresponsive after months of contact attempts. As stated, the objective is to share maintenance and avoid "plugin-abandon".

@pytest.inlineCallbacks but for async def's

How about a decorator to allow directly writing async def tests instead of having to wrap them with @pytest.inlineCallbacks decorated tests.

https://github.com/altendky/altendpyqt5/pull/10/files#diff-7531f542e8da5192e38a81a441db749e

async def async_await_for_signal():
    timer = altendpyqt5.tests.utils.singleshot_immediate_timer()
    await altendpyqt5.twisted.signal_as_deferred(timer.timeout)

@pytest.inlineCallbacks
def test_await_for_signal():
    yield twisted.internet.defer.ensureDeferred(async_await_for_signal())

Unless maybe I just haven't thought of an easy way to handle this with existing pieces.

Handle non-function scope for async fixtures

@pytest.mark.skip
def test_async_fixture_module_scope(testdir, cmd_opts):
    test_file = """
    from twisted.internet import reactor, defer
    import pytest
    import pytest_twisted

    check_me = 0

    @pytest_twisted.async_yield_fixture(scope="module")
    async def foo():
        global check_me

        if check_me != 0:
            raise Exception('check_me already modified before fixture run')

        check_me = 1

        yield 42

        if check_me != 3:
            raise Exception(
                'check_me not updated properly: {}'.format(check_me),
            )

        check_me = 0

    def test_first(foo):
        global check_me

        assert check_me == 1
        assert foo == 42

        check_me = 2

    def test_second(foo):
        global check_me

        assert check_me == 2
        assert foo == 42

        check_me = 3
    """
    testdir.makepyfile(test_file)
    rr = testdir.run(sys.executable, "-m", "pytest", "-v", *cmd_opts)
    assert_outcomes(rr, {"passed": 2})

Mixing pytest tests and twisted.trial.unitest.TestCase results in exception when running with asyncio reactor

I have a project with tests inheriting from both twisted.trial.unittest.TestCase and tests not inheriting from anything, just plain Python objects. When running without asyncio reactor, they all work fine. When I pass --reactor flag to pytest via pytest-twisted they are failing with "RuntimeError: This event loop is already running".

My expectation is that both trial and non-trial tests will work, because why not? If they worked with default reactor why not work with asyncio reactor.

Steps to reproduce. Take following file:

# test_reactor.py
from twisted.trial.unittest import TestCase


class TestCrawlCase: 
    def test_if_parametrize_works(self):
        assert 1 == 1

    def test_another(self):
        assert 1 == 1


class AnotherTest(TestCase):
    def test_another_test_case(self):
        assert 2 == 2

Now run this file without any arguments:

ython -m pytest test_reactor.py -vsx
============================================================================================================ test session starts =============================================================================================================
platform linux -- Python 3.9.4, pytest-6.2.3, py-1.11.0, pluggy-0.13.1 -- /home/.../bin/python
cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/.../.hypothesis/examples')
benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /home/pawel/Documents/projects/Spiders, configfile: pytest.ini
plugins: Faker-8.1.3, hypothesis-6.10.1, benchmark-3.4.1, twisted-1.13.4
collected 3 items                                                                                                                                                                                                                            

test_reactor.py::TestCrawlCase::test_if_parametrize_works PASSED
test_reactor.py::TestCrawlCase::test_another PASSED
test_reactor.py::AnotherTest::test_another_test_case PASSED

Now run same file with --reactor=asyncio

 python -m pytest test_reactor.py -vsx --reactor=asyncio
============================================================================================================ test session starts =============================================================================================================
platform linux -- Python 3.9.4, pytest-6.2.3, py-1.11.0, pluggy-0.13.1 -- /home/pawel/.pyenv/versions/../bin/python
cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home...')
benchmark: 3.4.1 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /home/.../directory, configfile: pytest.ini
plugins: Faker-8.1.3, hypothesis-6.10.1, benchmark-3.4.1, twisted-1.13.4
collected 3 items                                                                                                                                                                                                                            

test_reactor.py::TestCrawlCase::test_if_parametrize_works PASSED
test_reactor.py::TestCrawlCase::test_another PASSED
test_reactor.py::AnotherTest::test_another_test_case FAILED
test_reactor.py::AnotherTest::test_another_test_case ERROR

This fails with:

=================================================================================================================== ERRORS ===================================================================================================================
__________________________________________________________________________________________ ERROR at teardown of AnotherTest.test_another_test_case ___________________________________________________________________________________________

self = <test_reactor.AnotherTest testMethod=test_another_test_case>, result = <TestCaseFunction test_another_test_case>

    def _classCleanUp(self, result):
        try:
>           util._Janitor(self, result).postClassCleanup()


self = <_UnixSelectorEventLoop running=True closed=False debug=False>

    def _check_running(self):
        if self.is_running():
>           raise RuntimeError('This event loop is already running')
E           RuntimeError: This event loop is already running

../../../.pyenv/versions/3.9.4/lib/python3.9/asyncio/base_events.py:578: RuntimeError

Update AppVeyor references to pytestbot builds

https://ci.appveyor.com/project/pytestbot/pytest-twisted

Build badge
GitHub required checks

Trouble with constructing a QWidget

I'm digging, more details to follow.

investigate using corotwine

corotwine is a greenlet integration for twisted,

i cant exactly asses its value for writing testsuites

but it comes with a blockOn primitive which means less code in the plugin

Add Python 3.7 support

Add Python 3.7 support as soon as it is supported by Travis CI.
See the upstream issue travis-ci/travis-ci#9815.

[Discussion] Customisibility for pytest-twisted

I have a fairly specific use case with my project:
I'm using both Twisted (with asyncioreactor setup) and native asyncio awaits somewhat interchangeably, but when it came to testing, I've gotten into troubles with pytest_twisted's and pytest_asyncio's incompatibility.

The issue is that when using pytest_twisted it becomes impossible for pytest_asyncio to make a new loop for every test, since it's basically running inside a global twisted loop.

So basically, for my use case I would want to have pytest-twisted applied only in tests I need it to be applied and making a separate loop for every test and getting rid of it after test is done.

I've dealt with this by doing some tinkering, which resulted in:

introducing pytest.mark.twisted that is checked by pytest_pyfunc_call that skips if the mark is absent;
adding a --twisted-marked-only cmdopt that pytest_collection_modifyitems checks and if it's False (which it is by default to preserve current behaviour) adds the twisted mark to any test that doesn't have it already;
adding a --twisted-scope cmdopt and a twisted_greenlet_fixture_factory function that is called in pytest_configure that depending on the scope:
- behaves as if it wasn't there (except the resulting fixture isn't autouse) if scope="session" (default);
- makes a fixture with the specified scope that dels twisted.internet.reactor before and after creating a new reactor and mocks it with it's created reactor for the users otherwise;
in pytest_runtest_setup adding the twisted_greenlet fixture for every test that has the pytest.mark.twisted (like autouse, that you can opt out of :з ).

I'll make a pull request for it, but the real question is:
Is it of any use for anybody except me and could this the direction the project goes in?

Maintenance handoff

#34 (comment)

@altendky Unfortunately, I do not have enough time to maintain the repository. Do you want to take over maintainership?

@vtitor, I'll do what I can. I guess we'll need to get me full admin rights (or whatever they are called) to this repository and PyPI? I see that last time @nicoddemus was involved (#11). If there are other points we can add them below as well.

PyPI rights for @altendky
GitHub rights for @altendky
AppVeyor
Travis

Black line length

Black defaults to a line length of 88 and reformats some stuff because of that. Shall we configure it to 79?

pyproject.toml:

[tool.black]
line-length = 79

release new version with pytest 4.1 compatibility

Would you please release a new version based on master? It would be great to run pip install pytest-twisted and have a version that works with the latest pytest 4.1.

Proper ordering of async fixture setup and teardown including inter-fixture dependencies

Follow up on #37

Support no-call async_fixture and async_yield_fixture to align with pytest.fixture

Currently, you must call pytest_twisted.async_fixture and pytest_twisted.async_yield_fixture, and (until #74 fixes it) the readme demonstrates no-call semantics. I propose allowing this form by mimicking pytest.fixtures's behavior, to improve consistency and UX.

It looks like pytest.fixture treats its first argument as either the fixture scope name if it's a str, and otherwise assumes it's the fixture function:

https://github.com/pytest-dev/pytest/blob/fd1a51a23fa687cf344f3506dff6cde0166faf2c/src/_pytest/fixtures.py#L1029-L1068

At first glance, I think this is a reasonable approach, and should be relatively straightforward to implement.

missing license

Would be great to include a license file in git and also in the tar-ball. Setup.py mentions BSD, but there are several version (2-clause, 3-clause, etc). Having the LICENSE in the tar-ball also makes packaging for linux distributions easier (can't currently update the openSUSE package because of this issue).

Setup daily builds with actual CI (latest release, prerelease, and vcs deps)

Tests fail on Python 3.10 due to deprecation of threading.currentThread()

When running the tests on Python 3.10, I get failures because of an extra warning in the Pytest output:

/usr/lib/python3.10/site-packages/twisted/python/threadable.py:107: DeprecationWarning: currentThread() is deprecated, use current_thread() instead

The currentThread() function was deprecated in Python 3.10.

Don't run unmarked tests in the reactor greenlet.

I have a project using pytest-twisted, and I'd like to write some tests using hypothesis' stateful testing. Since the control-flow of stateful testing is handled by hypothesis, I use pytest_twisted.blockon in the tests to interact with twisted.

However, this doesn't work by default, as pytest-twisted forces every test function to run in the reactor greenlet. There is a comment in the pytest_pyfunc_call hook suggesting this change:

pytest-twisted/pytest_twisted.py

Lines 357 to 363 in cc03b11

    
           def pytest_pyfunc_call(pyfuncitem): 
        
               """Interface to async test call handler.""" 
        
               # TODO: only handle 'our' tests?  what is the point of handling others? 
        
               #       well, because our interface allowed people to return deferreds 
        
               #       from arbitrary tests so we kinda have to keep this up for now 
        
               _run_inline_callbacks(_async_pytest_pyfunc_call, pyfuncitem) 
        
               return not None

This is perhaps a duplicate of #64.

What is the public interface? (let me know what you actually use)

With everything in one file, being a module distribution (rather than package), and having only the readme as documentation I'm left a little unclear about what is 'public interface' and what isn't. I'd like to make this more explicit so that going forward we don't have to think about this on a case by case basis every time we feel like tweaking something. (also, maybe I want a package with versioneer and explicit public interface and tests inside like I'm used to...)

The readme references:

pytest options
- -p no:twisted
- --reactor
pytest fixtures
- twisted_greenlet
import pytest
- Not in the readme anymore but they were as of v1.6 (2018.01.11) and have no deprecation notices
- @inlineCallbacks
- blockon()
import pytest_twisted
- init_default_reactor()
- init_qt5_reactor()
- @pytest_twisted.inlineCallbacks
- blockon()

The remaining items are:

'public'
- blockon_default()
- block_from_thread()
- init_twisted_greenlet()
- stop_twisted_greenlet()
- reactor_installers
'private'
- _config
- _instances
- _install_reactor()
pytest_*
- pytest_pyfunc_call()
- pytest_addoption()
- pytest_configure()
- _pytest_pyfunc_call()

Maybe we can handle multiple calls to pytest.main()

When running pytest from Python code you might run it twice in the same process and this produces a twisted.internet.error.ReactorNotRestartable error. Maybe we can sensibly deal with this situation.

Pytest 5.4.0 breaks tests

It seems to happen in the cleanup stage after all the tests of a TestCase, and marks the last test of that TestCase as a failure.

===================================================================== test session starts ======================================================================
platform linux -- Python 3.6.7, pytest-5.4.0, py-1.8.0, pluggy-0.13.0
rootdir: /home/adrian/proxectos/scrapy, inifile: pytest.ini
plugins: pylint-0.14.0, twisted-1.12, forked-1.0.1, xdist-1.26.0, cov-2.8.1
collected 4 items                                                                                                                                              

tests/test_command_fetch.py F...                                                                                                                         [100%]

=========================================================================== FAILURES ===========================================================================
____________________________________________________________________ FetchTest.test_headers ____________________________________________________________________

self = <Process pid=None status=0>

    def reapProcess(self):
        """
        Try to reap a process (without blocking) via waitpid.
    
        This is called when sigchild is caught or a Process object loses its
        "connection" (stdout is closed) This ought to result in reaping all
        zombie processes, since it will be called twice as often as it needs
        to be.
    
        (Unfortunately, this is a slightly experimental approach, since
        UNIX has no way to be really sure that your process is going to
        go away w/o blocking.  I don't want to block.)
        """
        try:
            try:
>               pid, status = os.waitpid(self.pid, os.WNOHANG)
E               TypeError: an integer is required (got type NoneType)

/home/adrian/.local/share/virtualenvs/scrapy-R00A9LxV/lib/python3.6/site-packages/twisted/internet/process.py:299: TypeError

During handling of the above exception, another exception occurred:

self = <twisted.internet.epollreactor.EPollReactor object at 0x7fe23908a828>, selectable = <twisted.internet.posixbase._SIGCHLDWaker object at 0x7fe2365af358>
fd = 15, event = 1

    def _doReadOrWrite(self, selectable, fd, event):
        """
        fd is available for read or write, do the work and raise errors if
        necessary.
        """
        why = None
        inRead = False
        if event & self._POLL_DISCONNECTED and not (event & self._POLL_IN):
            # Handle disconnection.  But only if we finished processing all
            # the pending input.
            if fd in self._reads:
                # If we were reading from the descriptor then this is a
                # clean shutdown.  We know there are no read events pending
                # because we just checked above.  It also might be a
                # half-close (which is why we have to keep track of inRead).
                inRead = True
                why = CONNECTION_DONE
            else:
                # If we weren't reading, this is an error shutdown of some
                # sort.
                why = CONNECTION_LOST
        else:
            # Any non-disconnect event turns into a doRead or a doWrite.
            try:
                # First check to see if the descriptor is still valid.  This
                # gives fileno() a chance to raise an exception, too.
                # Ideally, disconnection would always be indicated by the
                # return value of doRead or doWrite (or an exception from
                # one of those methods), but calling fileno here helps make
                # buggy applications more transparent.
                if selectable.fileno() == -1:
                    # -1 is sort of a historical Python artifact.  Python
                    # files and sockets used to change their file descriptor
                    # to -1 when they closed.  For the time being, we'll
                    # continue to support this anyway in case applications
                    # replicated it, plus abstract.FileDescriptor.fileno
                    # returns -1.  Eventually it'd be good to deprecate this
                    # case.
                    why = _NO_FILEDESC
                else:
                    if event & self._POLL_IN:
                        # Handle a read event.
>                       why = selectable.doRead()

/home/adrian/.local/share/virtualenvs/scrapy-R00A9LxV/lib/python3.6/site-packages/twisted/internet/posixbase.py:614: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/home/adrian/.local/share/virtualenvs/scrapy-R00A9LxV/lib/python3.6/site-packages/twisted/internet/posixbase.py:227: in doRead
    process.reapAllProcesses()
/home/adrian/.local/share/virtualenvs/scrapy-R00A9LxV/lib/python3.6/site-packages/twisted/internet/process.py:63: in reapAllProcesses
    process.reapProcess()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Process pid=None status=0>

    def reapProcess(self):
        """
        Try to reap a process (without blocking) via waitpid.
    
        This is called when sigchild is caught or a Process object loses its
        "connection" (stdout is closed) This ought to result in reaping all
        zombie processes, since it will be called twice as often as it needs
        to be.
    
        (Unfortunately, this is a slightly experimental approach, since
        UNIX has no way to be really sure that your process is going to
        go away w/o blocking.  I don't want to block.)
        """
        try:
            try:
                pid, status = os.waitpid(self.pid, os.WNOHANG)
            except OSError as e:
                if e.errno == errno.ECHILD:
                    # no child process
                    pid = None
                else:
                    raise
        except:
>           log.msg('Failed to reap %d:' % self.pid)
E           TypeError: %d format: a number is required, not NoneType

/home/adrian/.local/share/virtualenvs/scrapy-R00A9LxV/lib/python3.6/site-packages/twisted/internet/process.py:307: TypeError
--------------------------------------------------------------------- Captured stderr call ---------------------------------------------------------------------
Unhandled Error
Traceback (most recent call last):
  File "/home/adrian/.local/share/virtualenvs/scrapy-R00A9LxV/lib/python3.6/site-packages/twisted/python/log.py", line 103, in callWithLogger
    return callWithContext({"system": lp}, func, *args, **kw)
  File "/home/adrian/.local/share/virtualenvs/scrapy-R00A9LxV/lib/python3.6/site-packages/twisted/python/log.py", line 86, in callWithContext
    return context.call({ILogContext: newCtx}, func, *args, **kw)
  File "/home/adrian/.local/share/virtualenvs/scrapy-R00A9LxV/lib/python3.6/site-packages/twisted/python/context.py", line 122, in callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
  File "/home/adrian/.local/share/virtualenvs/scrapy-R00A9LxV/lib/python3.6/site-packages/twisted/python/context.py", line 85, in callWithContext
    return func(*args,**kw)
--- <exception caught here> ---
  File "/home/adrian/.local/share/virtualenvs/scrapy-R00A9LxV/lib/python3.6/site-packages/twisted/internet/posixbase.py", line 614, in _doReadOrWrite
    why = selectable.doRead()
  File "/home/adrian/.local/share/virtualenvs/scrapy-R00A9LxV/lib/python3.6/site-packages/twisted/internet/posixbase.py", line 227, in doRead
    process.reapAllProcesses()
  File "/home/adrian/.local/share/virtualenvs/scrapy-R00A9LxV/lib/python3.6/site-packages/twisted/internet/process.py", line 63, in reapAllProcesses
    process.reapProcess()
  File "/home/adrian/.local/share/virtualenvs/scrapy-R00A9LxV/lib/python3.6/site-packages/twisted/internet/process.py", line 307, in reapProcess
    log.msg('Failed to reap %d:' % self.pid)
builtins.TypeError: %d format: a number is required, not NoneType

=================================================================== short test summary info ====================================================================
FAILED tests/test_command_fetch.py::FetchTest::test_headers - TypeError: %d format: a number is required, not NoneType
================================================================= 1 failed, 3 passed in 4.51s ==================================================================

Review trial features for checking 'health' of... reactor, ...?

It was suggested that trial provides various checks after each test (or each test class?) to verify the reactor is in a good state. Let's review those features and see how they can be applied. Perhaps a disable-able auto-use fixture?

keep compatibility with latest twisted versions

Do we really need our own test decorators vs. using t.i.d.inlineCallbacks

pytest-twisted/pytest_twisted.py

Lines 104 to 106 in 16abc59

    
           @decorator.decorator 
        
           def inlineCallbacks(fun, *args, **kw): 
        
               return defer.inlineCallbacks(fun)(*args, **kw)

likewise for other test decorators

keep track of the returnvalue issue of pytest

https://bitbucket.org/hpk42/pytest/issue/207/remember-return-returnvalues-of-test

once done, the plugin no longer has to keep a copy of the call a test function machinery

Add Windows automated testing (AppVeyor)

Any interest in adding AppVeyor builds for testing on Windows? I'd be happy to do a PR but I expect someone else should have an AppVeyor account and connect to this repo. I guess there would also be a step to add it to the GitHub checks.

Missing git tag for 1.6 release

It would be nice to keep PyPI releases and git tags in sync :)

Please make a new release

We are working on the asyncio support in Scrapy and need the asyncio reactor support in pytest-twisted, so can you please release a version with it included so that we could install it from PyPI during testing?

Thank you.

Unhandled Exceptions

I ran across the following problem: if you have an unhandled exception somewhere in your Twisted stuff (e.g. a missing errback handler), pytest-twisted "hangs" and needs a ctrl-c to stop the test -- at which point you get a traceback into pytest-twisted internals.

Obviously, having a Deferred without an errback is a problem, but if pytest-twisted can do something nicer here, that'd be amazing. (FWIW, _result ends up being None in this case). I don't know enough about greenlets nor pytest to suggest something.

I do have a SSCCE that I detailed in this blog post: https://meejah.ca/pytest-twisted-blockon (and I hereby declare that's Unlicensed if you want any of it).

pytest-twisted 1.8 not playing well with pytest 4.1

Running tests
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/home/oheddela/.virtualenvs/default/lib/python3.6/site-packages/_pytest/main.py", line 203, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/home/oheddela/.virtualenvs/default/lib/python3.6/site-packages/_pytest/main.py", line 242, in _main
INTERNALERROR>     config.hook.pytest_collection(session=session)
INTERNALERROR>   File "/home/oheddela/.virtualenvs/default/lib/python3.6/site-packages/pluggy/hooks.py", line 284, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/home/oheddela/.virtualenvs/default/lib/python3.6/site-packages/pluggy/manager.py", line 67, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/home/oheddela/.virtualenvs/default/lib/python3.6/site-packages/pluggy/manager.py", line 61, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "/home/oheddela/.virtualenvs/default/lib/python3.6/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/home/oheddela/.virtualenvs/default/lib/python3.6/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/home/oheddela/.virtualenvs/default/lib/python3.6/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/home/oheddela/.virtualenvs/default/lib/python3.6/site-packages/_pytest/main.py", line 252, in pytest_collection
INTERNALERROR>     return session.perform_collect()
INTERNALERROR>   File "/home/oheddela/.virtualenvs/default/lib/python3.6/site-packages/_pytest/main.py", line 466, in perform_collect
INTERNALERROR>     self.config.pluginmanager.check_pending()
INTERNALERROR>   File "/home/oheddela/.virtualenvs/default/lib/python3.6/site-packages/pluggy/manager.py", line 250, in check_pending
INTERNALERROR>     % (name, hookimpl.plugin),
INTERNALERROR> pluggy.manager.PluginValidationError: unknown hook 'pytest_namespace' in plugin <module 'pytest_twisted' from '/home/oheddela/.virtualenvs/default/lib/python3.6/site-packages/pytest_twisted.py'>

Stop putting stuff in the pytest namespace

I personally dislike the use of the pytest namespace as a place to collapse other namespaces. @pytest_twisted.inlineCallbacks seems perfectly fine instead of @pytest.inlineCallbacks. Presently #34 is adding two more items to pytest_namespace() for consistency, but I'd like to consider not adding those and also deprecating the existing pytest.inlineCallbacks and pytest.blockon. Or at least deciding if we will or won't add anything more.

Use coverage.py

The reason behind disabling the plugin by default

Please see my description of my previous problem here.

I wanted to ask why do we have to explicitly enable this plugin? Based on what's written here:

If a plugin is installed, pytest automatically finds and integrates it, there is no need to activate it.

But having to activate this plugin separately goes against this.

AppVeyor doesn't seem to be running tests

https://ci.appveyor.com/project/vtitor/pytest-twisted/build/1.0.26/job/mh8dkwe724scjo8u

[00:00:00] Build started
[00:00:00] if (Get-ChildItem Env:ENABLE_RDP -ErrorAction SilentlyContinue) {iex ((new-object net.webclient).DownloadString('https://raw.githubusercontent.com/appveyor/ci/master/scripts/enable-rdp.ps1'))} else {echo RDP not enabled}
[00:00:00] RDP
[00:00:00] not
[00:00:00] enabled
[00:00:00] echo "TOXENV- %TOXENV%"
[00:00:00] "TOXENV- py35"
[00:00:00] git clone -q https://github.com/pytest-dev/pytest-twisted.git C:\projects\pytest-twisted
[00:00:01] git fetch -q origin +refs/pull/26/merge:
[00:00:02] git checkout -qf FETCH_HEAD
[00:00:02] Running Install scripts
[00:00:02] pip install -U git+https://github.com/pypa/virtualenv@e8163e83a92c9098f51d390289323232ece15e3b
[00:00:04] Collecting git+https://github.com/pypa/virtualenv@e8163e83a92c9098f51d390289323232ece15e3b
[00:00:04]   Cloning https://github.com/pypa/virtualenv (to e8163e83a92c9098f51d390289323232ece15e3b) to c:\users\appveyor\appdata\local\temp\1\pip-u7rha7-build
[00:00:14]   Could not find a tag or branch 'e8163e83a92c9098f51d390289323232ece15e3b', assuming commit.
[00:00:15] Installing collected packages: virtualenv
[00:00:15]   Found existing installation: virtualenv 15.0.1
[00:00:15]     Uninstalling virtualenv-15.0.1:
[00:00:15]       Successfully uninstalled virtualenv-15.0.1
[00:00:15]   Running setup.py install for virtualenv: started
[00:00:15]     Running setup.py install for virtualenv: finished with status 'done'
[00:00:15] Successfully installed virtualenv-15.2.0.dev0
[00:00:16] virtualenv -p "%PYTHON%\\python.exe" "%VENV%"
[00:00:24] Using base prefix 'C:\\Python35'
[00:00:24] New python executable in C:\projects\pytest-twisted\venv\Scripts\python.exe
[00:00:33] Installing setuptools, pip, wheel...done.
[00:00:33] Running virtualenv with interpreter C:\Python35\\python.exe
[00:00:33] %VENV%\Scripts\activate
[00:00:33] pip install git+https://github.com/pypa/virtualenv@e8163e83a92c9098f51d390289323232ece15e3b
[00:00:33] Collecting git+https://github.com/pypa/virtualenv@e8163e83a92c9098f51d390289323232ece15e3b
[00:00:33]   Cloning https://github.com/pypa/virtualenv (to e8163e83a92c9098f51d390289323232ece15e3b) to c:\users\appveyor\appdata\local\temp\1\pip-ghz43u1e-build
[00:00:40]   Could not find a tag or branch 'e8163e83a92c9098f51d390289323232ece15e3b', assuming commit.
[00:00:41] Installing collected packages: virtualenv
[00:00:41]   Running setup.py install for virtualenv: started
[00:00:41]     Running setup.py install for virtualenv: finished with status 'done'
[00:00:41] Successfully installed virtualenv-15.2.0.dev0
[00:00:41] pip install tox
[00:00:42] Collecting tox
[00:00:42]   Downloading tox-2.9.1-py2.py3-none-any.whl (73kB)
[00:00:42] Requirement already satisfied: virtualenv>=1.11.2; python_version != "3.2" in c:\projects\pytest-twisted\venv\lib\site-packages (from tox)
[00:00:42] Collecting pluggy<1.0,>=0.3.0 (from tox)
[00:00:42]   Downloading pluggy-0.6.0.tar.gz
[00:00:43] Collecting six (from tox)
[00:00:43]   Downloading six-1.11.0-py2.py3-none-any.whl
[00:00:43] Collecting py>=1.4.17 (from tox)
[00:00:43]   Downloading py-1.5.2-py2.py3-none-any.whl (88kB)
[00:00:43] Building wheels for collected packages: pluggy
[00:00:43]   Running setup.py bdist_wheel for pluggy: started
[00:00:43]   Running setup.py bdist_wheel for pluggy: finished with status 'done'
[00:00:43]   Stored in directory: C:\Users\appveyor\AppData\Local\pip\Cache\wheels\df\44\8e\e136760ae525eac46b3e3db643ef58ff1753177b5a722b0c96
[00:00:43] Successfully built pluggy
[00:00:43] Installing collected packages: pluggy, six, py, tox
[00:00:44] Successfully installed pluggy-0.6.0 py-1.5.2 six-1.11.0 tox-2.9.1
[00:00:44] where tox
[00:00:44] C:\projects\pytest-twisted\venv\Scripts\tox.exe
[00:00:45] tox --version
[00:00:45] 2.9.1 imported from c:\projects\pytest-twisted\venv\lib\site-packages\tox\__init__.py
[00:00:45] tox
[00:00:46] GLOB sdist-make: C:\projects\pytest-twisted\setup.py
[00:00:46] py35 create: C:\projects\pytest-twisted\.tox\py35
[00:00:53] py35 installdeps: greenlet, pytest, twisted
[00:01:22] py35 inst: C:\projects\pytest-twisted\.tox\dist\pytest-twisted-1.7.1.zip
[00:01:25] py35 installed: attrs==17.4.0,Automat==0.6.0,colorama==0.3.9,constantly==15.1.0,decorator==4.2.1,greenlet==0.4.13,hyperlink==18.0.0,idna==2.6,incremental==17.5.0,pluggy==0.6.0,py==1.5.2,pytest==3.4.2,pytest-twisted==1.7.1,six==1.11.0,Twisted==17.9.0,zope.interface==4.4.3
[00:01:25] py35 runtests: PYTHONHASHSEED='420'
[00:01:25] ___________________________________ summary ___________________________________
[00:01:25]   py35: commands succeeded
[00:01:25]   congratulations :)
[00:01:25] if (Get-ChildItem Env:ENABLE_RDP -ErrorAction SilentlyContinue) {$blockRdp = $true; iex ((new-object net.webclient).DownloadString('https://raw.githubusercontent.com/appveyor/ci/master/scripts/enable-rdp.ps1'))} else {echo RDP not enabled}
[00:01:25] RDP
[00:01:25] not
[00:01:25] enabled
[00:01:25] Build success

Deprecate inlineCallbacks and blockon in pytest namespace

My preference... but it's pytest's as well #41 (comment). Let's do it.

PyPy3 failing CI on Windows

actions/setup-python#151
tox-dev/tox#1704
https://foss.heptapod.net/pypy/pypy/-/issues/3331

pytest_configure is specified twice

My colleague noticed that pytest_configure function is defined twice in pytest_twisted.py and thus pytest.inlineCallbacks and pytest.blockon attributes are not set.

pytest_twisted hides erroneous yield from non-deferred objects

% cat a.py
from pytest_twisted import inlineCallbacks

def do_io():
    return None # should return Deferred

@inlineCallbacks
def test_foo():
    yield do_io()

% pytest ./a.py
============================================================================================================= test session starts =============================================================================================================
platform linux -- Python 3.7.3, pytest-4.6.3, py-1.8.0, pluggy-0.12.0
rootdir: /home/yac
plugins: twisted-1.10
collected 1 item

a.py .                                                                                                                                                                                                                                  [100%]

========================================================================================================== 1 passed in 0.02 seconds ===========================================================================================================

Figure out how to handle asyncioreactor on windows: ProactorEventLoop, loop policies, py38 oh my!

Ref #75 (comment) for the gory details, but I'll try to summarize, as there are a few similar yet distinct issues here. First...

The Background

Twisted's AsyncioSelectorReactor uses asyncio.get_event_loop if no event loop is supplied to it explicitly. This is reasonable everywhere... except Windows, because it's the only platform with a non-"selector" event loop implementation in the stdlib: ProactorEventLoop, which is incompatible with AsyncioSelectorReactor. There does not appear to be a Twisted implementation of a proactor-friendly asyncio reactor (although Twisted has a normal IOCP-capable reactor).
Before Python 3.8, the default event loop on windows is a SelectorEventLoop, so users would have to change the event loop policy to notice anything wrong.
Python 3.8 changed that default to ProactorEventLoop, so now without any user input, Twisted's AsyncioSelectorReactor is broken by way of NotImplementedError from the (unimplemented) add_reader method. To avoid this, users must either explicitly set the event loop policy back to one that gives selector loops, or create a loop distinct from the event loop policy (which I have a feeling isn't supported with the built-in policies) and pass it explicitly into AsyncioSelectorReactor

Pytest-twisted needs to react (haha) to this in some fashion, in some way better than "belch NotImplementedError from the depths of twisted and asyncio", a la

<pytest_twisted.init_asyncio_reactor is called>

  File "c:\projects\pytest-twisted\.tox\py38-asyncioreactor\lib\site-packages\twisted\internet\asyncioreactor.py", line 320, in install
    reactor = AsyncioSelectorReactor(eventloop)
  File "c:\projects\pytest-twisted\.tox\py38-asyncioreactor\lib\site-packages\twisted\internet\asyncioreactor.py", line 69, in __init__
    super().__init__()
  File "c:\projects\pytest-twisted\.tox\py38-asyncioreactor\lib\site-packages\twisted\internet\base.py", line 571, in __init__
    self.installWaker()
  File "c:\projects\pytest-twisted\.tox\py38-asyncioreactor\lib\site-packages\twisted\internet\posixbase.py", line 286, in installWaker
    self.addReader(self.waker)
  File "c:\projects\pytest-twisted\.tox\py38-asyncioreactor\lib\site-packages\twisted\internet\asyncioreactor.py", line 151, in addReader
    self._asyncioEventloop.add_reader(fd, callWithLogger, reader,
  File "C:\Python38-x64\Lib\asyncio\events.py", line 501, in add_reader
    raise NotImplementedError
NotImplementedError

The Battleground

One option is to unconditionally switch the policy:

def init_asyncio_reactor():
    if sys.platform == 'win32':
        if sys.version_info >= (3, 8):
            # If twisted releases a fix/workaround we can check that version too
            # https://twistedmatrix.com/trac/ticket/9766
            import asyncio

            selector_policy = asyncio.WindowsSelectorEventLoopPolicy()
            asyncio.set_event_loop_policy(selector_policy)

This certainly does the job, but is rather heavy-handed. @cdunklau is not a huge fan, and suggested feature detection over version checking, and refusing the temptation to guess:

def init_asyncio_reactor():
    from twisted.internet import asyncioreactor
    try:
        _install_reactor(
            reactor_installer=asyncioreactor.install,
            reactor_type=asyncioreactor.AsyncioSelectorReactor,
        )
    except NotImplementedError as e:
        raise RuntimeError(
            "Failed to install AsyncioSelectorReactor. "
            "If you're on windows and python 3.8+, you might need to change "
            "the event loop policy to WindowsSelectorEventLoopPolicy."
        ) from e  # modulo `six.raise_from` for py2 support

This is not pretty, as @altendky rightfully points out: "NotImplementedError isn't very specific".

The Road to Reconciliation

Once #75 lands, we should have a working (if godawfully slow) CI rig for experimentation via PRs, and maybe get some speedup by switching to another provider... then we'll want to start off by adding some test cases that validate specifically for Windows that:

A user is able to reasonably change the event loop policy before pytest-twisted has a chance to make any decisions about it. This point is super important, as it will tell us what kinds of solutions actually bring value.

I have a feeling that because the misbehaving function winds up getting called by the pytest_configure hook, a user might not even be able to swap the policy before the fault is triggered.
My interpretation of that hook's reference is that it is call first for plugins (us) and then for the initial conftest file (the reasonable place for the user to make the tweak)... assuming that's correct, in order to allow the user even the chance, some kind of deferral (haha) mechanism for the reactor_installers[config.getoption("reactor")]() call would become necessary.

The default event loop policy works with pytest-twisted in Py 3 < 3.8, and on 3.8+ either breaks in a friendlyish way or magically works (there's my lack of objectivity showing again 😬), depending on the solution chosen.
Explicitly setting the policy to asyncio.WindowsProactorEventLoopPolicy on all supported 3.x Pythons either breaks in a friendlyish way or is magically worked around.

async/await fixtures

I haven't done anything yet but maybe we end up needing something like pytest-dev/pytest-asyncio#45

Missing git tag for 1.13.3 release

It would be nice to keep PyPI releases and git tags in sync :)

Add support for using twisted clock in tests

When testing twisted based functions, it is often desirable to be able to test with deterministic and controllable passage of time. Twisted's task.Clock provides this. If the function under test is using callLater or similar methods to create a delay, time can be fast-forwarded when the reactor is idle.

I'd like to propose adding functionality to pytest-twisted that allows for using Clock to accelerate passage of time during testing. Coincidentally it will be comparable to pytest-trio's autojump_clock, see https://pytest-trio.readthedocs.io/en/latest/quickstart.html#trio-s-magic-autojump-clock

I am currently working on some concepts and will discuss them here.

Add deployment to PyPI using Travis

Here's how pytest does it: https://github.com/pytest-dev/pytest/blob/a4c426b1a891a22ae1b63d0a0fa2dcdf690e69db/.travis.yml#L61-L75

new reactor for each test, like pytest-asyncio

Pytest process hanging

Hi and thanks for this pytest plugin.
I'm using these packages versions

pytest==3.0.5
pytest-twisted==1.5
Twisted==15.4.0

and I got the tests working now, however after they are done the pytest process hangs there forever, and I actually have to do a kill -9 to terminate it..
Any idea why this would happen?
Let me know if you need any more information, thanks

========================================================================================== 154 passed, 2 skipped, 2 pytest-warnings in 49.20 seconds ==========================================================================================


^C^C^C

	def pytest_pyfunc_call(pyfuncitem):
	"""Interface to async test call handler."""
	# TODO: only handle 'our' tests? what is the point of handling others?
	# well, because our interface allowed people to return deferreds
	# from arbitrary tests so we kinda have to keep this up for now
	_run_inline_callbacks(_async_pytest_pyfunc_call, pyfuncitem)
	return not None

	@decorator.decorator
	def inlineCallbacks(fun, args, *kw):
	return defer.inlineCallbacks(fun)(args, *kw)