Writing tests against testdir.runpytest() and local plugin (no tox) about pytest-check HOT 6 CLOSED

Just reporting a little bit of progress with this, the following test loads the plugin via args into testdir.runpytest, but, interestingly, the test report sees that two tests passed, versus two tests failing.

def test_mix_of_checks_and_asserts_are_reported_ok(testdir):
    testdir.makepyfile(
        """
        from pytest_check import check

        def test_foo():
            with check:
                assert 4 == 5

        def test_mix():
            with check:
                assert 1 == 2
            # assert 2 == 3    
        """
    )
    result = testdir.runpytest('--trace-config', '-ppytest_check')
    result.assert_outcomes(failed=2, passed=0)

PYTHONPATH=src envs/default/bin/pytest tests/test_check_context_manager.py::test_mix_of_checks_and_asserts_are_reported_ok

Outputs:

collected 2 items

test_mix_of_checks_and_asserts_are_reported_ok.py ..                     [100%]

============================== 2 passed in 0.02s ===============================
==================================== short test summary info ====================================
FAILED tests/test_check_context_manager.py::test_mix_of_checks_and_asserts_are_reported_ok - A...
======================================= 1 failed in 0.06s =======================================

The output of the embedded run appears to be a clue: "2 passed in 0.02s" (should be two failed).

from pytest-check.

@lhayhurst Could you include the top header with versions?
With Python 3.9.1, pytest 6.2.1, on mac, I'm able to run just with pytest:

(pytest-check) $ pytest
============================= test session starts ==============================
platform darwin -- Python 3.9.1, pytest-6.2.1, py-1.10.0, pluggy-0.13.1
rootdir: /Users/okken/projects/pytest-check
plugins: pytest_check-0.4.0
collected 37 items                                                             

tests/test_check.py .........................                            [ 67%]
tests/test_check_context_manager.py ......                               [ 83%]
tests/test_check_errors.py ..                                            [ 89%]
tests/test_check_fixture.py .                                            [ 91%]
tests/test_check_func_decorator.py ...                                   [100%]

============================== 37 passed in 0.97s ==============================

from pytest-check.

@lhayhurst Also, do you have your local version of pytest-check installed while running the tests?

tox will create a new virtual env, then install the local pytest-check package, then run tests.
Without tox, you have to install pytest-check yourself.

After creating a new virtual environment, from the pytest-check directory, I do pip install -e ..

my pip list and pip freeze looks like this:

(pytest-check) $ pip list --not-required   
Package      Version
------------ -------
pip          20.3.3
pytest-check 0.4.0
setuptools   46.4.0
wheel        0.34.2
(pytest-check) $ pip freeze                
attrs==20.3.0
iniconfig==1.1.1
packaging==20.8
pluggy==0.13.1
py==1.10.0
pyparsing==2.4.7
pytest==6.2.1
pytest-check @ file:///Users/okken/projects/pytest-check
toml==0.10.2

from pytest-check.

Hi, I'm pytest 6.2.1 and Python 3.7. I do not pip install local version of pytest-check installed when running the tests -- I'll try that now.

from pytest-check.

That did the trick! Thank you!

Also, when I made this change to confest.py's pytest_runtest_makereport, the new test quoted above (that makes sure you get two failures) starts passing. I was just playing around, but if this looks like an ok fix (to #43 and #44), I can submit a PR.

            if report.longrepr:
                report.longrepr.append("\n".join(longrepr))
            else:
                report.longrepr = "\n".join(longrepr)

Feel free to close this ticket out, appreciate your time (and the excellent book!).

from pytest-check.

The longrepr trick doesn't work on newer versions of pytest.
I'm working on a fix for this issue. It was reported twice already, so I'm closing this.

from pytest-check.