Code has 3 functions func A() , func B() and func C() called a test function test_A

pytest check is not reporting all failures distributed across multiple python functions or setup fixture called within a test function about pytest-check HOT 3 CLOSED

C9943028 commented on September 22, 2024

pytest check is not reporting all failures distributed across multiple python functions or setup fixture called within a test function

from pytest-check.

Comments (3)

okken commented on September 22, 2024

Weird. I’ll take a look.

from pytest-check.

okken commented on September 22, 2024

I'm not able to reproduce the problem.
It looks like it's working to me.

test_check.py

import pytest
from pytest_check import check

def a():
    check.equal(1, 1)
    check.equal(1, 2, msg='a failure 1')
    check.equal(1, 3, msg='a failure 2')

def b():
    check.equal(1, 1)
    check.equal(1, 2, msg='a failure 1, total 3')
    check.equal(1, 3, msg='b failure 2, total 4')
    check.equal(1, 4, msg='c failure 3, total 5')


def c():
    check.equal(1, 1)
    check.equal(1, 2, msg='c failure 1, total 6')
    check.equal(1, 3, msg='c failure 2, total 7')

@pytest.fixture()
def a_fixture():
    yield
    # failure in teardown
    check.equal(1, 1)
    check.equal(1, 2, msg='fixture failure 1, total 8')
    check.equal(1, 3, msg='fixture failure 2, total 9')


def test_(a_fixture):
    a()
    b()
    c()

It's a little confusing because the teardown failures cause an ERROR and pytest reports errors separately from failures.
Also the errors are reported first.

But still, I see 7 errors in test, and 2 in the fixture.

$ pytest test_check.py
============================= test session starts =============================
collected 1 item

test_check.py FE                                                         [100%]

=================================== ERRORS ====================================
_________________________ ERROR at teardown of test_ __________________________
FAILURE: check 1 == 2: fixture failure 1, total 8
test_check.py:26 in a_fixture() -> check.equal(1, 2, msg='fixture failure 1, total 8')

FAILURE: check 1 == 3: fixture failure 2, total 9
------------------------------------------------------------
Failed Checks: 2
================================== FAILURES ===================================
____________________________________ test_ ____________________________________
FAILURE: check 1 == 2: a failure 1
test_check.py:31 in test_() -> a()
test_check.py:6 in a() -> check.equal(1, 2, msg='a failure 1')

FAILURE: check 1 == 3: a failure 2
FAILURE: check 1 == 2: a failure 1, total 3
FAILURE: check 1 == 3: b failure 2, total 4
FAILURE: check 1 == 4: c failure 3, total 5
FAILURE: check 1 == 2: c failure 1, total 6
FAILURE: check 1 == 3: c failure 2, total 7
------------------------------------------------------------
Failed Checks: 7
=========================== short test summary info ===========================
FAILED test_check.py::test_
ERROR test_check.py::test_
========================= 1 failed, 1 error in 0.19s ==========================

from pytest-check.

okken commented on September 22, 2024

If you've got a failure example, please share, and I can re-open the issue.

from pytest-check.

pytest check is not reporting all failures distributed across multiple python functions or setup fixture called within a test function about pytest-check HOT 3 CLOSED

Comments (3)

Related Issues (20)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent