fluxx / exam Goto Github PK

View Code? Open in Web Editor NEW

168.0 168.0 19.0 140 KB

Helpers for better testing.

License: MIT License

Ruby 1.66% Python 97.65% Makefile 0.69%

exam's People

Contributors

Stargazers

Watchers

Forkers

dcramer lujaw joskid elijahandrews alex lowks adamchainz wking ivanov jessamynsmith gterzian koutoftimer gi11es nagyzsuska fuziontech timxuhj mifengbushimifeng paolostivanin srinivas32

exam's Issues

Make after, around support being constructed with another function

Should work the same way @before does.

`assertDoesNotChange` error message is ambiguous if before or after values are incorrect

An assertion with this structure:

  with self.assertChanges(get_value, before=0, after=4):
      set_value(2)

will result in the following assertion error, even though the after value did change, just to the incorrect/unexpected value (2, rather than 4):

AssertionError: after value did not change (4)

Fixtures for "types" of abc.ABCMeta do not work

When used inline, fixture type checks the first argument to see if it's a type or method in order to apply it inline:

exam/exam/decorators.py

Line 31 in 1ab9435

if type(self.thing) in (type, types.MethodType):

This doesn't handle cases when the type of the thing is abc.ABCMeta.

Deterministic side effects

A lot of times when you have a mock with a side effect, you want to deterministically return values depending on the arguments. Generally you have to do something like this:

cache_return_valus = {
    'foo': 1
    'bar': 2
}

def mock_cache_get(self, key):
    return self.cache_return_values[key]

@patch('cache.get')
test returns_results(self, cache_get):
    cache_get.side_effect = self.mock_cache_get
    self.assertEquals(my_func(), 2)

Instead, it would nice if side_effect understood a dict (or provided some other syntax) to return values for arguments:

cache_get.side_effect = self.cache_return_values

Similar to how it allows an iterable to return a values in order.

Add docs to @fixture decorator

I'm using docstring type annotations for type hinting purposes and @fixture decorator didn't copy __doc__ of decorated attribute, but @property does. So you should add additional behavior or remove

The @fixture decorator turns a method into a property

from library documentation, because it is not fully truth.

Lifecycle decorators in subclasses do not overwrite their parents

The subclass should overwrite, just like normal subclassing.

Inline fixture calls seem to be the same object

thread1 = fixture(Thread.objects.get, id=1)
thread2 = fixture(Thread.objects.get, id=2)

thread1.pk == thread2.pk, which isn't correct.

Allow @around decorator to yield arguments to test method

It would be great to be able to do this:

@around
def build_db(self)
    db = db.connect()
    yield db
    db.close()

def test_foo(self, db):
    assert db.query('SELECT 1')

Raise exceptions inside assertRaises

They're not being re-raised in __exit__.

Usage of `assertDoesNotChange` with `before` or `after` parameters results in confusing error message.

Any combination of usage of before or after parameters results in an error, for example:

Traceback (most recent call last):
  File "example.py", line 10, in test_assertdoesnotchange_after
    with self.assertDoesNotChange(lambda: 0, before=0):
  File "/Users/ted/.virtualenvs/eaxm/lib/python2.7/site-packages/exam/asserts.py", line 33, in __enter__
    assert not check, message.format(**vars(self))
AssertionError: Value before is 0, not 0

This prevents being able to use the before or after keyword (or both) as a shorter form of the assertion:

value = 0
with self.assertDoesNotChange(lambda: value):
  assert value == 0

Add `assertDoesNotChange` context manager.

Allow properties to be used for assertChanges

Rather than this:

with self.assertChanges(lambda: Post.objects.get(pk=post.pk).killed, before=False, after=True):
   #  ...

Allow this:

with self.assertChanges(Post.objects.get(pk=post.pk).killed, before=False, after=True):
    # ...

Exam would just do the lambda wrap for you.

0.10.6: pytest warnings

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-exam-0.10.6-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-exam-0.10.6-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/python3 -Bm pytest -ra
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.9, pytest-6.2.3, py-1.10.0, pluggy-0.13.1
rootdir: /home/tkloczko/rpmbuild/BUILD/exam-0.10.6
plugins: forked-1.3.0, shutil-1.7.0, virtualenv-1.7.0, asyncio-0.14.0, expect-1.1.0, cov-2.11.1, mock-3.5.1, httpbin-1.0.0, xdist-2.2.1, flake8-1.0.7, timeout-1.4.2, betamax-0.8.1, pyfakefs-4.4.0, freezegun-0.4.2, flaky-3.7.0, cases-3.4.6, hypothesis-6.10.1
collected 55 items

tests/test_asserts.py ........                                                                                                                                       [ 14%]
tests/test_cases.py ................                                                                                                                                 [ 43%]
tests/test_decorators.py ..........                                                                                                                                  [ 61%]
tests/test_exam.py ..                                                                                                                                                [ 65%]
tests/test_helpers.py ...........                                                                                                                                    [ 85%]
tests/test_mock.py .....                                                                                                                                             [ 94%]
tests/test_objects.py ...                                                                                                                                            [100%]

============================================================================= warnings summary =============================================================================
exam/mock.py:19
  /home/tkloczko/rpmbuild/BUILD/exam-0.10.6/exam/mock.py:19: SyntaxWarning: "is not" with a literal. Did you mean "!="?
    assert len(self.__calls_matching(*args, **kwargs)) is not 1

exam/mock.py:22
  /home/tkloczko/rpmbuild/BUILD/exam-0.10.6/exam/mock.py:22: SyntaxWarning: "is" with a literal. Did you mean "=="?
    assert len(self.__calls_matching(*args, **kwargs)) is 0

tests/test_asserts.py::AssertChangesMixin::test_assertion_error_mentions_unexpected_result_at_after
  /home/tkloczko/rpmbuild/BUILD/exam-0.10.6/tests/test_asserts.py:61: DeprecationWarning: Please use assertRaisesRegex instead.
    with self.assertRaisesRegexp(AssertionError, msg):

tests/test_asserts.py::AssertChangesMixin::test_raises_assertion_error_if_value_changes
  /home/tkloczko/rpmbuild/BUILD/exam-0.10.6/tests/test_asserts.py:55: DeprecationWarning: Please use assertRaisesRegex instead.
    with self.assertRaisesRegexp(AssertionError, msg):

-- Docs: https://docs.pytest.org/en/stable/warnings.html
====================================================================== 55 passed, 4 warnings in 0.19s ======================================================================

Add version of @fixture that runs decorated method before test runs

Imagine it would look something like this:

class Test(Exam, TestCase):

    @fixture(execute='before')
    def user(self):
        return User.objects.create(name='jeff')

Still not 100% sold on it, but wanted to add it before I forget.

Make stacked @mock_import work like stacked @patch decorators

Right now if you write code like this:

@mock_import('os')
@mock_import('functools')
def test_things(self, *args):
    pass

*args is (mock.os, mock.functools) when it should be the other way around, similar to how stacked @patch decorators work.

Does exam only work with unittest?

Can I use it with nose/py.test/nose2 and expect it to work?

@before decorator breaks the nose discovery

When wrapping a normal test method, the fact that @before is returning the inner function, the __name__ is not retained, causing nose to not run the actual test.

def bar(*args, **kwargs):
    pass

class TestFoo(object):
    @before(bar)
    def test_bar(self):
        pass

Expose patch object when using patcher

When using patcher like inside a test case body:

foo = patcher('my.bar')

self.foo in the test case is the object returned from calling patcher.start(), which most of the time is a Mock object. This is great, but some times in a test you might want to stop the patcher, but there is no reference to the patch object for you to do so.

Add assertChanges context manager

Rather than having to do code like this:

self.assertEmpty(mail.outbox)  # sanity check
Follow.objects.create(user=follower, target=target)
self.assertEqual(len(mail.outbox), 1)

It would be great do do this:

with self.assertChanges(len, mail.outbox).from(0).to(1):
    Follow.objects.create(user=follower, target=target)

reset fixture api

Allow for an easy way to reset a fixture inside of a test case.

Fail before running when a test has a setUp method

The downstream failures can not be obviously linked to to having a setUp method that does not call super().

The expected behavior would be to abort the tests before they run with a useful log message.