This is an error in old.py (SHA#00a60ab0)

This is the patch I used to fix it:

@@ -116,9 +116,9 @@
         except Exception, e:
             if isinstance(exc, basestring):
                 msg = utf8_bytes(exc)
+                exc = type(e)

             err = utf8_bytes(str(e))
-            exc = type(e)

             if isinstance(exc, type) and issubclass(exc, Exception):
                 if not isinstance(e, exc):

There what I saw is that the type of e (the thrown exception) is assigned to exc (the expected exception type. That line makes sense only inside the if block, where exc gets the type of e because was originally just the exception message string.

trying to pip install sure ez_install sure or even python setup.py install does not work in python 3.4.0
why?
well if you ask me you got a little too clever for your own good.

the read_version() function in the setup.py file fails

Running setup.py (path:/tmp/pip_build_ubuntu/sure/setup.py) egg_info for package sure
    Traceback (most recent call last):
      File "<string>", line 17, in <module>
      File "/tmp/pip_build_ubuntu/sure/setup.py", line 53, in <module>
        version=read_version(),
      File "/tmp/pip_build_ubuntu/sure/setup.py", line 40, in read_version
        finder.visit(ast.parse(local_file('sure', '__init__.py')))
      File "/tmp/pip_build_ubuntu/sure/setup.py", line 45, in <lambda>
        open(os.path.join(os.path.dirname(__file__), *f)).read()
      File "/usr/lib/python3.4/encodings/ascii.py", line 26, in decode
        return codecs.ascii_decode(input, self.errors)[0]
    UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 143: ordinal not in range(128)
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):

  File "<string>", line 17, in <module>

  File "/tmp/pip_build_ubuntu/sure/setup.py", line 53, in <module>

    version=read_version(),

  File "/tmp/pip_build_ubuntu/sure/setup.py", line 40, in read_version

    finder.visit(ast.parse(local_file('sure', '__init__.py')))

  File "/tmp/pip_build_ubuntu/sure/setup.py", line 45, in <lambda>

    open(os.path.join(os.path.dirname(__file__), *f)).read()

  File "/usr/lib/python3.4/encodings/ascii.py", line 26, in decode

    return codecs.ascii_decode(input, self.errors)[0]

UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 143: ordinal not in range(128)

I've gotten around this for now by editing the setup.py file localy and hardcoding the version.

The pypi package has install dependencies on:
nose==1.3.0
rednose==0.4.1

Firstly these are not install dependencies they are test dependencies (and rednose probably is neither and belongs in development.txt because it is only referenced by Makefile)

Secondly, because this is imported directly into setup.py this means that the library is being published with a strict requirement on these versions. This is a similar problem to what we had in HTTPretty so i'm providing the following links which i hope will explain the difference between requirements.txt and setup.py and why it's bad to publish strict version requirements in a setup.py file:

• http://blog.miguelgrinberg.com/post/the-package-dependency-blues (get's a little lengthy with the whole story).
• https://caremad.io/blog/setup-vs-requirement/

Thanks for your help.

Steps to repro

$ mkvirtualenv trial
$ pip install py.test==2.5.1 py==1.4.19 sure==1.2.2

create a test_trial.py file with a simple unit test in current directory, there should be an import sure in the file

$ py.test

Actual output

============================= test session starts ==============================
platform linux -- Python 3.3.3 -- pytest-2.5.1
collected 1 items

test_trial.py .

=========================== 3 passed in 0.07 seconds ===========================

Expected output

============================= test session starts ==============================
platform linux -- Python 3.3.3 -- pytest-2.5.1
collected 1 items

test_trial.py ...

=========================== 1 passed in 0.10 seconds ===========================

Sometimes it's useful to test that a property eventually becomes true within a set timeout period, especially for slightly longer-running integration tests.

This is particularly useful for making assertions when using Pykka, a concurrency library based on the actor model.

In Objective-C, the default BDD test framework of choice Kiwi already comes with a couple of useful async assertion primitives out of the box; sure should too!

Commit cd511b5 broke tests for python 2!

Full log:

............................E...............................................................................................
======================================================================
ERROR: test_assertion_builder.test_throw_matching_regex
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/build/python-sure/src/sure-1.2.8/sure/old.py", line 125, in raises
    self._src(*self._callable_args, **self._callable_kw)
  File "/build/python-sure/src/sure-1.2.8/tests/test_assertion_builder.py", line 639, in blah
    raise ValueError(msg)
ValueError: this message

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.4/site-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/build/python-sure/src/sure-1.2.8/tests/test_assertion_builder.py", line 647, in test_throw_matching_regex
    expect(blah).when.called_with(1).should.throw(re.compile(r'invalid regex'))
  File "/build/python-sure/src/sure-1.2.8/sure/__init__.py", line 377, in wrapper
    value = func(self, *args, **kw)
  File "/build/python-sure/src/sure-1.2.8/sure/__init__.py", line 804, in throw
    return _that.raises(*args, **kw)
  File "/build/python-sure/src/sure-1.2.8/sure/old.py", line 154, in raises
    'Expected to match regex: %r\n against:\n %r' % (identify_callable_location(self._src), msg.pattern, err))
  File "/build/python-sure/src/sure-1.2.8/sure/old.py", line 45, in identify_callable_location
    filename = os.path.relpath(callable_object.func_code.co_filename)
AttributeError: 'function' object has no attribute 'func_code'

----------------------------------------------------------------------
Ran 124 tests in 0.763s

FAILED (errors=1)

The tarball which is up on PyPI doesn't contain test cases, so it might be a very good idea to include them. Especially useful if distributions want to pick up sure.

Otherwise, the user has to do that all the time to avoid the following error:

Here's a basic test case

import mock

def test_equals_handles_mock_call_list():
    ".equal() Should convert mock._CallList instances to lists"

    # Given the following mocked callback
    callback = mock.Mock()

    # When I call the callback with a few parameters twice
    callback(a=1, b=2)
    callback(a=3, b=4)

    # Then I see I can compare the call list without manually
    # converting anything
    callback.call_args_list.should.equal([
        mock.call(a=1, b=2),
        mock.call(a=3, b=4),
    ])

So that the code below is valid:

from sure import assertion, chain, chainproperty
from lxml import html


@assertion
def attribute(parent, is_negative, name):  # *args, **kw FTW
    if is_negative:
        return name not in parent.object.attrib
    else:
        return name in parent.object.attrib

@chain
def child(parent, name):  # *args, **kw FTW
    items = parent.object.cssselect(name)
    items.should.have.length_of(1)

    return items[0]

@chainproperty
def having(parent):
    return parent.object

def test_anchors_in_1st_level_headers():
    headers = dom.cssselect("h1")

    headers.should.have.length_of()

    h1 = headers[0]

    h1.should.have.attribute("name").being.equal("api-reference")
    h1.should.have.child("a").having.attribute("href").equal("#api-reference")

It looks like called_with() is broken in the newest version (1.2.5)

Failing test:

import sure

def raise_err(foobar):
    raise ValueError()

def test():
    raise_err.when.called_with("adsf").should.throw(ValueError)

Traceback

Traceback (most recent call last):
  File "nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "test.py", line 10, in test
    raise_err.when.called_with("adsf").should.throw(ValueError)
  File "sure/__init__.py", line 375, in wrapper
    value = func(self, *args, **kw)
  File "sure/__init__.py", line 789, in throw
    return _that.raises(*args, **kw)
  File "sure/old.py", line 129, in raises
    self._src, exc, e.__class__, traceback.format_exc(e)))
AssertionError: <function raise_err at 0x1036ec140> should raise <type 'exceptions.ValueError'>, but raised <type 'exceptions.TypeError'>:
ORIGINAL EXCEPTION:

Traceback (most recent call last):
  File "sure/old.py", line 117, in raises
    self._src(*self._callable_args, **self._callable_kw)
TypeError: raise_err() takes exactly 1 argument (0 given)

I think it may have to do with this commit, but am not positive yet: e292cd5

sure should be a test runner and just like lettuce, provide the following feature:

a "bootstrap" file

when running sure should try to import a confidency.py and there the developer will be able to declare some "hooks":

# confidence.py

from sure.hooks import before, after

@before.each_test
def clear_database(test):
    if "myapp.redis_layer" in test.__module__:
        # do whatever actions specific to tests under myapp.redis_layer

rant of possible hooks:

• @before.each_test
• @before.each_module
• @before.each_package
• @before.all
• @after.each_test
• @after.each_module
• @after.each_package
• @after.all

EDIT by @timofurrer: content of former TODO.md:

Test runner with chronometer and coverage reports

Sure provides you with a test runner that gives you full control
over the process to the granularity of a single test case.

You can write your tests anywhere, sure will find, run, count up the
duration and allow running every test case, in the end you get
acquainted with the statistics of your python codebase: what parts
have test coverage, where you should refactor or add coverage.

It also runs flake8 against
your test code and blame you when your tests are too complex

Ahh yes, you can also run only a given subset of test cases, force
slow tests to fail.

Just taking the module in for the first time. I really like the readability and clarity of the tests.

But on my first use, I tried doing some comparison operations. There appears to be a len_greater_than, len_greater_than_or_equals, etc, but the simple comparisons (which I assume would be named is_greater_than, is_greater_than_or_equals, is_less_than, doesnt_equal, and so on) are absent.

Are they really missing, or am I misunderstanding how to express those tests? I was able to add them with the @that.is_a_matcher decorator, but they seem like core tests that should be available to all.

The code for these is of course very simple. I would, however, be happy to submit a patch if you like.

It's due to the difference between the case of filenames and their entries in MANIFEST.in.

sometimes I wanna check a value of a key from a dictionary, for example, but the values from the other keys don't really matter. Currently there is no way to do this but copy all the values from the other dict on the assert dict. I propose the following:

a = {'key': 1, 'other_key': 4}
b = {'key': 1, 'other_key': sure.any()}

assert that(a).equals(b) // True

In the context of this code, the value of 'other_key' is completely unnecessary to test, so i create an instance of sure.any to match with it and 'any' is equal to everything.

This is not a new concept, some other libraries already implement it, like Jasmine.

direct link:

http://pivotal.github.com/jasmine/#section-Matching_Anything_with_jasmine.any

I'm sorry, the link was getting escaped so I had to do this.

After implementing the test runner functionality, sure should provide a test lint.

Using the ast library we can check if the body of the test contains test smells such as "loops", too many mocks, too many assertions, internal imports and so on....

According to the Voidspace mock documentation, you can verify that a method was invoked with specific parameters by doing:

>>> expected = [call(1, 2, 3), call(4, 5, 6), call()]
>>> mock.call_args_list == expected
True

The above example works and passes the test if put into an assert. i.e. the statement assert mock.call_args_list == expected is valid.

If I want to write the same thing in sure syntax, I get a failure. Let's say I want to write a test with this:

def test_oh_dear():
    # Set up the mock
    # Interact with the mock
    expected = [call(1, 2, 3), call(4, 5, 6), call()]
    mock.call_args_list.should.be.equal(expected) # !!! this line fails

The failure I get is:

     Failure/Error: given
     X = [...]
         and
     Y = [...]
     X is a _CallList and Y is a list instead

The contents of the arrays don't matter. I just find it inconsistent and counter-intuitive syntactically that == works with an assertion and .equal() does not!

here's the text version of the traceback to help search engines to index the problem:

>>> import sure
>>> import rethinkdb
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/lincoln/Work/Personal/dotfiles/.virtualenvs/canary/lib/python2.7/site-packages/rethinkdb/__init__.py", line 7, in <module>
    import rethinkdb.docs
  File "/Users/lincoln/Work/Personal/dotfiles/.virtualenvs/canary/lib/python2.7/site-packages/rethinkdb/docs.py", line 19, in <module>
    rethinkdb.ast.RqlQuery.do.__func__.__doc__ = u"    Evaluate the expr in the context of one or more value bindings.\nThe type of the result is the type of the value returned from expr.\n\n *Example:* The object(s) passed to do() can be bound to name(s). The\nlast argument is the expression to evaluate in the context of the\nbindings.\n>>> r.do(r.table('marvel').get('IronMan'),\n...     lambda ironman: ironman['name']).run(conn)"
  File "/Users/lincoln/Work/Personal/dotfiles/.virtualenvs/canary/lib/python2.7/site-packages/sure/__init__.py", line 454, in __getattr__
    return super(AssertionBuilder, self).__getattribute__(attr)
AttributeError: 'AssertionBuilder' object has no attribute '__func__'

Sometimes I want to test that an object is the same object that I set up, rather than another object that happens to equate to it. This is useful when using mutable values, for example.

At the moment, you have to explicitly use Python's is operator, or compare their ids:

>>> import sure
>>> d1 = {'four':4}
>>> d2 = d1
>>> d3 = {'four':4}
>>> (d2 is d1).should.be.true
True
>>> (d3 is d1).should.be.false
True
>>> # or...
... (id(d2)).should.equal(id(d1))
True
>>> (id(d3)).should_not.equal(id(d1))
True

I propose overloading be for this purpose:

>>> d2.should.be(d1)
True
>>> d3.should_not.be(d1)
True

Why is the build for python version 3.3 allowed to fail?

Sure should allow overwritting the do, does and all the other methods (see below) when using the CPython-compatible syntax sugar (i.e.: (1).should.be.an(int))

• do
• do_not
• does
• does_not
• doesnt
• should
• should_not
• shouldnt
• must

For instance, this test should pass:

import sure

class Foo(object):
    pass

instance = Foo()
instance.do = "anything"

mock==1.0.1 is specified in requirements.txt and it seems that this version is no more available.
For example, it is breaking installation of lettuce.

the VariablesBag should have methods for

• listing members
• supporting either getattr or getitem

In other words, this test should pass:

from sure import scenario

def prepare(context):
    context["some_cool_variable"]= 123

def dismiss(context):
    context.list_members().should.equal(["some_cool_variable", "times_2"])

@scenario(prepare, dismiss)
def test_something(context):
    context.times_2 = context.some_cool_variable * 2

Sometime we are just not gonna need the context variable.

So it MUST not be obligatory

example:

from sure import that_with_context

def st(context):
    pass
def td(context):
    pass

@that_with_context(st, td)
def test_foobar():
    assert True

Hi,
When I run my tests I get the same error for all tests:

  File "/home/vagrant/.tox/project/py34/lib/python3.4/site-packages/sure/__init__.py", line 377, in wrapper
    value = func(self, *args, **kw)
  File "/home/vagrant/.tox/project/py34/lib/python3.4/site-packages/sure/__init__.py", line 600, in equal
    comparison = DeepComparison(self.obj, what, epsilon).compare()
  File "/home/vagrant/.tox/project/py34/lib/python3.4/site-packages/sure/core.py", line 241, in compare
    if isinstance(X, mock._CallList):
AttributeError: 'module' object has no attribute '_CallList'

This is strange because the same test worked, and I dont know what I did to them breaking.
How can I fix this?

Using .raises doesn't seem to work - I can't break the test

I tried to install the latest commit (7d51e44) with PyPy, but I ran into a problem.

$ sudo pypy setup.py install
Traceback (most recent call last):
  File "app_main.py", line 51, in run_toplevel
  File "setup.py", line 20, in <module>
    from sure import version
  File "/private/tmp/sure/sure/__init__.py", line 39, in <module>
    from sure.magic import is_cpython, patchable_builtin
  File "/private/tmp/sure/sure/magic.py", line 41, in <module>
    ('ob_type', ctypes.POINTER(PyObject)),
  File "/opt/local/lib/pypy/lib_pypy/_ctypes/structure.py", line 137, in struct_setattr
    self.__dict__.get('_anonymous_', None))
  File "/opt/local/lib/pypy/lib_pypy/_ctypes/structure.py", line 72, in names_and_fields
    self.__dict__.update(fields)
TypeError: cannot add non-string keys to dict of a type

using @sure.that_with_context currently crashes with this:

http://somethingaboutorange.com/mrl/projects/nose/1.0.0/writing_tests.html#test-generators

Working case

def test_demo():
    assert True

py.test test_demo.py --junit-xml=junit.xml

junit.xml looks like:

<?xml version="1.0" encoding="utf-8"?>
<testsuite errors="0" failures="0" name="pytest" skips="0" tests="1" time="0.007">
  <testcase classname="test.test_demo" name="test_demo" time="0.000320196151733"/>
</testsuite>

Failing case

Just add an import sure:

import sure
def test_demo():
    assert True

py.test test_demo.py --junit-xml=junit.xml

Now junit.xml is empty:

<?xml version="1.0" encoding="utf-8"?>
<testsuite errors="0" failures="0" name="pytest" skips="0" tests="0" time="0.021">
</testsuite>

The following should raise an error, but doesn't:

import sure
def f():
    return 0

f.when.called_with().should_not.return_value(0)

It looks like should_not does not negate when used with called_with()

Use of called_with() and return_value() is documented at http://falcao.it/sure/reference.html#function-when-called_with-arg1--kwarg1-2--should-return_value-value- so I assume negating it with should_not should also work.

Can test-requirements be unpinned? e.g. use >=, rather than ==? There is a newer release of sure at least.

It can be useful within some kinds of tests

from sure import *

@within(five=microseconds):
def test_foobar(utcnow, now):
    # ... http request test
    assert http.headers['date'] == utcnow.strftime('%a, %d %b %Y %H:%M:%S GMT')

There are edge cases where the same exception can throw different messages in different operating systems, for example.

It would be really helpful if sure could optionally take a regular expression as an argument.

for instance

from collections import OrderedDict
result = {
    "fields": OrderedDict([
        ("name", "John"),
        ("age", "22"),
    ]),
    "children": OrderedDict([]),
}

when compared like this:

result.should.equal({
    "fields": OrderedDict([
        ("age", "22"),
        ("name", "John"),
    ]),
    "children": OrderedDict([]),
})

should raise something like

X = {
    "fields": OrderedDict([
        ("name", "John"),
        ("age", "22"),
    ]),
    "children": OrderedDict([]),
}

Y = {
    "fields": OrderedDict([
        ("age", "22"),
        ("name", "John"),
    ]),
    "children": OrderedDict([]),
}

X["fields"] and y["fields"] are in a different order

Hi!

I'm the maintainer of the pyshould library and it would be great to borrow your code from magic.py in order to support the x.should syntax in my library. It's a very interesting hack!

Since sure is licensed as GPL3 but pyshould is under MIT I would need your written consent before integrating that code if you agree to it. I guess it will suffice to simply reply to this issue stating that you allow such inclusion in pyshould.

PS: I would fully understand if you prefer to keep your code only under its current license.

I'm currently testing XML comparison, diffs would have been nice difflib.Differ

In testing, it is very useful to separate random exceptions thrown from code from actual test assertion failures (unittest, nose and friends go at this half way by categorizing failures vs. errors).

In some of the environments one would like to use a custom exception type for assertions, to prevent confusions from actual assert statements inside his or her code.

It would be nice to do something like:

import sure

...

with sure.set_assertion_class(MyAssertionError):
    (2).should.be.a(str)

Of course this can also be a global setup function, but as a context/thread local it would be more flexible.

There is no proper way to check floats for equality.
Can we add a new compare_float method in DeepComparison to check for float equality with an epsilon?

🍻

Hi,
I will be packaging "sure" for Fedora because I want to package lettuce and "sure" is its requirement. I'd like to ask you if it would be possible to add the COPYING file to official pypi releases - it is optimal for packaging, if the licensing is contained in the upstream source.

Thank you.

There is something wrong in the setup.py. Python 3.2 can't deal with that :(

$ pip install sure
Downloading/unpacking sure
  Downloading sure-1.2.2.tar.gz
  Running setup.py egg_info for package sure
    Traceback (most recent call last):
      File "<string>", line 16, in <module>
      File "/Users/lincoln/.virtualenvs/1b7ed5607e05fa90/build/sure/setup.py", line 36
        author=u'Gabriel Falcao',
                               ^
    SyntaxError: invalid syntax
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
  File "<string>", line 16, in <module>
  File "/Users/lincoln/.virtualenvs/1b7ed5607e05fa90/build/sure/setup.py", line 36
    author=u'Gabriel Falcao',
                           ^
SyntaxError: invalid syntax

At present, one can only write

biglist = [1, 2, 3]
biglist.should.have.length_of(3)

Whereas it's sometimes useful to have

biglist = [1, 2, 3]
biglist.should.have.length.greater_or_equal_to(3)

that fails if a TCP port is busy

from sure import expect

@expect.free_port(8000)
def test_my_server():
    # run the server
    # run assertions against http request/responses