Add pytest as a testing framework for Python about runner HOT 7 OPEN

@Blind4Basics pytest actually produces nice failure message from simple assertions.

Similar to power asserts from Groovy. So

x = 1
assert x == 2

Fails with something like

Assertion failed:
  assert x == 2
         |  |
         1  false

It can produce more useful output depending on the test case.

from runner.

Codewars can only support one test framework per language at the moment because of how the database is set up. So we won't be able to add another test framework until we do the rewrite which I'm planning to start working on after my vacation.

I personally don't like having to maintain custom frameworks when there're existing ones we can use (e.g., JS/Python/Ruby, etc.), so I'd like to deprecate them in the future for new contents. We still need to keep them for a while for existing contents though.

For Python, I like pytest too and if I remember correctly, it supports custom reporter so it should be possible to add.

from runner.

@kazk: Thanks for quick reply. I'll wait patiently. :) Also, I completely agree with your stance on custom frameworks. Enjoy your vacation and good luck with the rewrite.

from runner.

@kazk: I came back to this after some time. I guess there's been no progress on this and the single test framework per language limitation is still there. One possible option would be to emulate the existing framework with pytest (i.e. the current framework would be an alternative syntax for pytest. Is this a sensible idea according to you?

from runner.

No progress on Codewars part.

Have you looked into if pytest supports custom reporters?

One possible option would be to emulate the existing framework with pytest (i.e. the current framework would be an alternative syntax for pytest. Is this a sensible idea according to you?

I'll consider it if it's fully backwards compatible and also depending on the amount of effort required.

from runner.

@suic86 : but what is the feeback for the user in case a test fails? Seems to me you're comparing assert_equals with an equivalent of expect. Not a good idea, imo.

from runner.

For reporting pytest results, we should be able to make custom reporter similar to https://github.com/pchomik/pytest-spec

I haven't looked into if we can simulate Codewars test framework with pytest so please let me know if you have some proposals. I'd be surprised if it turns out possible because codewars_test has lots of quirks.

from runner.