Code Monkey home page Code Monkey logo

specify's Introduction

Specify

BDD style code blocks for PHPUnit or Codeception

Latest Stable Version Total Downloads Latest Unstable Version License StandWithUkraine

Specify allows you to write your tests in more readable BDD style, the same way you might have experienced with Jasmine. Inspired by MiniTest of Ruby now you combine BDD and classical TDD style in one test.

Installation

Requires PHP >= 7.4

  • Install with Composer:
composer require codeception/specify --dev
  • Include Codeception\Specify trait in your tests.

Usage

Specify $this->specify method to add isolated test blocks for your PHPUnit tests!

public function testValidation()
{
    $this->assertInstanceOf('Model', $this->user);

    $this->specify('username is required', function() {
        $this->user->username = null;
        $this->assertFalse($this->user->validate(['username']));
    });

    $this->specify('username is too long', function() {
        $this->user->username = 'toolooooongnaaaaaaameeee';
        $this->assertFalse($this->user->validate(['username']));
    });
}

BDD Example

Specify supports describe-it and describe-should BDD syntax inside PHPUnit

public function testValidation()
{
    $this->describe('user', function () {
        $this->it('should have a name', function() {
            $this->user->username = null;
            $this->assertFalse($this->user->validate(['username']));
        });
    });

    // you can use chained methods for better readability:
    $this->describe('user')
        ->should('be ok with valid name', function() {
            $this->user->username = 'davert';
            $this->assertTrue($this->user->validate(['username']));
        })
        ->shouldNot('have long name', function() {
            $this->user->username = 'toolooooongnaaaaaaameeee';
            $this->assertFalse($this->user->validate(['username']));
        })
        // empty codeblocks are marked as Incomplete tests
        ->it('should be ok with valid name') 
    ;
}

Specify + Verify Example

Use Codeception/Verify for simpler assertions:

public function testValidation()
{
    $this->specify('username is too long', function() {
        $this->user->username = 'toolooooongnaaaaaaameeee';
        expect_not($this->user->validate(['username']));
    });

    $this->specify('username is ok', function() {
        $this->user->username = 'davert';
        expect_that($this->user->validate(['username']));
    });
}

Use Case

This tiny library makes your tests readable by organizing them in nested code blocks. This allows to combine similar tests into one but put them inside nested sections.

This is very similar to BDD syntax as in RSpec or Mocha but works inside PHPUnit:

<?php

class UserTest extends PHPUnit\Framework\TestCase 
{
    use Codeception\Specify;

    /** @specify */
    protected $user; // is cloned inside specify blocks

    public function setUp(): void
    {
        $this->user = new User;
    }

    public function testValidation()
    {
        $this->user->name = 'davert';
        $this->specify('i can change my name', function() {
           $this->user->name = 'jon';
           $this->assertEquals('jon', $this->user->name);
        });
        // user name is 'davert' again
        $this->assertEquals('davert', $this->user->name);
    }
}

Each code block is isolated. This means call to $this->specify does not change values of properties of a test class. Isolated properties should be marked with @specify annotation.

Failure in specify block won't get your test stopped.

<?php
$this->specify('failing but test goes on', function() {
	$this->fail('bye');
});
$this->assertTrue(true);

// Assertions: 2, Failures: 1
?>

If a test fails you will see specification text in the result.

Isolation

Isolation is achieved by cloning object properties for each specify block. Only properties marked with @specify annotation are cloned.

/** @specify */
protected $user; // cloning

/** 
 * @specify 
 **/
protected $user; // cloning

protected $repository; // not cloning

Objects are cloned using deep cloning method.

If object cloning affects performance, consider turning the clonning off.

Mocks are isolated by default.

A mock defined inside a specify block won't be executed inside an outer test, and mock from outer test won't be triggered inside codeblock.

<?php
$config = $this->createMock(Config::class);
$config->expects($this->once())->method('init');

$config->init();
// success: $config->init() was executed

$this->specify('this should not fail', function () {
    $config = $this->createMock(Config::class);
    $config->expects($this->never())->method('init')->willReturn(null);
    // success: $config->init() is never executed 
});

Examples: DataProviders alternative

<?php
$this->specify('should calculate square numbers', function($number, $square) {
	$this->assertEquals($square, $number*$number);
}, ['examples' => [
		[2,4],
		[3,9]
]]);

You can also use DataProvider functions in examples param.

<?php
$this->specify('should calculate square numbers', function($number, $square) {
	$this->assertEquals($square, $number*$number);
}, ['examples' => $this->provider()]);

Can also be used with real data providers:

<?php
/**
 * @dataProvider someData
 */
public function testExamplesAndDataProvider($param)
{
    $this->specify('should assert data provider', function ($example) use ($param) {
        $this->assertGreaterThanOrEqual(5, $param + $example);
    }, ['examples' => [[4], [7], [5]]]);
}

public function someData()
{
    return [[1], [2]];
}

Before/After

There are also before and after callbacks, which act as setUp/tearDown but for specify.

<?php
$this->beforeSpecify(function() {
	// prepare something;	
});
$this->afterSpecify(function() {
	// reset something
});
$this->cleanSpecify(); // removes before/after callbacks
?>

API

Available methods:

// Starts a specify code block:
$this->specify(string $thing, callable $code = null, $examples = [])

// Starts a describe code block. Same as 'specify' but expects chained 'it' or 'should' methods.
$this->describe(string $feature, callable $code = null)

// Starts a code block. If 'code' is null, marks test as incomplete.
$this->it(string $spec, callable $code = null, $examples = [])
$this->its(string $spec, callable $code = null, $examples = [])

// Starts a code block. Same as 'it' but prepends 'should' or 'should not' into description.
$this->should(string $behavior, callable $code = null, $examples = [])
$this->shouldNot(string $behavior, callable $code = null, $examples = [])

Printer Options

For PHPUnit, add Codeception\Specify\ResultPrinter printer into phpunit.xml

<phpunit colors="true" printerClass="Codeception\Specify\ResultPrinter">
</phpunit>

Recommended

License: MIT.

specify's People

Contributors

501st-alpha1 avatar arhell avatar davertmik avatar dizews avatar electricjones avatar evgenyorekhov avatar glensc avatar grahamcampbell avatar hungryzi avatar ischenko avatar jeroendedauw avatar moltam avatar montogeek avatar mortonfox avatar naktibalda avatar namek avatar purwandi avatar sergeyklay avatar tankist avatar tavoniievez avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

specify's Issues

output the specification for report to for detailed tests info

can you add enhancement to this BDD library like when you run

phpunit

or

codecept run unit

you get the specification (description) of any specify block one by one and the percent of the passed specifications like :

username is required ---------------------------- [ X ]
username is too long ---------------------------- [ X ]
username is OK ---------------------------------- [   ]

66 % passed
[===== ===== ===== ===== ===== ===== ===++ +++++ +++++ +++++]

Failed test can't be re runned if the failed test use Codeception/Specify

What are you trying to achieve?

Re run the failed tests:

./codecept run -g failed

What do you get instead?

Failed tests that use Codeception/Specify won't run. This problem only affects these tests.

How to reproduce

Create test file:

class FailedTest extends \Codeception\Test\Unit
{
    use \Codeception\Specify;

    public function testWithSpecify()
    {
        $this->specify('Specification', function () {
            $this->assertFalse(true, 'Message 1');
        });
    }

    public function testWithoutSpecify()
    {
        $this->assertFalse(true, 'Message 2');
    }
}

Run tests:

$ ./codecept run
Codeception PHP Testing Framework v2.2.3
Powered by PHPUnit 5.4.8 by Sebastian Bergmann and contributors.

Acceptance Tests (0) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Functional Tests (0) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Unit Tests (2) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
✖ FailedTest: With specify | specification | examples index 0 
✔ FailedTest: With specify (0.01s)
✖ FailedTest: Without specify (0.00s)
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------


Time: 167 ms, Memory: 8.00MB

There were 2 failures:

---------
1) FailedTest: With specify | specification | examples index 0
 Test  tests/unit/FailedTest.php:testWithSpecify | Specification | examples index 0
Message 1
Failed asserting that true is false.
Codeception/Codeception#1  /mnt/store-ssd/work/codeception/tests/unit/FailedTest.php:11
Codeception/Codeception#2  FailedTest->{closure}
Codeception/Codeception#3  /mnt/store-ssd/work/codeception/tests/unit/FailedTest.php:12
Codeception/Codeception#4  FailedTest->testWithSpecify

---------
2) FailedTest: Without specify
 Test  tests/unit/FailedTest.php:testWithoutSpecify
Message 2
Failed asserting that true is false.
Codeception/Codeception#1  /mnt/store-ssd/work/codeception/tests/unit/FailedTest.php:17
Codeception/Codeception#2  FailedTest->testWithoutSpecify

FAILURES!
Tests: 2, Assertions: 2, Failures: 2.

_output/failed now contains the tests:

tests/unit/FailedTest.php:testWithSpecify | Specification | examples index 0
tests/unit/FailedTest.php:testWithoutSpecify

Run failed tests:

$ ./codecept run -g failed
Codeception PHP Testing Framework v2.2.3
Powered by PHPUnit 5.4.8 by Sebastian Bergmann and contributors.
[Groups] failed 

Acceptance Tests (0) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Functional Tests (0) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Unit Tests (1) --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
✖ FailedTest: Without specify (0.01s)
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------


Time: 195 ms, Memory: 8.00MB

There was 1 failure:

---------
1) FailedTest: Without specify
 Test  tests/unit/FailedTest.php:testWithoutSpecify
Message 2
Failed asserting that true is false.
Codeception/Codeception#1  /mnt/store-ssd/work/codeception/tests/unit/FailedTest.php:17
Codeception/Codeception#2  FailedTest->testWithoutSpecify

FAILURES!
Tests: 1, Assertions: 1, Failures: 1.

Note: Only testWithoutSpecify runned.

Possible solution

I did some digging, and maybe I found the problem. I think that in Lib\GroupManager an strpos() call's parameters are swapped.

The $testPattern contains: /mnt/store-ssd/work/codeception/tests/unit/FailedTest.php:testWithSpecify | Specification | examples index 0.
And $filename . ':' . $test->getName(false) is: /mnt/store-ssd/work/codeception/tests/unit/FailedTest.php:testWithSpecify.

So the strpos() returns false. But if I swap the two parameters, the all the failed tests are re runned.

I am new to Codeception, so I might be wrong (I haven't set up the tests, so I can't verify that this change doesn't break anything).

Details

  • Codeception version: 2.2.3
  • PHP Version: 7.0.8
  • Operating System: Ubuntu 16.04.1 LTS
  • Installation type: Composer
  • List of installed packages (composer show)
behat/gherkin                      v4.4.1 Gherkin DSL parser for PHP 5.3
codeception/codeception            2.2.3  BDD-style testing framework
codeception/specify                0.4.3  BDD code blocks for PHPUnit and Codeception
doctrine/instantiator              1.0.5  A small, lightweight utility to instantiate objects in PHP without invoking their constructors
facebook/webdriver                 1.1.2  A PHP client for WebDriver
guzzlehttp/guzzle                  6.2.1  Guzzle is a PHP HTTP client library
guzzlehttp/promises                1.2.0  Guzzle promises library
guzzlehttp/psr7                    1.3.1  PSR-7 message implementation
myclabs/deep-copy                  1.5.1  Create deep copies (clones) of your objects
phpdocumentor/reflection-common    1.0    Common reflection classes used by phpdocumentor to reflect the code structure
phpdocumentor/reflection-docblock  3.1.0  With this component, a library can provide support for annotations via DocBlocks or otherwise retrieve information that is embedded in a DocBlock.
phpdocumentor/type-resolver        0.2    
phpspec/prophecy                   v1.6.1 Highly opinionated mocking framework for PHP 5.3+
phpunit/php-code-coverage          4.0.1  Library that provides collection, processing, and rendering functionality for PHP code coverage information.
phpunit/php-file-iterator          1.4.1  FilterIterator implementation that filters files based on a list of suffixes.
phpunit/php-text-template          1.2.1  Simple template engine.
phpunit/php-timer                  1.0.8  Utility class for timing
phpunit/php-token-stream           1.4.8  Wrapper around PHP's tokenizer extension.
phpunit/phpunit                    5.4.8  The PHP Unit Testing framework.
phpunit/phpunit-mock-objects       3.2.3  Mock Object library for PHPUnit
psr/http-message                   1.0    Common interface for HTTP messages
sebastian/code-unit-reverse-lookup 1.0.0  Looks up which function or method a line of code belongs to
sebastian/comparator               1.2.0  Provides the functionality to compare PHP values for equality
sebastian/diff                     1.4.1  Diff implementation
sebastian/environment              1.3.7  Provides functionality to handle HHVM/PHP environments
sebastian/exporter                 1.2.2  Provides the functionality to export PHP variables for visualization
sebastian/global-state             1.1.1  Snapshotting of global state
sebastian/object-enumerator        1.0.0  Traverses array structures and object graphs to enumerate all referenced objects
sebastian/recursion-context        1.0.2  Provides functionality to recursively process PHP variables
sebastian/resource-operations      1.0.0  Provides a list of PHP built-in functions that operate on resources
sebastian/version                  2.0.0  Library that helps with managing the version number of Git-hosted PHP projects
symfony/browser-kit                v3.1.3 Symfony BrowserKit Component
symfony/console                    v3.1.3 Symfony Console Component
symfony/css-selector               v3.1.3 Symfony CssSelector Component
symfony/dom-crawler                v3.1.3 Symfony DomCrawler Component
symfony/event-dispatcher           v3.1.3 Symfony EventDispatcher Component
symfony/finder                     v3.1.3 Symfony Finder Component
symfony/polyfill-mbstring          v1.2.0 Symfony polyfill for the Mbstring extension
symfony/yaml                       v3.1.3 Symfony Yaml Component
webmozart/assert                   1.0.2  Assertions to validate method input/output with nice error messages.
  • Suite configuration:

I have set up a new project to demonstrate this. I've installed Codeception and Specify with composer (composer require codeception/codeception codeception/specify), and I've run bootstrap: ./codecept bootstrap.

So, there is nothing special, default installation:

actor: Tester
paths:
    tests: tests
    log: tests/_output
    data: tests/_data
    support: tests/_support
    envs: tests/_envs
settings:
    bootstrap: _bootstrap.php
    colors: true
    memory_limit: 1024M
extensions:
    enabled:
        - Codeception\Extension\RunFailed
modules:
    config:
        Db:
            dsn: ''
            user: ''
            password: ''
            dump: tests/_data/dump.sql
# Codeception Test Suite Configuration
#
# Suite for unit (internal) tests.

class_name: UnitTester
modules:
    enabled:
        - Asserts
        - \Helper\Unit

Specify exhausts available memory

I've started using Specify and Verify in a new project and find that progressively it exhausts all available memory. Here's a simple class, whose test is called after 4 other tests.

<?php

namespace App\Entity;

class Name extends Base
{
    protected $first;
    protected $last;

    public function getFirst()
    {
        return $this->first;
    }

    public function getLast()
    {
        return $this->last;
    }
}

And here is the test for it:

use \App\Entity\Name;

class NameTest extends \Codeception\TestCase\Test
{
    use \Codeception\Specify;

    protected $tester;
    protected $faker;

    protected function _before()
    {
        $this->faker = Faker\Factory::create();
    }

    protected function _after()
    {
        $this->faker = null;
    }

    public function testCanInstantiateANameObject()
    {
        $properties = [
            'first' => $this->faker->firstName,
            'last' => $this->faker->lastName
        ];
        $name = new Name($properties);
        $this->specify("Can initialize and hydrate a name object", function() use($name, $properties) {
            $this->assertSame($properties['first'], $name->getFirst());
            $this->assertSame($properties['last'], $name->getLast());
        });
    }
}

This will exhaust memory. But removing the specify closure, and just calling the two assertSame methods doesn't. Any help on resolving this would be appreciated.

It will be slow when use many test function and specify

when I use many test function such as follow,it wiil be very slow and use much memory。

qq 20160127144804

public function testRead() {
    codecept_debug('11');
    $this->specify('can read roomTable', function () {
        codecept_debug('22');
    });
    codecept_debug('33');
    $this->specify('can read roomTable', function () {
        codecept_debug('22');
    });
    codecept_debug('33');
}

public function testRead2() {
    codecept_debug('11');
    $this->specify('can read roomTable', function () {
        codecept_debug('22');
    });
    codecept_debug('33');
    $this->specify('can read roomTable', function () {
        codecept_debug('22');
    });
    codecept_debug('33');
}

public function testRead3() {
    codecept_debug('11');
    $this->specify('can read roomTable', function () {
        codecept_debug('22');
    });
    codecept_debug('33');
    $this->specify('can read roomTable', function () {
        codecept_debug('22');
    });
    codecept_debug('33');
}

public function testRead4() {
    codecept_debug('11');
    $this->specify('can read roomTable', function () {
        codecept_debug('22');
    });
    codecept_debug('33');
    $this->specify('can read roomTable', function () {
        codecept_debug('22');
    });
    codecept_debug('33');
}

public function testRead5() {
    codecept_debug('11');
    $this->specify('can read roomTable', function () {
        codecept_debug('22');
    });
    codecept_debug('33');
    $this->specify('can read roomTable', function () {
        codecept_debug('22');
    });
    codecept_debug('33');
}

public function testRead6() {
    codecept_debug('11');
    $this->specify('can read roomTable', function () {
        codecept_debug('22');
    });
    codecept_debug('33');
    $this->specify('can read roomTable', function () {
        codecept_debug('22');
    });
    codecept_debug('33');
}

public function testRead7() {
    codecept_debug('11');
    $this->specify('can read roomTable', function () {
        codecept_debug('22');
    });
    codecept_debug('33');
    $this->specify('can read roomTable', function () {
        codecept_debug('22');
    });
    codecept_debug('33');
}

Do not display "examples index" when examples are not set

Hi! I have some nested specify blocks with no examples set, but i get this output bloated with "examples index 0" stuff:

ExtractorTest::tests | extracts package spo | **examples index 0** | extracts agent discount percent | **examples index 0** | when percent is set directly in tour | **examples index 0**

Definitely there is a way to show "examples index" only when examples are set.

Type error when using Codeception 5th version

More information:
If you uses specify out of box (as described in github page), it will throw an TypeError: Codeception\Specify\SpecifyTest::run(): Return value must be of type PHPUnit\Framework\TestResult, null returned

I have looked in and noticed that TestResult object was got from TestCase object. That how it works in 4th version. And codeception runs test using TestCase::run method. But in 5th version it uses TestCaseWrapper. That's why there is not TestResult object in TestCase to give it in Specify

Codeception has default configurations (just after codecept bootstrap)

Please support php 7.1

In ResultPrinter.php line 24:

Declaration of Codeception\Specify\ResultPrinter::writeProgress($progress): void should be compatible with PHPUnit\TextUI\ResultPrinter::writeProgress(string $progress): void

Code inside "specify" closure changes the value of the class attribute

Hello. I have the following test class:

class ValidatorTest extends \Codeception\TestCase\Test
{
    use \Codeception\Specify;

    /**
     * @var \mb\tools\url\Validator $instance ;
     */
    public $instance;

    /**
     * @var \UnitTester
     */
    protected $tester;

    /**
     * @inheritdoc
     */
    protected function _before()
    {
        $this->instance = new \mb\tools\url\Validator();
        $this->assertInstanceOf('\mb\tools\url\Validator', $this->instance);
    }

    /**
     * @covers \mb\tools\url\Validator::url
     */
    public function testUrl()
    {
        $dataDir = \Codeception\Configuration::dataDir();
        $urlSamples = include("{$dataDir}/url_samples.php");

        var_dump($this->instance->validSchemes);

        $this->specify('check URLs with invalid scheme if $validSchemes property is set', function ($testData) {
            $this->instance->validSchemes = ['https', 'ftps'];
            verify($this->instance->url($testData))->false();
        }, ['examples' => $urlSamples['restrictedScheme']]);

        var_dump($this->instance->validSchemes);
        die;
    }
}

The results are:

array(2) {
  [0] => string(4) "http"
  [1] => string(5) "https"
}
array(2) {
  [0] => string(5) "https"
  [1] => string(4) "ftps"
}

Means that $this->instance class attribute has been changed. What am I doing wrong?

For now, I was able to workaround this with help of

        $this->beforeSpecify(function() {
            $this->instance = new \mb\tools\url\Validator();
        });

But it doesn't looks like correct solution.

Specify does not isolate mock counters

In general, the specify isolation is great. Unfortunately, the use of mock invocation counters limits the usefulness of specify.

(Test style sacrificed for clarity here)

Given a test of class 'BaseModel'

class BaseModelClass
{

    public function save($data = array(), $useValidation = true){
           if($useValidation){
                $this->validate($data);
           }

          //some save logic
     }

     public function validate(){
         // does some validation logic
     }

}

And a test class using specify

class BaseModelClassTest extends \Codeception\TestCase\Test
{

    use \Codeception\Specify;

    public function TestSaveMethod(){

        $data = array('key'=>'value');

        $this->specify('Should never call validate unless requested', function() use($data){

             $Model = Stub::makeEmptyExcept('BaseModel', 'save', array(
                      'validate'=>Stub::never();
             ));

             $result = $Model->save($data, false);

        });

        $this->specify('Should call validate once when requested', function() use($data){

             $Model = Stub::makeEmptyExcept('BaseModel', 'save', array(
                      'validate'=>Stub::once();
            ));

             $result = $Model->save($data, true);

        });

    }
}

This test will fail on the stub counter for validate.

recursion error when specifying an anonymous function

I have this piece of code:

<?php

use Codeception\Specify;

class IntrospectTest extends \Codeception\TestCase\Test
{
    use Specify;

    /**
    * @var \UnitTester
    */
    protected $tester;

    protected function _before()
    {

    }

    protected function _after()
    {
    }

    // tests
    public function testGetColumns()
    {
        $this->specify("this works");

        $this->specify("this doesnt' work", function() {
            return true;
        });
    }

}

Seems like the most basic example. When running the suite I get a recursion error on the specify where I define an anonymous function: http://pastebin.com/maXdaJsi . It seems related to the deep copying, but I don't know why as I'm not doing anything special yet. Any suggestions?

Steps report generation

What do you think about having an option like --steps
to be able to prepare some report on test passing
as in BDD style each step is quite well described with human readable strings
to have something like report here http://vowsjs.org/

          A non-promise return value
            ✓ should be converted to a promise
          A topic not emitting an error
            ✓ should pass null if the test is expecting an error
            ✓ should pass the result otherwise

it seems that it require introduction of additional function to Specify
$this->describe('some text') - just a simple function to add some grouping description (or maybe to use some phpdoc comment with desciption as tests are already grouped by testMethods, to avoid additional closure nesting)

Output is not cleared after specify

Hi.

I've a test:

public function testSomething()
{
    $this->specify(
        'expect output',
        function () {
            $this->expectOutputString('lorem ipsum');
            (new TestedClass)->doSomething();
        }
    );
    $this->specify(
        'expect another output',
        function () {
            $this->expectOutputString('lorem ipsum');
            (new TestedClass)->doSomethingElse();
        }
    );
}

The first specification passes.
Second specification is failing. It expect output to be "lorem ipsum 2", but gots "lorem ipsum 1lorem ipsum 2".

Fatal Error: Call to member function CeptTest::addFailure on null

Hello.

I am working on configuring and installing Codeception alongside Yii 1.1. So far, I have been successfully able to do this, and I am working on building a few unit test examples for my team using Specify and Verify.
However, on executing the unit test suite from codeception, PHP returns a fatal error that it reports is occurring within Specify.

I am using:

  • Yii 1.1
  • Codeception 2.3
  • Specify 0.4.6
  • Verify 0.3.3

/opt/snafu/protected/tests/codeception.yml

actor: Dev
# Note the context for this config is relative to
# /opt/snafu/protected/tests/codeception
paths:
    tests: codeception/tests
    log: codeception/log
    data: codeception/tests/_data
    helpers: codeception/tests/_helpers
    output: codeception/tests/_output
    support: codeception/tests/_support
    envs: codeception/tests/_envs
settings:
    bootstrap: ../../../bootstrap.php
    colors: true
    memory_limit: 1024M
    log: true
    debug: true
modules:
    enabled: [Yii1]
    config:
        PhpBrowser:
            url: http://localhost/index.php
        Yii1:
            appPath: /opt/snafu/test.php
            url: http://localhost/test.php
            part: init
actor_suffix: Tester
extensions:
    enabled:
        - Codeception\Extension\RunFailed

/opt/snafu/protected/tests/codeception/tests/unit.suite.yml

# Codeception Test Suite Configuration
#
# Suite for unit or integration tests.

actor: UnitTester
modules:
    enabled:
        - Yii1:
            part: init
        - Asserts
        - \Helper\Unit

/opt/snafu/protected/tests/codeception/tests/unit/FirstCest.php

<?php


class FirstCest extends \Codeception\Test\Unit
{
    use Codeception\Specify;
    
    private $adminEmail;

    public function _before(UnitTester $I)
    {
    }

    public function _after(UnitTester $I)
    {
    }

    // tests
    public function testAdminEmailParam()
    {
        $this->adminEmail = Yii::app()->params['adminEmail'];

        $this->specify("adminEmail is required", function(){
            $this->assertTrue(isset($this->adminEmail));
        });

        $this->specify("adminEmail is in email format", function(){
            $this->assertTrue(filter_var($this->adminEmail, FILTER_VALIDATE_EMAIL));
        });
    }
}

When I execute codeception from within the tests directory, this is the output that the unit tester will generate.

root@3b2cc148fa0a:/opt/snafu/protected/tests# codecept run unit
Codeception PHP Testing Framework v2.3.3
Powered by PHPUnit 4.8.36 by Sebastian Bergmann and contributors.

Unit Tests (208) ------------------------------------------------------------------------------------------------------------------------
FirstCest: Test admin email param
Signature: FirstCest:testAdminEmailParam
Test: codeception/tests/unit/FirstCest.php:testAdminEmailParam
Scenario --

Fatal error: Call to a member function addFailure() on null in /opt/snafu/vendor/codeception/specify/src/Codeception/Specify.php on line 130



FATAL ERROR. TESTS NOT FINISHED.
Call to a member function addFailure() on null 
in /opt/snafu/vendor/codeception/specify/src/Codeception/Specify.php:130

If it would be helpful in reproducing, this is the (tagged) work space I am using, complete with a Dockerfile for duplicating the issue I am experiencing.
https://github.com/mashiox/yii-codecept/releases/tag/specify-addFailure-issue

In order to simplify this process, please ignore the developer scripts Makefile, mount.sh, and start.sh and bootstrap the container using:

# Build the image, from within the cloned repository.
docker build -t codecept-test .

# Run the codecept-test container
docker run -it codecept-test

# Grab the container's name with docker ps, then
# Shell into the container with docker exec
docker exec -it "$(docker ps --format "{{.Image}} {{.Names}}" | grep codecept-test | cut -d" " -f2)" /bin/bash

# Execute the tests --within the container--, done in one of two ways
# Either should generate the error. 
cd /opt/snafu/protected/tests
codecept run unit

codecept --config=/opt/snafu/protected/tests/codeception.yml run unit

Empty `specify()` and `describe()` blocks pass tests

The documentation states that specify() and describe() calls that don't pass anything to $fn will be marked as incomplete, but when I run them they are marked OK. These tests should either fail or be marked as risky to ensure that at least some assertions are being made.

$this->specify('some test');

CLI output:

ExampleTest ✔︎ ✔︎ ✔︎ ✔︎ ✔︎

use Codeception\Specify in Cest classes

I would really like to use Codeception\Specify in Cest classes (in a functional test suite / TestHelper). The goal is to have PHPUnit style DataProvider functionality (in other words, using your "examples" functionality).

When I import the trait:

class SomeCest
{
    use Codeception\Specify;

I'm getting a "Call to undefined method" error:

Fatal error: Call to undefined method Codeception\Scenario::bindTo() in /path_to_project/vendor/codeception/codeception/src/Codeception/Lib/Parser.php(49) : eval()'d code on line 1

Call Stack:
    0.0005     221016   1. {main}() /path_to_project/vendor/codeception/codeception/codecept:0
    0.0394    1137776   2. Symfony\Component\Console\Application->run(???, ???) /path_to_project/vendor/codeception/codeception/codecept:27
    0.0472    1243920   3. Symfony\Component\Console\Application->doRun(class Symfony\Component\Console\Input\ArgvInput, class Symfony\Component\Console\Output\ConsoleOutput) /path_to_project/vendor/symfony/console/Symfony/Component/Console/Application.php:126
    0.0473    1244840   4. Symfony\Component\Console\Application->doRunCommand(class Codeception\Command\Run, class Symfony\Component\Console\Input\ArgvInput, class Symfony\Component\Console\Output\ConsoleOutput) /path_to_project/vendor/symfony/console/Symfony/Component/Console/Application.php:195
    0.0474    1245312   5. Symfony\Component\Console\Command\Command->run(class Symfony\Component\Console\Input\ArgvInput, class Symfony\Component\Console\Output\ConsoleOutput) /path_to_project/vendor/symfony/console/Symfony/Component/Console/Application.php:874
    0.0477    1250304   6. Codeception\Command\Run->execute(class Symfony\Component\Console\Input\ArgvInput, class Symfony\Component\Console\Output\ConsoleOutput) /path_to_project/vendor/symfony/console/Symfony/Component/Console/Command/Command.php:252
    0.1035    1781376   7. Codeception\Codecept->run(string(10), string(25)) /path_to_project/vendor/codeception/codeception/src/Codeception/Command/Run.php:185
    0.1114    1810312   8. Codeception\Codecept->runSuite(array(14), string(10), string(25)) /path_to_project/vendor/codeception/codeception/src/Codeception/Codecept.php:134
    4.2402    3249544   9. Codeception\SuiteManager->loadTests(string(25)) /path_to_project/vendor/codeception/codeception/src/Codeception/Codecept.php:150
    4.2658    3696424  10. Codeception\SuiteManager->addToSuite(class Codeception\TestCase\Cest) /path_to_project/vendor/codeception/codeception/src/Codeception/SuiteManager.php:98
    4.2659    3696424  11. Codeception\TestCase\Cest->preload() /path_to_project/vendor/codeception/codeception/src/Codeception/SuiteManager.php:125
    4.2663    3699528  12. Codeception\Lib\Parser->parseScenarioOptions(string(1177), string(8)) /path_to_project/vendor/codeception/codeception/src/Codeception/TestCase/Cest.php:39
    4.2663    3704720  13. eval('$callable->bindTo($this);') /path_to_project/vendor/codeception/codeception/src/Codeception/Lib/Parser.php:49

My guess is that Codeception\Specify is designed to work with PHPUnit_Framework_TestCase (or Codeception\TestCase\Test which extends PHPUnit_Framework_TestCase), but the Cept doesn't extend that class.

Is using Codeception\Specify in Cept classes even possible?

If so, any tips on how to solve this issue?

requires phpunit/phpunit version maybe error.

I use Yii2 which requires codeception/*,but just for version is old.
It will be has [Error] Class 'PHPUnit_Framework_Assert' not found
So I upgrade the codeception/*.
But I find that codeception/specify 1.0 comoser.json is not same with codeception/verify 1.0.0
Maybe phpunit/phpunit:~6.0 should change to `phpunit/phpunit:>6.0'

Orelse phpunit will be force to install v6.

Problem 1
    - codeception/specify 1.0 requires phpunit/phpunit ~6.0 -> satisfiable by phpunit/phpunit[6.0.0, 6.0.1, 6.0.10, 6.0.11, 6.0.12, 6.0.13, 6.0.2, 6.0.3, 6.0.4, 6.0.5, 6.0.6, 6.0.7, 6.0.8, 6.0.9, 6.0.x-dev, 6.1.0, 6.1.1, 6.1.2, 6.1.3, 6.1.4, 6.1.x-dev, 6.2.0, 6.2.1, 6.2.2, 6.2.3, 6.2.4, 6.2.x-dev, 6.3.0, 6.3.1, 6.3.x-dev, 6.4.0, 6.4.1, 6.4.2, 6.4.3, 6.4.4, 6.4.x-dev, 6.5.0, 6.5.1, 6.5.2, 6.5.3, 6.5.4, 6.5.5, 6.5.6, 6.5.7, 6.5.x-dev] but these conflict with your requirements or minimum-stability.

If phpunit is v6.There will be error:

Codeception PHP Testing Framework v2.4.0
Powered by PHPUnit 6.5.7 by Sebastian Bergmann and contributors.
PHP Fatal error:  Declaration of Codeception\PHPUnit\Listener::addError(PHPUnit\Framework\Test $test, Exception $e, $time) must be compatible with PHPUnit\Framework\TestListener::addError(PHPUnit\Framework\Test $test, Throwable $t, float $time): void in /home/lijun/works/basic/vendor/codeception/phpunit-wrapper/src/Listener.php on line 13
PHP Stack trace:
PHP   1. {main}() /home/lijun/.config/composer/vendor/codeception/codeception/codecept:0
PHP   2. Codeception\Application->run() /home/lijun/.config/composer/vendor/codeception/codeception/codecept:42
PHP   3. Codeception\Application->run() /home/lijun/.config/composer/vendor/codeception/codeception/src/Codeception/Application.php:108
PHP   4. Codeception\Application->doRun() /home/lijun/.config/composer/vendor/symfony/console/Application.php:143
PHP   5. Codeception\Application->doRunCommand() /home/lijun/.config/composer/vendor/symfony/console/Application.php:241
PHP   6. Codeception\Command\Run->run() /home/lijun/.config/composer/vendor/symfony/console/Application.php:865
PHP   7. Codeception\Command\Run->execute() /home/lijun/.config/composer/vendor/symfony/console/Command/Command.php:252
PHP   8. Codeception\Codecept->__construct() /home/lijun/.config/composer/vendor/codeception/codeception/src/Codeception/Command/Run.php:352
PHP   9. Codeception\Codecept->registerPHPUnitListeners() /home/lijun/.config/composer/vendor/codeception/codeception/src/Codeception/Codecept.php:87
PHP  10. spl_autoload_call() /home/lijun/.config/composer/vendor/codeception/codeception/src/Codeception/Codecept.php:107
PHP  11. Composer\Autoload\ClassLoader->loadClass() /home/lijun/.config/composer/vendor/codeception/codeception/src/Codeception/Codecept.php:107
PHP  12. Composer\Autoload\includeFile() /home/lijun/.config/composer/vendor/composer/ClassLoader.php:322
PHP  13. include() /home/lijun/.config/composer/vendor/composer/ClassLoader.php:444

Show test specification text when it fails

One thing that is missing for me is the feature to write the specification name in the console when the test fails. This way when the test fails I have a nice descriptive name of the specification that failed.

For example I have this test

    public function testFoo(): void
    {
        $this->specify('Should fail and display this text in the console', function () {
            $this->assertTrue(false);
        });
    }

Is this possible to have?

Couldn't update?

Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - The requested package codeception/specifys could not be found in any version, there may be a typo in the package name.

Potential causes:
 - A typo in the package name
 - The package is not available in a stable-enough version according to your minimum-stability setting
   see <https://groups.google.com/d/topic/composer-dev/_g3ASeIFlrc/discussion> for more details.

Read <http://getcomposer.org/doc/articles/troubleshooting.md> for further common problems.

Undefined property: UserTest::$name

I get an exception thrown when assertion fails inside specify block. See an example below:

<?php
class UserTest extends \Codeception\TestCase\Test
{

    use \Codeception\Specify;
   /**
    * @var \CodeGuy
    */
    protected $guy;
    public function testSave()
    {
        $this->specify('this will fail', function(){
            $this->assertTrue(false);
        });
    }
}
?>

"with data set" message duplicating in test name

I have a test with dataProvider and 4 "specify" calls inside. That made test log look really weird:

Codeception PHP Testing Framework v2.0.16
Powered by PHPUnit 4.7.7 by Sebastian Bergmann and contributors.

Tests\codeception\common.unit Tests (1) ----------------------------------------------------------------------------------------------
Test parse path | #0 | #0 | #0 | #0 | (tests\codeception\...Test::testParsePath with data set #0 with data set #0 with data set #0 with data set #0 | SPECIFY NAME | examples index 0)Fail
Test parse path | #0 | #0 | #0 | #0  (tests\codeception\...Test::testParsePath with data set #0 with data set #0 with data set #0 with data set #0)Ok
--------------------------------------------------------------------------------------------------------------------------------------


Time: 118 ms, Memory: 12.75Mb

There was 1 failure:

---------
1) tests\codeception\...Test::testParsePath with data set #0 with data set #0 with data set #0 with data set #0 | SPECIFY NAME | examples index 0 with data set #0
Failed asserting that ...

Look at that "with data set #0 with data set #0 with data set #0 with data set #0" in test name.
If I comment one "specify" block 0 there's only 3 "with data set #0"'s in test name.

specify version 0.4.3

[enhancement] Test message when exception is thrown

It would be great to be able to:

$this->specify("it throws an exception without a default connection", function () {
   // Do something
}, ['throws' => 'Spider\Exceptions\ConnectionNotFoundException', 'message' => 'The exception message']);

And I'm happy to put together a PR if you let me know what syntax/api you would like (ie message, exception message [throws => ['class', 'message']] as well as ['throws' => 'class']

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.