Behat broken logs with --list-scenarios about fastest HOT 8 CLOSED

Hi thanks for reporting,
@diegosainz please could you help us?

from fastest.

Sure, I'm out of the city but as soon as I get back (next week) will look
into it.
On Nov 27, 2014 2:24 AM, "Giulio De Donato" [email protected]
wrote:

Hi thanks for reporting,
@diegosainz https://github.com/diegosainz please could you help us?

—
Reply to this email directly or view it on GitHub
#20 (comment).

from fastest.

I think I may have come across this same issue with JUnit log files, but it has to do more with how the Behat JUnitFormatter generates the log file path in its beforeFeature() method, not necessarily what fastest is doing (other than parallel processing). Essentially, you need to provide a unique directory path to the --out parameter. Otherwise, multiple parallel tests will output to the exact same file path for an individual feature file and overwrite each others content (in the examples you pasted and the ones that I've seen that are similar, the content was not overwritten completely).

The exact command I use is this:

#!/bin/sh -x

mkdir -p app/build/behat_report;
mkdir -p app/build/logs/junit/parallel;
mkdir -p app/build/logs/coverage;

bin/behat --list-scenarios -p continuous_integration src | \
    bin/fastest \
        -vvv \
        --preserve-order \
        --rerun-failed \
        'php app/console doctrine:database:drop --env=test --force; \
        php app/console doctrine:database:create --env=test; \
        php app/console doctrine:schema:update --env=test --force; \
        feature_name=`echo $ENV_TEST_ARGUMENT | sed 's/[\/\.\:]/_/g'`;
        BEHAT_PARAMS="extensions[VIPSoft\\CodeCoverageExtension\\Extension][report][options][target]=app/build/logs/coverage/behat_parallel_${feature_name}.cov" \
        bin/behat \
            --format="progress,junit,html" \
            --out="null,app/build/logs/junit/parallel/${feature_name},app/build/behat_report/behat_report_parallel_${feature_name}.html" \
            -p continuous_integration \
            {} \
        ;' \
;

The key part here is the feature_name variable. It takes the ENV_TEST_ARGUMENT value, which should include the entire feature file path plus the line number of the scenario, and changes slashes, periods, and colons to underscores. This provides a unique enough value for the feature file and the specific scenario that is being executed which could be used as a directory name.

So to adapt to your specific command, I might try something like this:

./bin/behat -c app/config/behat.yml "@AppWebBundle" --list-scenarios |
    ./bin/fastest --verbose --preserve-order
    "feature_name=`echo $ENV_TEST_ARGUMENT | sed 's/[\/\.\:]/_/g'`; ./bin/behat --format=junit --out=build/logs/behat/${feature_name} -c app/config/behat.yml -v {};

Then in your Jenkins configuration, you could change the JUnit report path to something like build/logs/behat/**/*.xml so that it can account for the new sub-directories and find the generated JUnIt *.xml files.

from fastest.

@wcluijt
Thanks, i will try it.

from fastest.

@wcluijt excellent post - @kristijanhusak is this still an issue so I can look into it?

from fastest.

@liuggio I think this issue can be closed as resolved - we have been running tests this way for a while now and Jenkins seems to work OK with the junit output.

from fastest.

I haven't tested @wcluijt solution, stayed with the testing by feature. IF @diegosainz says it is working, it can be closed.

from fastest.

thanks

from fastest.