Scenario with both 'Passed' and 'Skipped' steps shows 'Failed' scenario. about cluecumber HOT 12 CLOSED

That is actually true. I will add this in for the next release.
Thanks for reporting!

Best,
Benjamin

from cluecumber.

Fix is included in this branch: https://github.com/trivago/cluecumber-report-plugin/tree/bugfix/wrong-status-on-skipped-and-passed-steps

from cluecumber.

@laxersaz
Thanks for the fix.
Could you please make a sub release, so that I can use this?

One more question:
Is it possible to show "doc_string" below each steps in the report with expand and collapse feature.
Like how it is displaying error message for failed steps, it has to display for all the steps with option to show/hide.

If yes, I will open a new ticket.

Thanks,

from cluecumber.

Yes I will make a sub release very soon for this fix that will also incorporate the correct step hook considerations.
For the feature, please open a new ticket 🙂

from cluecumber.

This will be released in 1.2.1 today.

from cluecumber.

@laxersaz
Apologies.
I realized after trying with the latest release that scenario with "passed" and "skipped" steps should have result as "skipped" since all the steps in that scenario is not "passed".

Please check the attached image. I have purposefully skipped 10 scenarios using karate.abort(), but then the result is shown as "passed".

Thanks,

from cluecumber.

According to the Cucumber developers, in this case a scenario should be passed. Cluecumber considers it skipped if it only consists of skipped or pending steps.

from cluecumber.

@laxersaz

Yes I agree that if a scenario should be skipped if all the steps in it are skipped.

But I observe a behaviour in "Cluecumber" that Background and Scenario steps are merge together to generate a report.

In that case, if any steps in background is passed, then the scenario result will always be passed irrespective of whether all steps in the scenario is 'Skipped'.

Kindly refer the attached two json file.
One feature file(ScenarioStatusSample.feature) with Background and other(ScenarioStatusSample2.feature) without Background. The later one seems to work fine.

target.zip

I think Cluecumber should not combine "Background steps" and "Scenario Steps" as one.
Kindly clarify me.

Thanks

from cluecumber.

Hi @svarunbe ,
Background steps are basically syntactic sugar for steps that are repeated in each scenario. So technically it is correct that the scenario as a whole is marked as passed when Background steps pass and the rest are skipped. Your idea is definitely possible to implement but I guess this will be against the official Cucumber rules. I will ask in the Cucumber OSS group to get some more input on this.

For visibility, I could separate Background steps from normal steps in the report and put them into a separate section.

from cluecumber.

I asked the Cucumber developers and they said the same. So the scenarios with passing background steps and skipped steps after it should be considered passed.

from cluecumber.

@laxersaz
Yes. We have to follow the standards. But I do not see any practical use case where we will skip a scenario from start of "Background" step.
Could it be implemented with a flag as "strict". If the flag set as 'true', it can produce report as cucumber team suggested and if not, the report can produce as how user wants.

Please consider this use case.

Thanks,

from cluecumber.

I created a follow up, also to continue the discussion there:
#74

from cluecumber.