Cucumber Scala final output shows 0 failures even when a scenario fails about cucumber-jvm-scala HOT 12 CLOSED

@ejaoua you might want to take it to Slack to discuss these kind of things, or open a new issue/discussion at least.

Short answer: I don't think that's possible this way. You have to run testOnly SomeClass where SomeClass targets only some features file.

from cucumber-jvm-scala.

Hello @rhande-mdsol ,

Could it be that you have multiple scenarios with the same name (in the same file), or Scenario Outlines?

If so, this is is a known issue in sbt-junit interface:
sbt/junit-interface#84. See also #22

from cucumber-jvm-scala.

Might be worth adding a work around to detect if sbt-junit is used and act accordingly. The project seems dead.

from cucumber-jvm-scala.

Or ensure the names provided to the JUnit Description are always unique. That way sbt will also be able to handle identical features, scenarios, examples, ect. Shouldn't be too much of a burden if the #n suffix only shows up when necessary.

from cucumber-jvm-scala.

Would either of you mind checking if cucumber/cucumber-jvm#2045 solves the problem.

from cucumber-jvm-scala.

Sorry Closed by mistake.

from cucumber-jvm-scala.

@mpkorstanje I am using

"io.cucumber" % "cucumber-core" % "5.7.0",
"io.cucumber" % "cucumber-junit" % "5.7.0"
"io.cucumber" %% "cucumber-scala" % "5.7.0"

So it is cucumber scala. Your code in cucumber/cucumber-jvm#2045
seems to be Java code. Can I use it ?

I will try publishing your code and let you know

from cucumber-jvm-scala.

@rhande-mdsol Cucumber Scala is using Cucumber JVM which is the foundation for various Cucumber libraries targeting the JVM.

In this case, it's the project cucumber-junit that you could try to build on your own with the PR mentioned by @mpkorstanje and use in your project.

I will also try the PR on my side but probably later today.

@rhande-mdsol Do you confirm that your case is multiple scenarios with the same name in the same file, or using Scenario Outlines ?

from cucumber-jvm-scala.

@rhande-mdsol Cucumber Scala is using Cucumber JVM which is the foundation for various Cucumber libraries targeting the JVM.

In this case, it's the project cucumber-junit that you could try to build on your own with the PR mentioned by @mpkorstanje and use in your project.

I will also try the PR on my side but probably later today.

@rhande-mdsol Do you confirm that your case is multiple scenarios with the same name in the same file, or using Scenario Outlines ?

Hi @gaeljw ,

Yes, this problem happens when I have a Scenario Outline: and then have a table like structure in Examples: section. For me 3rd example actually failed, but final result showed all tests passed.
Also this happens when you have a Step with parameterized value and reuse the step in the same Feature file at multiple places.

For example :

Scenario 1 =>

Scenario:
Given I know "Java"
Then I teach "Java"

Scenario 2 =>

Scenario:
Given I know "Scala"
Then I teach "Scala"

If Scenario 2 fails here, the final result still shows all have passed.

I also tried failing a scenario without an outline (i.e. Scenario:) in this case final result shows failures properly.

Thanks

from cucumber-jvm-scala.

Thanks for confirming.

cucumber/cucumber-jvm#2045 should fix the issue then.
You just have to wait a bit for the next cucumber-junit release (you'll have to update to 6.x but the upgrade from 5.x to 6.x is pretty easy).

from cucumber-jvm-scala.

Thanks for fixing the problem. Now final result shows failure when there is a failure in scenario outline.

from cucumber-jvm-scala.

Hi @rhande-mdsol ,
can you please share what was the plugin and/or correct syntax that you used to run one test of Cucumber :
sbt "testOnly <cucumber_class_name>".
I tried the following but it didn't work out sbt "testOnly *classpath:features/folder1/endpoint.feature
Thanks.

from cucumber-jvm-scala.