Code Monkey home page Code Monkey logo

cucumberswift's Introduction

Build Status Pod Platform Pod Version codecov Codacy Badge

Welcome to CucumberSwift

CucumberSwift is a lightweight Swift only Cucumber implementation for iOS, tvOS, and macOS. It was born out of frustration with current iOS Cucumber implementations. The whole goal is to make it easy to install and easy to use, so please feel free to give feedback.

NOTE: WatchOS support coming soon!

What's Still Missing?

  • Some Gherkin language errors

Attributions:

Powered by MacStadium

cucumberswift's People

Contributors

bitsmakerde avatar dependabot[bot] avatar devmobileas avatar hsilgos avatar jcislinsky avatar ondrejstasek avatar rocklobster avatar tyler-keith-thompson avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cucumberswift's Issues

Add a screenshot to scenario in AfterScenario

Is your feature request related to a problem? Please describe.
Since Cucumber does not extend XCTestCase, I am unable to add a screenshot at the end of scenario

Describe the solution you'd like
In AfterScenario Hook, an additional parameter is provided to take screenshot at the start of afterScenario and save. That would be great.

Describe alternatives you've considered

AfterScenario { scenario in
let fullScreenshot = XCUIScreen.main.screenshot()
let screenshot = XCTAttachment(screenshot: fullScreenshot)
screenshot.lifetime = .keepAlways
// if we don't set lifetime to .keepAlways, Xcode will delete the image if the test passes.

           add(screenshot)

In the above code, add method is not accepted as it is used in extension of XCTestCase.

Additional context
This would greatly help in debugging apps when running scenarios on an end to end basis.

Have CucumberSwift clearly report when no feature files are found

Is your feature request related to a problem? Please describe.
Multiple consumers have had problems with understanding whether features are being found in files. This has lead to a pain point with early library adoption.

Describe the solution you'd like
One obvious approach is to have CucumberSwift throw an XCTFail if it discovered no features. Bonus points if it can link to documentation and/or provide tips on what might be wrong.

Describe alternatives you've considered
No alternatives worth mentioning come to mind.

Additional context
It's important to realize it's not just iOS devs who are trying to work with CucumberSwift. This means that we should try and write documentation and errors with as little assumptions as possible while still providing terse, useful output.

Warning shown in xCode 12.0.1

Describe the bug
A very minor issue but happy to put into a PR if you want. But when compiling with the latest release of xCode there is a warning in CucumberSwift.swift file on line 20:

_ = XCTContext.runActivity(named: "\(self.keyword.toString()) \(self.match)") { _ in

The call to XCTContext.runActivity is now a void returning function which means that you can remove the leading _ =

To Reproduce
Steps to reproduce the behavior:

  1. Build the pod on 12.0.1 version of xCode
  2. Observe the below error:

Screenshot 2020-10-15 at 14 52 47

Couldn't execute tests/run project - x86_64-apple-macos vs arm64-apple-macos issue

Describe the bug
After adding CucumberSwift via SPM to our project we were facing with a lot of issues with libraries (our project is very characteristic and has a lot of dependencies) but we only have one left which is related to CucumberSwift:

/Repos/app/UITests/FirstCucumberTests.swift:11:8 Could not find module 'CucumberSwift' for target 'x86_64-apple-macos'; found: arm64-apple-macos, at: /Library/Developer/Xcode/DerivedData/App-eaadrssknjauggcltgqnqhhvhwzo/Build/Products/Local/CucumberSwift.swiftmodule
Path has been changed for security.

To Reproduce
Steps to reproduce the behavior:

  1. Setup CucumberSwift via SPM
  2. Do everything from https://tyler-keith-thompson.github.io/CucumberSwift/tutorials/cucumberswift/spm-step-by-step
  3. Try run tests/run project etc

Expected behavior
Tests should execute, project should run, for sure if should build itself.

Extra info
Tried run Xcode as Rosetta - same result
Tried change architecture for not standard one (like you could find on internet) - same result
Tried changing targets/files/tests/features files etc - same result
Tried clean build folder/ Devired data/ SPM etc with restarting Xcode - same result
Tried previous versions of CucumberSwift (from the beginning of December even) - same result

Additional context
Add any other context about the problem here.

  • XCode Version: 14.2
  • CucumberSwift Version: main
  • Example Feature File - same like in the instruction

Quoted string works incorrect in scenario outline when there is a parameter

When scenario outline parameter is in quotes it's not taken from examples table.

Consider the next example:

Scenario Outline: Test outline
    Given Something is done with "<parameter name>"

    Examples:
      | parameter name  | 
      | value                     |

And Swift implementation:

Given("^Something is done with \"(.*)\"$") { args, _ in
    XCTAssertEqual(args[1], "value")
}

And attempt to run fails with

XCTAssertEqual failed: ("<parameter name>") is not equal to ("value")

This is inconsistent with known to me Cucumber behaviour in other languages.

Looking at Lexer.swift I see that text in quotes like "blah blah" is read as single token and I don't understand why? What is special about quoted string? I haven't found any special about quotes here https://cucumber.io/docs/gherkin/reference/
I would say that when it's clear that it's not Doc String it should not apply any special logic and concatenate everything to previous token?

continueTestingAfterFailure does not work as expected

Hey, first of all, thanks for this up to date Pod!

I am rather new to testing, especially with cucumber, and encountered some problems with failing tests. First of all, something minor I noticed: empty steps are shown as successful tests, is that on purpose?

Anyway, I failed all empty tests explicitly with XCTFail("Not implemented yet") and even though continueTestingAfterFailure is set to false, the tests still continue when a test failed. I looked up the code a bit and it seems the continueTestingAfterFailure / continueAfterFailure values are set, but actually are nowhere checked, is that possible? Maybe I also just overlooked something.

Furthermore, I wanted to ask if it is somehow possible to run single tests once they have been run at least once? As far as I understood, the test cases are generated out of the scenarios. Afterwards they are all shown in the test navigator, but I cannot run them as single tests.

To Reproduce
Steps to reproduce the behavior:

  1. Set continueTestingAfterFailure in the StepImplementation to false.
  2. Run a test with multiple scenarios where one of them fails.
  3. See that the testing still continues after one of them failed.

Expected behavior
The tests stop as soon as one of them fails.

Additional context

  • XCode Version: 12.3
  • CucumberSwift Version: 3.1.1

BeforeFeature and BeforeScenario hooks are ignored for Scenario Outline

Describe the bug
BeforeFeature and BeforeScenario hooks are ignored for Scenario Outline

To Reproduce
Steps to reproduce the behavior:

  1. Create feature file containing Scenario Outline
Feature: Minimal Scenario Outline

@123
Scenario Outline: minimalistic
Given the <what>

Examples:
| what       |
| minimalism |
  1. Add BeforeFeature and BeforeScenario hooks
BeforeFeature { feature in
      print(feature.debugDescription)
 }
        
BeforeScenario { scenario in
      print(scenario.debugDescription)
}
  1. Execute tests

Expected behavior
BeforeFeature and BeforeScenario hooks are executed printing appropriate debugDescription of each entity

Actual behavior
BeforeFeature and BeforeScenario hooks are omitted

Additional context
Add any other context about the problem here.

  • XCode Version: XCode 10.2.1
  • CucumberSwift Version: 2.2.3

Please note that BeforeStep hook is still executed

Need help with setting up CucumberSwift

I have the project setup as below:

Screenshot 2023-01-11 at 11 17 21 AM

When trying to Run CucumberSwiftTestAppUITest, I am not getting any errors although none of the steps are defined. Also, I do not see the test method testGherkin() in the tests navigator.

Also, I see a warning/error in xcode:
/Users/bsathya/Library/Developer/Xcode/DerivedData/CucumberSwiftTestApp-dxhvqbfgfivuqrdpomludwbgwqlx/SourcePackages/checkouts/CucumberSwift/Sources/CucumberSwift/Gherkin/XCode-Specific Couldn't load project at: /Users/bsathya/Library/Developer/Xcode/DerivedData/CucumberSwiftTestApp-dxhvqbfgfivuqrdpomludwbgwqlx/SourcePackages/checkouts/CucumberSwift/Sources/CucumberSwift/Gherkin/XCode-Specific

CucumberTest no longer discovered by Xcode

Xcode no longer detects and runs CucumberTest automatically. So anybody new trying to adopt CucumberSwift is doomed to failure.

Potential fix

If we make the CucumberTest class open we can have consumers inherit from it, forcing discovery

Consequence

Test parallelization will no longer work (it never really has for UI testing targets)...we can deal with that problem a little later. I think we can go back to using Principal Class for the purpose of swizzling the correct methods to get test parallelization, and use the potential fix method for anybody who doesn't care about parallel test execution.

Additional

It's not just CucumberTest, the ObjC "load" method isn't being called either. Other testing libraries may have run into this problem, I'll be investigating them, cause using that load method was a popular way of injecting tests.

We'll need to rev the major version to try and indicate the breaking change. What's weird is that people who already set up CucumberSwift aren't reporting problems and my test harness still works, so this might have something to do with new projects? It's hard to say.

Carthage support

Hello, do you plan to add Carthage support?
I'm very interested to try your lib in your project, but we use Carthage as a dependency manager

Tags do not support colons

Hello (again)!

I've found a small issue ---

If, for example I have the below:

@launchPage:Consumer18_4
  Scenario: Valid load
    ....

The tags property on the Scenario object, only shows having a tag of "launchPage". It seems to be leaving off the second part.

My use-case is, again from creating a framework point of view, I need the developers to be able to define extra meta-data about the scenario. In this case, this tag is performing 2 steps which are extremely boiler-platey that nearly all scenarios will have.

Thanks,
Cody

CucumberSwift doesn't run my feature files

I'm trying to get a basic setup of CucumberSwift up and running but so far I haven't been successful 😕. Since I already spent a few hours investigating I thought someone else could have a look. I don't see what I am missing.
The problem is that CucumberSwift doesn't run the feature file in my Unit Testing bundle. When running the tests it only runs CucumberTest.testGherkin() but nothing else.
Steps to reproduce:

  • Setup an empty project
  • Setup CocoaPods (install CucumberSwift)
  • Add Features folder to the test target by creating a folder reference
  • Add a subclass of StepImplementation

My sample code is here https://github.com/c-st/CucumberSwiftExample

  • XCode Version: 11.4.1 (11E503a)
  • CucumberSwift Version: 2.2.35

Architecture of tests with multiple features

Not actually a bug, however was the most appropriate choice (sorry).

Issue:
Library is forcing us to write all our tests under a single test case which makes storing state difficult. If we have 2 feature files, with a shared step with a data table, what is the best practice to access this data for that specific Scenario.

We are trying to maintain a clean architecture so are splitting up our files to have single responsibility. Currently we can use a singleton to store and share state, however this doesn't feel like best practice.

In our specific case we have a shared Given step that contains the email in the data table, we want to use this step for 2 different Scenarios in 2 different Features, I have written an example of our current setup below, could you please advise on how we should actually be going about this however?

class BaseState {
    static let shared = BaseState()

    var username: String = ""
    var password: String = ""

    func reset() {
        username = ""
        password = ""
    }
}
extension Cucumber: StepImplementation {
    public var bundle: Bundle {
        return Bundle(for: BaseUITests.self)
    }

    var baseState: BaseState {
        return BaseState.shared
    }

    public func setupSteps() {
        AfterScenario { [self] _ in
            self.baseState.reset()
        }

         sharedSteps() // In here we have a Given step with a data table that gives us email and password
         loginFeatureSteps()
         menuFeatureSteps()
   }

   private func sharedSteps() { 
         sharedGivenStep()
   }

We would like to access the data from the table in the shared step in these features below

extension Cucumber {
    public func loginFeatureSteps() {
         givenSteps() 
         whenSteps()
         thenSteps()
   }
extension Cucumber {
    public func menuFeatureSteps() {
         givenSteps()
         whenSteps()
         thenSteps()
   }

We could obviously pass the data through the methods, storing them in the setupSteps method, however this isn't scalable with lots of tests and variables, of which most tests don't care about.

Proposal: Strongly Typed Gherkin

CucumberSwift supports .feature files that it reads from today and parses them. While this is helpful there are times when a more strongly typed Gherkin would be appreciated.

An example of this taken from an article called SwiftyGherkins:

struct StrongStep {
  
  @discardableResult
  init(I handler: () -> Void) {
    handler()
  }
  
  @discardableResult
  init(I handler: @autoclosure () -> Void) {
    handler()
  }
  
}

typealias Given = StrongStep
typealias When = StrongStep
typealias Then = StrongStep

The result:

func test_navigatingToSettings() {
  Given(I: logIn)
  When(I: selectTabBar(tab: .settings))
  Then(I: seeSettingsScreen)
}

The CucumberSwift implementation of this should be a little more in depth but explore what a DSL for Gherkin in Swift would look like.

Data Table does not substitute parameter from Example in Scenario Outline

Describe the bug
When using a Scenario Outline and a step that uses a Data Table, the parameter in the Example is not substituted for the row entry in the data table.

To Reproduce
Steps to reproduce the behavior:

  1. Add a step definition:
Then("^I have a sample step with a=(\\d+),b=(\\d+)$") { matches, step in
    print("Match A = \(matches[1])")
    print("Match B = \(matches[2])")

    let dataTable = step.dataTable!
    for (rowIndex, row) in dataTable.rows.enumerated() {
        for (columnIndex, column) in row.enumerated() {
            print("Row[\(rowIndex)][\(columnIndex)] = \(column)")
        }
    }
}
  1. Add a scenario outline such as:
Scenario Outline: Sample scenario
    Then I have a sample step with a=<paramA>,b=<paramB>
        | a        | b        |
        | <paramA> | <paramB> |

    Examples:
        | paramA | paramB |
        | 0      | 1      |
        | 5      | 10     |

Expected behavior
For the second example, I expected an output like:

Match A = 5
Match B = 10
Row[0][0] = a
Row[0][1] = b
Row[1][0] = 5
Row[1][1] = 10

but instead received an output like:

Match A = 5
Match B = 10
Row[0][0] = a
Row[0][1] = b
Row[1][0] = <paramA>
Row[1][1] = <paramB>

Screenshots
N/A

Additional context
Add any other context about the problem here.

  • XCode Version: XCode 13.3.1
  • CucumberSwift Version: 3.3.21
  • Example Feature File (if applicable)
Feature: Sample

Scenario Outline: Sample scenario
    Then I have a sample step with a=<paramA>,b=<paramB>
        | a        | b        |
        | <paramA> | <paramB> |

    Examples:
        | paramA | paramB |
        | 0      | 1      |
        | 5      | 10     |

Group steps per scenario in test report

Hi again. May I ask if there is a reason why all steps are listed as plain list in test results?
I tried the following:

public final class CucumberTest: XCTestCase {
    override public class var defaultTestSuite: XCTestSuite {
        ....
        // Instead of this:
        // allGeneratedTests.forEach { suite.addTest($0) }
        // Call this:
        generateAlltests(suite)
        return suite
    }

    static func generateAlltests(_ rootSuite: XCTestSuite) {
        for feature in Cucumber.shared.features.taggedElements(with: Cucumber.shared.environment, askImplementor: false) {
            let className = feature.title.toClassString() + readFeatureScenarioDelimiter()

            for scenario in feature.scenarios.taggedElements(with: Cucumber.shared.environment, askImplementor: true) {
                let childSuite = XCTestSuite(name: scenario.title.toClassString())
                var tests = [XCTestCase]()
                createTestCaseFor(className: className, scenario: scenario, tests: &tests)
                tests.forEach{ childSuite.addTest($0) }
                rootSuite.addTest(childSuite)
            }
        }
    }

And my test reports become much nicer, see screenshot.
Does it make sense to continue to improve in this direction?

Screenshot 2023-02-02 at 23 07 56

Cucumber Expressions containing plain texts are mismatched

Describe the bug
Cucumber Expressions containing plain texts are mismatched

To Reproduce
Codes to reproduce the behavior:

Feature file:
Given the member has A
And the member has B
And the member has C

CucumberSwift code:
Given("the member has A" as CucumberExpression) { _, _ in
print("Print the member has A")
}
Given("the member has B" as CucumberExpression) { _, _ in
print("Print the member has B")
}
Given("the member has C" as CucumberExpression) { _, _ in
print("Print the member has C")
}

Expected output
...
Print the member has A
...
Print the member has B
...
Print the member has C
...

Actual output
...
t = 26.88s Given the member has A
Print the member has C
...
t = 0.03s Given the member has B
Print the member has C
...
t = 0.03s Given the member has C
Print the member has C
...

Additional context

  • XCode Version: 13.4.1
  • CucumberSwift Version: 3.3.20

Verbose output option for easier debugging

Is your feature request related to a problem? Please describe.
This comment lead to the issue creation

Describe the solution you'd like
Have some friendly way to enable verbose output from CucumberSwift. Possible approach with environment variables, info.plist settings, or even in-code solutions like we use with the StepImplementation protocol.

Additional context
XCTest can be really finicky about what it puts through its output. I'm tempted to use the new Logger APIs, but that'd only work on iOS 14. There may be a backup I can use instead. More investigation necessary.

Duplicate Symbols for each Test Case

Describe the bug
When running CucumberSwift tests from feature files, the console output contains warnings about duplicate symbols. E.g.
objc[51303]: Class CucumberSwiftLibrary|BeforeFeatureHookWorksCorrectly is implemented in both ?? (0x100fe2ac0) and ?? (0x100fe3850). One of the two will be used. Which one is undefined.
objc[51303]: Class CucumberSwiftLibrary|BeforeFeatureHookWorksCorrectly is implemented in both ?? (0x100fe2ac0) and ?? (0x100fe4260). One of the two will be used. Which one is undefined.

To Reproduce
Steps to reproduce the behavior:

  1. Clone the CucumberSwift repo
  2. Run the test suites
  3. Examine the output

Expected behavior
I would expect no duplicate symbols to be created.

Screenshots
N/A

Additional context
Add any other context about the problem here.

  • XCode Version: [e.g. XCode 13.2.1]
  • CucumberSwift Version: [e.g. 3.3.6]
  • Example Feature File: Files are included in the CucumberSwift Repo.

I took a look through the source code and found that this part

        let uniqueName = { () -> String in
            var count = 0
            var name = className
            while NSClassFromString(name) != nil {
                count += 1
                name = "\(name)\(count)"
            }
            return name
        }()

of TestCaseGenerator.swift is supposed to protect against this but the classes aren't being registered in CucumberTest.swift until after all the test steps for the scenario have been run through.

        scenario.steps.compactMap { step -> (step: Step, XCTestCase.Type, Selector)? in
           if let (testCase, methodSelector) = TestCaseGenerator.initWith(className: className.appending(scenario.title.toClassString()),
                                                                          method: step.method) {
               return (step, testCase, methodSelector)
           }
           return nil
       }
       .map { step, testCaseClass, methodSelector -> (Step, XCTestCase) in
           objc_registerClassPair(testCaseClass)
           return (step, testCaseClass.init(selector: methodSelector))
       }

Moving the objc_registerClassPair up to the compactMap solves the problem but then makes the test navigator output look weird.

use Feature and Scenario for better test results

@Tyler-Keith-Thompson I try make a better test case struct in my project, but I'm not sure which is the way to get this.

Here my first idea:

I create one StepImplementation and import here my Features:

import CucumberSwift

extension Cucumber: StepImplementation {
	public var bundle: Bundle {
		class FindMe {}
		return Bundle(for: FindMe.self)
	}

	public func setupSteps() {
		SaveImageLocallyTests().steps()
		DeleteImageLocallyTests().steps()
	}
}

Inside every feature I have something like this:

import CucumberSwift
import CucumberSwiftExpressions

class DeleteImageLocallyTests: XCTestCase {
	func steps() {
		Feature("Delete an image") {
			Scenario("Delete an image locally") {
				Given("the user has a local stored image" as CucumberExpression) { _, _ in
				}

				When("he delete the image" as CucumberExpression) { _, _ in
				}

				Then("the local store should be empty" as CucumberExpression) { _, _ in
				}
			}
		}
	}
}

For me is to work with feature and scenario very useful to have a good strutted code. but if I look to the test results in Xcode I get a different list, which not use my feature and scenario. Is there a best practice to have all in a good struct?

image

Get result from scenario in afterScenario hook

Hello,

is it possible to get result of scenario, in afterScenario hook? I need it for reporting results to TestRail. At the moment a can not find any option to get the result.

Thank You

Code suggestion should fail without changes

@Tyler-Keith-Thompson I think it can be good behavior if an empty Given, When, Then should be failed. At the moment I write an empty test and its not fail.

I think in TDD should a empty test fail.

Here some possible solutions:

  1. in the code suggestion is automatic an placeholder code like this
    XCTFail("Empty test, please add code to your test")
  2. The Framework search for empty test an create a failure
  3. The Framework throw only a warning that the test is empty, but here the TDD is not complete fulfill.

Thank you

XCTestCase ContinueAfterFailure behavior

There is a wiki page describing how to use ContinueAfterFailure property of XCTestCase but I would like to clarify once again if it works as intended. What I would like to achieve is to step test execution immediately upon failed assertion. As I can see current implementation of CucumberSwift allows me only to stop scenario execution while the step continue running. This artificial example can demonstrate the issue I am facing now:

public var continueTestingAfterFailure:Bool {
    return false
 }

MatchAll("^some step definition$") { _, step in
         XCTFail("I would like to stop now")
         var foo:String?
         print("but it's going to be a fatal error \(foo!)")
      }

while below implementation stops immediately

MatchAll("^some step definition$") { _, step in
         step.continueAfterFailure = false
         XCTFail("indeed stops now")
         var foo:String?
         print("not reachable statement \(foo!)")
      }

And I wonder if existing implementation of CucumberSwift assumed existing behavior of step.continueAfterFailure = false?

"Given something #comment" vs "Given the color is #123456"

In our project I tried defining a color for a test in hex representation.

To Reproduce
The .feature file contained something like this:
"Given the color is #123456"

Actual auto generated stub
Then("^Given the color is$")

What I'd expect to be a correct implementation
Then(CucumberExpression("Given the color is {color}")
(using a custom Parameter implementation)

Additional context

  • XCode Version: 14.2
  • CucumberSwift Version: main branch

I found that CucumberSwift treats the color as just a comment. There seems to be no way to escape the hashtag character. So the only way I found to work around this issue was to put the color into quotes.

After doing some quick research I found this interesting comment:
https://stackoverflow.com/questions/26676543/whats-the-official-way-to-escape-a-comment-in-gherkin-cucumber

Essentially it is quoting some gherkin syntax description:
"Comment lines are allowed anywhere in the file. They begin with zero or more spaces, followed by a hash sign (#) and some amount of text."

I'd try to find the issue in the code and create a PR, but at first I'd like to hear your thoughts about if that's even an issue or implemented as intended?

Kind regards
Björn

Cucumber Expression support

Describe the bug
Cucumber Expression support

To Reproduce
Steps to reproduce the behavior:

  1. Create a function with a Cucumber Expression
    For example:
    Given("the member has {string}") { [self] matches, _ in
    let condition = matches[1]
    }
    Feature file:
    Given the member has "something"

Expected behavior
Expect the Cucumber Expression to be matched, but instead got an error:
invalid regex: The value “the member has {string}” is invalid.

Additional context

  • CucumberSwift Version: 3.3.17
  • XCode Version: 13.3

No error when a Gherkin statement doesn't match any CucumberSwift expressions

Describe the bug
No error when a Gherkin statement doesn't match any CucumberSwift expressions

To Reproduce
Steps to reproduce the behavior:

  1. Add a Gherkin statement
  2. No CucumberSwift expression that matches the Gherkin statement

Expected behavior
Error for Gherkin statement with no matching CucumberSwift expressions

Additional context

  • CucumberSwift Version: 3.3.6

Steps separated as rT and Command line support

Hi, i am trying to implement this to my project but there are some concepts that i can't understand. I read the docs and checked the CucumberSwiftSample but it didnt help.

How i use it:
Screen Shot 2022-11-02 at 10 12 36

As you can see in the image I defined steps inside the test func and expect them to be grouped in the test results with a big T but every single step looks like they are a separate test. My questions are:

  • Is there a way to group them under the func name? Why does every step look like another test case?
  • Is it possible to run any test seperately? Right now if i run any test separately, it gets succeeded without running cucumber steps but if i run them all (cmd+U) cucumber works.
  • When i run tests from the terminal and CI/CD, cucumber also doesn't work. Is there a documentation to follow for this implementation?

Code suggestion for Cucumber Expressions

Code suggestion for Cucumber Expressions. At moment I get only suggestion in regex format.

testGherkin(): No CucumberSwift expression found that matches this step. Try adding the following Swift code to your step implementation file: 
When("^I execute the three \\{(\\d+)\\} tests$") { matches, _ in
    let integer = matches[1]
}

Solution can be a mode which the developer can set in the info plist for change from regex to Cucumber Expressions. On this way we can remove the "as CucumberExpression" in the steps.

@Tyler-Keith-Thompson If you can say me there I can at this support I can help you.

Cucumber does not conform to protocol StepImplementation

I am following your docs and in the docs you implement Given, When and Then. I did the same but was greeted with the error "Type Cucumber does not conform to protocol StepImplementation".

I have a Features folder which contain the following feature:

Feature: Example

Scenario: First Scenario

Given I have a very cool app
When I type my name in the name text field
Then I should see my name displayed in a label 

I have a Steps group with file "WeatherStepsImplementation.swift" which contains the following code.

extension Cucumber: StepImplementation {
    
    public func setupSteps() {
        
        Given("I have a very cool app") { (matches, _) in
            print(matches)
        }
        
        When("I type my name in the name text field") { (matches, _) in
            print(matches)
        }
        
        Then("I should see my name displayed in a label ") { (matches, _) in
            print(matches)
            
        }
        
    }
}

Best Practices for Decoupling Step Files?

Hello,

Based on my understanding of the project, all step definitions need to be defined in the extension of the Cucumber class. Do you have an example of how someone would extend this paradigm such that multiple files can contain the step definitions?

Bonus points if there is a way to do this via extension points that don't require access to the Cucumber class itself

Display Scenario name in test run results

Is your feature request related to a problem? Please describe.
No problem, just a new feature.

Describe the solution you'd like
When I look into test failures to start diagnosing an issue with my project, it would be useful to see the Scenario name tacked on to the Steps that have failed in a test. Something along the lines of this would be incredible, but really any solution would be great:

Failing tests:
AwesomeCucumberTests:
-[Scenario ElementsAreAddedCorrectly] -[Step andIAmShownOnlyElements]
-[Scenario ElementsAreRemovedCorrectly] -[Step andIAmShownOnlyElements]

Describe alternatives you've considered
None

Additional context
None

Stub generation uses deprecated Matcher initializer

Describe the bug
With a .feature line like this:
"Angenommen ich starte die App frisch"

... the automatic step generation actually generates stubs like this:
Given("^ich starte die App frisch$") ...

This creates the following warning:
'init(_:callback:line:file:)' is deprecated: To use regular expressions with CucumberSwift please migrate to use regex literals.

Expected behavior
Looking at the Matcher, the best matching code would be like:
Given(/^ich starte die App frisch$/) ...

→ I'll create a pull request for this in a seconds.

Feature including scenario outline is not executed if features are filtered by tag and target tag is placed above 'Feature' keyword

Describe the bug
Feature including scenario outline is not executed if features are filtered by tag and target tag is placed above 'Feature' keyword

To Reproduce
Steps to reproduce the behavior:

  1. Create feature file with some content like
@1234567
Feature: Minimal Scenario Outline

Scenario Outline: minimalistic
Given the <what>

Examples:
| what       |
| minimalism |
  1. Specify tagged feature
public func shouldRunWith(tags: [String]) -> Bool {
      return tags.contains("1234567")
}
  1. Execute tests

Expected behavior
Minimal Scenario Outline is executed (or step definitions are generated in case of missing step definitions)

Actual behavior
No tests found to execute, step definitions are not generated

Additional context

  • XCode Version: XCode 10.2.1
  • CucumberSwift Version: 2.2.3

Please note, that feature is executed if tag is moved above Scenario Outline keyword

Feature: Minimal Scenario Outline

@1234567
Scenario Outline: minimalistic
Given the <what>

Examples:
| what       |
| minimalism |

After methods not called on failure

AfterStep, AfterScenario, and AfterFeature are not called if the Step, Scenario, or Feature fails. This means cleanup that has to happen even if the test fails will not be executed.

No error when multiple functions have the same expression

Describe the bug
No error when multiple functions have the same expression

To Reproduce
Steps to reproduce the behavior:

  1. Create 2 functions with the same expression
    For example:
    override func setup() {
    When("^the member goes to the messages screen$") {
    // do something
    }
    When("^the member goes to the messages screen$") {
    // do something else
    }
    }

Expected behavior
Error for multiple functions having the same expression

Additional context

  • CucumberSwift Version: 3.3.6

Error Could't load project at: ....

Tests throw always Couldn't load project at: ... error

/Users/user/Library/Developer/Xcode/DerivedData/CucumberSwiftSample-gkijucsclvpfxlaonviycrdrenry/SourcePackages/checkouts/CucumberSwift/Sources/CucumberSwift/Gherkin/XCode-Specific Couldn't load project at: /Users/user/Library/Developer/Xcode/DerivedData/CucumberSwiftSample-gkijucsclvpfxlaonviycrdrenry/SourcePackages/checkouts/CucumberSwift/Sources/CucumberSwift/Gherkin/XCode-Specific

To Reproduce
Steps to reproduce the behavior:

  1. Download the exmaple App
  2. change the package version in the Swift Package manager from 3.1.1 to 3.0.0
  3. run the tests
  4. See error

Expected behavior
No error and the tests should run like under version 3.1.1

Additional context
Add any other context about the problem here.

  • XCode Version: 14.1
  • CucumberSwift Version: 3.3.22

Should parallel execution of tests work?

TLDR: Should parallel execution of tests work (by choosing Xcode parallelization)?

I can provide a minimum working example but I guess it's not needed as this does not seem to be a setup-dependent issue. But here is how to reproduce, trying to achieve test execution:

  • In Xcode, choose „Edit Scheme…“ (be sure to select the right one)
  • select the „Test“ tab
  • Select „Options“
  • select „Execute in parallel (if possible)“
  • Try to run tests as usual

My observation:

  • Xcode will start a new emulator „Cone 1 […]“.
  • The app is built
  • Tests won't run
  • In the logs, you'll find this and only this:
    • DATE CucumberSwiftParallelUITests-Runner[7950:157096] Running tests...
  • XCode will show the following execution feedback:
    • PROJECT | Build Succeeded | Today at TIME

My expectation:
If you define two .feature Files and run tests in parallel as stated above, the two .feature files should be executed in parallel

Filtering by Line Number

Hi,

I am using CucumberSwift for UI automation. And of the weakness of UI automation is the tests stability. Sometimes it happens to have flickering test which I might want to re-run. It's not a problem to rerun single scenario but it becomes troublesome when there is a scenario outline with several examples and only one example is failed.

The desired solution exists in Cucumber-JVM but not in CucumberSwift. Cucumber-JVM existing solution is described in http://grasshopper.tech/557/ ("Filtering by Line Number" chapter). In terms of technical implementation, it seems like it is needed to add uri property to scenario entity and modify logic of scenario creation from scenario outline. And it sounds like quite an effort, to be honest.

Please let me know if I can provide with more details about this feature request. Thanks.

Cucumber export JSON have missing Property's

Hi @Tyler-Keith-Thompson

I try to import the test report inside my Jira, but it was never possible.

There a 4 option to import:

  1. JVM
  2. JS
  3. Ruby
  4. other

All of them aren't working for me. Then I Crete JSON test Projekt and find out in your report some property's are missing.

Uncaught Error Error: Could not validate Cucumber JSON.
Errors from #/definitions/BehaveJson validation: [
  {
    "instancePath": "/0",
    "schemaPath": "#/required",
    "keyword": "required",
    "params": {
      "missingProperty": "status"
    },
    "message": "must have required property 'status'"
  }
]
Errors from #/definitions/CucumberJsJson validation: [
  {
    "instancePath": "/0",
    "schemaPath": "#/required",
    "keyword": "required",
    "params": {
      "missingProperty": "line"
    },
    "message": "must have required property 'line'"
  }
]
Errors from #/definitions/CucumberJvmJson validation: [
  {
    "instancePath": "/0",
    "schemaPath": "#/required",
    "keyword": "required",
    "params": {
      "missingProperty": "line"
    },
    "message": "must have required property 'line'"
  }
]
Errors from #/definitions/CucumberRubyJson validation: [
  {
    "instancePath": "/0",
    "schemaPath": "#/required",
    "keyword": "required",
    "params": {
      "missingProperty": "line"
    },
    "message": "must have required property 'line'"
  }
]
Error: Could not validate Cucumber JSON.
Errors from #/definitions/BehaveJson validation: [
  {
    "instancePath": "/0",
    "schemaPath": "#/required",
    "keyword": "required",
    "params": {
      "missingProperty": "status"
    },
    "message": "must have required property 'status'"
  }
]
Errors from #/definitions/CucumberJsJson validation: [
  {
    "instancePath": "/0/elements/0/steps/0",
    "schemaPath": "#/required",
    "keyword": "required",
    "params": {
      "missingProperty": "arguments"
    },
    "message": "must have required property 'arguments'"
  },
  {
    "instancePath": "/0/elements/0/steps/0",
    "schemaPath": "#/required",
    "keyword": "required",
    "params": {
      "missingProperty": "hidden"
    },
    "message": "must have required property 'hidden'"
  },
  {
    "instancePath": "/0/elements/0/steps/0",
    "schemaPath": "#/anyOf",
    "keyword": "anyOf",
    "params": {},
    "message": "must match a schema in anyOf"
  }
]
Errors from #/definitions/CucumberJvmJson validation: [
  {
    "instancePath": "/1",
    "schemaPath": "#/required",
    "keyword": "required",
    "params": {
      "missingProperty": "line"
    },
    "message": "must have required property 'line'"
  }
]
Errors from #/definitions/CucumberRubyJson validation: [
  {
    "instancePath": "/1",
    "schemaPath": "#/required",
    "keyword": "required",
    "params": {
      "missingProperty": "line"
    },
    "message": "must have required property 'line'"
  }
]

I use for the check this tool: https://github.com/cucumber/cucumber-json-converter

Perhaps its possible to fix this

"and" & "but" keywords should not be handled as dedicated entities

Background:
The gherkin keywords „And“ and „But“ are just like a placeholder to connect multiple main keywords (eg „Given“, „When“ and „Then“) together in a human-readable sentence.
For example, if you want to combine three given steps like this:


Given step1
Given step2
Given step3

To make it more readable, it makes more sense to combine the phrases with "And", like:

Given step1
And step2
And step3

Thus, in my opinion, „And“ is not a dedicated keyword, it's more like a replacement for a main keyword. I think this matches with the cucumber documentation and concept. See more here https://cucumber.io/docs/gherkin/reference/#and-but

Issue:
In CucumberSwift, the „And“ keyword is handled as an own entity, it's not used as a replacement for the keyword used in one of the previous steps. For example, given this code
:

And(„test“) {_,_ in}

This gherkin syntax would be totally fine and the „test“ step would be executed in every 'and' phrase:


Given step1
And test
When step2
And test
Then step3
And test

Another problem occurs when defining a 'Given' step and you want it to be used for 'And' steps:

Given(„test123“) {_,_ in}
Given step1
And test123

This does not work, but in my opinion, it should. Sure, you can find workarounds like

Let step = „test123“

Func testCode() {}

Given(step) {_,_ in testCode()}
And(step) {_,_ in testCode()}

But then you would still face the first problem and your 'Given' step could be used for other keywords which might not be intented.

Suggestion:

When you parse the feature file and combine it with the matching step implementation (attachClosureToSteps) you need to check for the „and“ & „but“ keywords, and in that case search for the most-recently parsed main keyword, evaluating a match based on that. Something like that:

private func filterSteps(
        keyword: Step.Keyword?,
        regex: String
    ) -> [Step] {
        var filterIndex = -1
        
        let steps = features
            .flatMap { $0.scenarios.flatMap { $0.steps } }
        
        return steps.filter { step -> Bool in
            filterIndex += 1
            if  let k = keyword {
                if step.keyword.contains(k) {
                    return !step.match.matches(for: regex).isEmpty
                }
                
                if step.keyword.contains(.and) || step.keyword.contains(.but) {
                    let stepsBefore = steps[0..<filterIndex].reversed()
                    guard let nextMainStep = stepsBefore.first(where: { [Step.Keyword.given, .when, .then].contains($0.keyword) }), nextMainStep.keyword.contains(k) else {
                        return false
                    }
                    
                    return !step.match.matches(for: regex).isEmpty
                }
                return false
            } else if keyword == nil { // Is this still needed? I dont see any caller with a potential "nil" as keyword value (attachClosureToSteps)
                return !step.match.matches(for: regex).isEmpty
            }
            return false
        }
    }



func attachClosureToSteps(keyword: Step.Keyword? = nil, regex: String, callback:@escaping (([String], Step) -> Void)) {
        filterSteps(
            keyword: keyword,
            regex: regex
        ).forEach { step in
            step.result = .undefined
            step.execute = callback
            step.regex = regex
        }
    }

This specific solution would have at least one problem. Scenario bounds will be ignored. So if you start a scenario with an 'And', it will still be valid:

scenario: 1:

Given step1

Scenario 2:
And step1

You could probably see this as a gherkin syntax fault and not make it a responsibility of CucumberSwift to handle these edge cases. But im sure there's a fix for this somewhere around...

What do you think about the problem / solution? If you think it's a solution worth to invest in I might be able to find some time in the future to create a PR.

Non-Latin characters in test names

Describe the bug
Non-Latin characters in the test navigator come out mangled.

To Reproduce

  1. Write a feature file with non-Latin characters such as German umlaute in the text
  2. Implement step definitions
  3. Run tests
  4. Look at list of tests in Xcode's test navigator

Expected behavior
Expected non-Latin characters to be preserved in the output. One of my steps is:
--8<--
Dann wird eine Begrüßung auf angezeigt
--8<--
(Then a welcome message is shown in )

Screenshots
The generated name is "ThenWirdEineBegrUngAufEnglischAngezeigt". The two German characters 'ü' and 'ß' produce a single 'U'.

Additional context
Add any other context about the problem here.

  • XCode Version: 13.3.1
  • CucumberSwift Version: 3.3.6

DataTable.rows is inaccessible due to internal protection level

Describe the bug

When trying to access rows property in DataTable, like step.dataTable!.rows, the compiler complains that this property is not accessible due to internal protection level.

When looking at DataTable implementation, the rows property is not actually public, maybe this is the problem?

According to documentation related to Test Data this should be possible.

To Reproduce
Steps to reproduce the behavior:

  1. Implement basic When("^Step$") { maches, step in ... } step definition
  2. Try to access step.dataTable!.rows property
  3. See error like the one attached

Expected behavior
The rows property on DataTable instance should be accessible.

Screenshots
Screenshot 2021-04-22 at 13 20 01

Additional context
Add any other context about the problem here.

  • XCode Version: 12.4
  • CucumberSwift Version: 3.1.2

Want: Ability to specify path to "features" folder

Possibly use the info.plist for this since we have to edit it anyways. It needs a relative path to the features folder in case somebody decided not to call it "Features", or if they have a personal bias against capitalization

Missing required module 'CucumberSwift_ObjC' in new swift UITest target

Describe the bug
Compiler fails to build test target with the error "Missing required module 'CucumberSwift_ObjC'"

To Reproduce
Steps to reproduce the behavior:

  1. I created a new UITest target.
  2. I added the CucumberSwift package dependency to the project.
  3. I included feature files in a reference folder "Features".
  4. I created a swift file with the code in the screenshot.
  5. I run the test target to start a build.
  6. The build fails with the error.

Expected behavior
The build should succeed without error.

Screenshots
Screenshot 2022-05-18 at 14 58 16
Screenshot 2022-05-18 at 16 13 33

Additional context
Add any other context about the problem here.

  • XCode Version: [e.g. XCode 13.3]
  • CucumberSwift Version: [e.g. 3.3.11]
  • Example Feature File (if applicable) N/A

Call another step from current

Hi. I'm looking for replacement of Cucumberish and I think CucumberSwift is good alternative. Unfortunately there is a problem for which I haven't found a good workaround.
Cucumberish provides function SStep(definition) which allows to call a step during execution of the current step:

class MySteps: StepImplementation {
  public func setupSteps() {
        Given("Some basic setup") { _, _ in
        }

        Given("Some additional setup") { _, _ in
           // Blah blah
           // Calling basic setup.
           SStep("Some basic setup")
        }
    }
}

Why is it important? I use Cucumber to automate tests for Java and Swift and logic is very similar in both languages. In Java it's necessary to declare function with name for each step and when it's necessary this function can be called directly... But in CucumberSwift (and Cucumberish) almost always lambdas are used.

Yes, it's possible to extract logic to function and call this function:

class MySteps: StepImplementation {
  public func setupSteps() {
        Given("Some basic setup") { _, _ in
           self.someBasicSetup()
        }

        Given("Some additional setup") { _, _ in
           // Blah blah
           // Calling basic setup.
           self.someBasicSetup()
        }
    }

   fun someBasicSetup() {
   }
}

but readability and test report really suffer.

So it would be nice to either provide a way to call step from step like Cucumberish's SStep(...) does. Looking at implementation in Cucumber.swift it doesn't look too complicated.

Ability to define Execution Order of Hooks

Hello again,

Another nice-to-have feature (that is available on the Java side of things) would be to be able to define the execution order of the hooks.

For example:

BeforeScenario(Int.Max_Value) { scenario in
....
}
        
BeforeScenario(1) { scenario in
.....
}

In the above case, the contents of the first hook would be run after the contents of the second. This is more of a nice-to-have, especially at a frame-work use case.

If you're open to PRs, this is something I could probably easily tackle :)

Crashes when the Test are Run

Running the following test crashes all the time:

//
//  WeatherCucumberSwiftTests.swift
//  WeatherCucumberSwiftTests
//
//  Created by Mohammad Azam on 4/13/20.
//  Copyright © 2020 Mohammad Azam. All rights reserved.
//

import XCTest
import UIUTest
import CucumberSwift

@testable import WeatherCucumberSwift

class Findme {}

extension Cucumber: StepImplementation {
   
    public var bundle: Bundle {
        return Bundle(for: Findme.self)
    }
    
    
    public func setupSteps() {
        
        Given("I have a very cool app") { (matches, _) in
            print(matches)
        }
        
        When("I type my name in the name text field") { (matches, _) in
                print(matches)

        }
        
        Then("I should see my name displayed in a label ") { (matches, _) in
                print(matches)

        }
        
    }
    
}

Screen Shot 2020-04-13 at 3 27 52 PM

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.