serenity-bdd / serenity-core Goto Github PK
View Code? Open in Web Editor NEWSerenity BDD is a test automation library designed to make writing automated acceptance tests easier, and more fun.
Home Page: http://serenity-bdd.info
License: Other
Serenity BDD is a test automation library designed to make writing automated acceptance tests easier, and more fun.
Home Page: http://serenity-bdd.info
License: Other
We've found a problem when a Spring Framework based unit test fails to wire (@autowire not matched by Spring bean). The test class is a @RunWith(SerenityRunner.class). We're running integration tests under gradle.
The wiring failure is only visible on the debug level output of gradle. It doesn't report it as a test failure. When building under Jenkins the build is not stopped by the failure, and the test reports report as clear.
Looking at the stack trace, and the SerenityRunner source the reporting context for the class seems to be closed in the finally clause of the run method, but the exception is just printStackTrace'd and rethrown. Looks like the finally will close the reporting context before anything useful is done with the exception.
Seems to be a recall to present issue
Serenity 1.0.47
Implicitly timeout in WebDriver is not reset to the previous value correctly. For example after isCurrentlyVisible method was called it creates two similar entries in TimeoutStack instance with value "0" that is used in net.thucydides.core.webdriver.WebDriverFactory.resetTimeouts() method so the implicitly timeout is not reset to the previous value and stays equal to "0" causing unexpected failures. It seems that operation with TimeoutStack while finding elements has some bugs that cause creating of duplicates of timeout values. Please investigate what can be done to fix the issue. I suggest in to be critical for those who move to serenity and actively use implicitly waits.
The goal of this enhancement is to provide full Appium integration for Serenity, including support for Appium-specific FindBy annotations in the Serenity page objects and the ability to use WebElementFacades for Appium-based web elements.
Appium server needs app property to be defined relatively to appium server path or to be absolute path to app/apk file on the server wgere appium was started.
In case appium is started on another server then Serenity BDD serenity appium.app property check will fale cause it checks file existens locally on the server where Serenity tests were executed.
Class: net.thucydides.core.webdriver.appium.AppiumConfiguration
Methods: ensureAppPathDefinedIn and validPath (lines 75-87).
It seems that the easiest way to fix it is to reject from checking if path valid on serenity level cause appium has its own check of app file path (check error looks like: "Bad app: /Applications/Appium.app/Contents/Resources/node_modules/appium/D:\concierge\ConciergeDemoApp.app. App paths need to be absolute, or relative to the appium server install dir, or a URL to compressed file, or a special app name.")
-Dsysinfo.test.value=112233 works as expected, but if I put -Dsysinfo.test.value="11 2233", the key is in the report, but the value is blank.
Hi, I've got the following step which works fine for all tests except this one:
@Step("When I invoke the factory manufacturing for {0}")
public <T> T whenIInvokeTheFactoryForClass(Class<T> className, PodamFactory podamFactory) throws Exception {
return podamFactory.manufacturePojo(className);
}
And the following JUnit test:
@Test(expected=PodamMockeryException.class)
@Title("Podam should throw an exception if a POJO contains invalid getters or setters")
public void podamShouldThrowAnExceptionIfAPojoContainsInvalidGettersOrSetters() throws Exception {
PodamFactory podamFactory = podamFactorySteps.givenAStandardPodamFactory();
BadlyTypedPojo pojo = podamFactory.manufacturePojo(BadlyTypedPojo.class);
Assert.assertNotNull("Manufacturing failed", pojo);
}
The above passes.
However, if I use Serenity steps, like so:
@Test(expected=PodamMockeryException.class)
@Title("Podam should throw an exception if a POJO contains invalid getters or setters")
public void podamShouldThrowAnExceptionIfAPojoContainsInvalidGettersOrSetters() throws Exception {
PodamFactory podamFactory = podamFactorySteps.givenAStandardPodamFactory();
BadlyTypedPojo pojo = podamInvocationSteps.whenIInvokeTheFactoryForClass(BadlyTypedPojo.class, podamFactory);
}
I get the following ClassCastException:
java.lang.Exception: Unexpected exception, expected<uk.co.jemos.podam.exceptions.PodamMockeryException> but was<java.lang.ClassCastException>
at org.junit.internal.runners.statements.ExpectException.evaluate(ExpectException.java:28)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at net.serenitybdd.junit.runners.SerenityRunner.runChild(SerenityRunner.java:435)
at net.serenitybdd.junit.runners.SerenityRunner.runChild(SerenityRunner.java:59)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at net.serenitybdd.junit.runners.SerenityRunner.run(SerenityRunner.java:248)
at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:78)
at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:212)
at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:68)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)
Caused by: java.lang.ClassCastException: uk.co.jemos.podam.test.unit.steps.PodamInvocationSteps$$EnhancerByCGLIB$$625e929c cannot be cast to uk.co.jemos.podam.test.dto.BadlyTypedPojo
at uk.co.jemos.podam.test.unit.features.edgeCases.AbnormalPojosTest.podamShouldThrowAnExceptionIfAPojoContainsInvalidGettersOrSetters(AbnormalPojosTest.java:136)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at net.serenitybdd.junit.runners.SerenityStatement.evaluate(SerenityStatement.java:24)
at org.junit.internal.runners.statements.ExpectException.evaluate(ExpectException.java:19)
... 20 more
It's best for all users if each component's release enforces the limits of the deps it exposes to prevent these problems.
For example, I just upgraded to serenity-core, serenity-junit, serenity-report-resources, serenity-spring 1.0.31, and serenity-cucumber 1.0.8, and each time there is a lot of dependency cleanup we have to do to keep a tight build in our apps. This means delay on using the new versions as I fix it or push it to a future time.
This is Maven Enforcer Plugin output for this upgrade:
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (enforcer) @ my-module-name ---
[WARNING]
Dependency convergence error for cglib:cglib:2.2.2 paths to dependency are:
+-my-group-name:my-module-name:1.0.0-SNAPSHOT
+-net.serenity-bdd:core:1.0.31
+-cglib:cglib:2.2.2
and
+-my-group-name:my-module-name:1.0.0-SNAPSHOT
+-net.serenity-bdd:core:1.0.31
+-io.appium:java-client:2.1.0
+-cglib:cglib:3.1
[WARNING]
Dependency convergence error for org.seleniumhq.selenium:selenium-java:2.44.0 paths to dependency are:
+-my-group-name:my-module-name:1.0.0-SNAPSHOT
+-net.serenity-bdd:core:1.0.31
+-org.seleniumhq.selenium:selenium-server:2.44.0
+-org.seleniumhq.selenium:selenium-java:2.44.0
and
+-my-group-name:my-module-name:1.0.0-SNAPSHOT
+-net.serenity-bdd:core:1.0.31
+-io.appium:java-client:2.1.0
+-org.seleniumhq.selenium:selenium-java:2.43.1
[WARNING]
Dependency convergence error for com.google.guava:guava:18.0 paths to dependency are:
+-my-group-name:my-module-name:1.0.0-SNAPSHOT
+-net.serenity-bdd:core:1.0.31
+-com.google.guava:guava:18.0
and
+-my-group-name:my-module-name:1.0.0-SNAPSHOT
+-net.serenity-bdd:core:1.0.31
+-org.seleniumhq.selenium:selenium-server:2.44.0
+-org.seleniumhq.selenium:selenium-java:2.44.0
+-org.seleniumhq.selenium:selenium-chrome-driver:2.44.0
+-org.seleniumhq.selenium:selenium-remote-driver:2.44.0
+-org.seleniumhq.selenium:selenium-api:2.44.0
+-com.google.guava:guava:18.0
and
+-my-group-name:my-module-name:1.0.0-SNAPSHOT
+-net.serenity-bdd:core:1.0.31
+-org.seleniumhq.selenium:selenium-server:2.44.0
+-org.seleniumhq.selenium:selenium-java:2.44.0
+-org.seleniumhq.selenium:selenium-chrome-driver:2.44.0
+-org.seleniumhq.selenium:selenium-remote-driver:2.44.0
+-com.google.guava:guava:18.0
and
+-my-group-name:my-module-name:1.0.0-SNAPSHOT
+-net.serenity-bdd:core:1.0.31
+-io.appium:java-client:2.1.0
+-com.google.guava:guava:17.0
and
+-my-group-name:my-module-name:1.0.0-SNAPSHOT
+-net.serenity-bdd:serenity-junit:1.0.31
+-com.google.guava:guava:18.0
and
+-my-group-name:my-module-name:1.0.0-SNAPSHOT
+-net.serenity-bdd:serenity-report-resources:1.0.31
+-com.google.guava:guava:18.0
and
+-my-group-name:my-module-name:1.0.0-SNAPSHOT
+-net.serenity-bdd:serenity-spring:1.0.31
+-com.google.guava:guava:18.0
[WARNING]
Dependency convergence error for commons-codec:commons-codec:1.9 paths to dependency are:
+-my-group-name:my-module-name:1.0.0-SNAPSHOT
+-net.serenity-bdd:core:1.0.31
+-commons-codec:commons-codec:1.9
and
+-my-group-name:my-module-name:1.0.0-SNAPSHOT
+-net.serenity-bdd:core:1.0.31
+-io.appium:java-client:2.1.0
+-org.apache.httpcomponents:httpclient:4.3.6
+-commons-codec:commons-codec:1.6
and
+-my-group-name:my-module-name:1.0.0-SNAPSHOT
+-net.serenity-bdd:serenity-junit:1.0.31
+-commons-codec:commons-codec:1.9
and
+-my-group-name:my-module-name:1.0.0-SNAPSHOT
+-net.serenity-bdd:serenity-report-resources:1.0.31
+-commons-codec:commons-codec:1.9
and
+-my-group-name:my-module-name:1.0.0-SNAPSHOT
+-net.serenity-bdd:serenity-spring:1.0.31
+-commons-codec:commons-codec:1.9
[WARNING] Rule 4: org.apache.maven.plugins.enforcer.RequireUpperBoundDeps failed with message:
Failed while enforcing RequireUpperBoundDeps. The error(s) are [
Require upper bound dependencies error for com.google.code.gson:gson:2.3 paths to dependency are:
+-my-group-name:my-module-name:1.0.0-SNAPSHOT
+-net.serenity-bdd:core:1.0.31
+-com.google.code.gson:gson:2.3 (managed) <-- com.google.code.gson:gson:2.3.1
and
+-my-group-name:my-module-name:1.0.0-SNAPSHOT
+-net.serenity-bdd:core:1.0.31
+-io.appium:java-client:2.1.0
+-com.google.code.gson:gson:2.3 (managed) <-- com.google.code.gson:gson:2.2.4
,
Require upper bound dependencies error for cglib:cglib:2.2.2 paths to dependency are:
+-my-group-name:my-module-name:1.0.0-SNAPSHOT
+-net.serenity-bdd:core:1.0.31
+-cglib:cglib:2.2.2
and
+-my-group-name:my-module-name:1.0.0-SNAPSHOT
+-net.serenity-bdd:core:1.0.31
+-io.appium:java-client:2.1.0
+-cglib:cglib:3.1
,
Require upper bound dependencies error for org.codehaus.groovy:groovy:2.3.8 paths to dependency are:
+-my-group-name:my-module-name:1.0.0-SNAPSHOT
+-net.serenity-bdd:core:1.0.31
+-org.codehaus.groovy:groovy:2.3.8 (managed) <-- org.codehaus.groovy:groovy:2.4.0
]
[WARNING] Rule 5: org.apache.maven.plugins.enforcer.DependencyConvergence failed with message:
Failed while enforcing releasability the error(s) are [
Dependency convergence error for cglib:cglib:2.2.2 paths to dependency are:
+-my-group-name:my-module-name:1.0.0-SNAPSHOT
+-net.serenity-bdd:core:1.0.31
+-cglib:cglib:2.2.2
and
+-my-group-name:my-module-name:1.0.0-SNAPSHOT
+-net.serenity-bdd:core:1.0.31
+-io.appium:java-client:2.1.0
+-cglib:cglib:3.1
,
Dependency convergence error for org.seleniumhq.selenium:selenium-java:2.44.0 paths to dependency are:
+-my-group-name:my-module-name:1.0.0-SNAPSHOT
+-net.serenity-bdd:core:1.0.31
+-org.seleniumhq.selenium:selenium-server:2.44.0
+-org.seleniumhq.selenium:selenium-java:2.44.0
and
+-my-group-name:my-module-name:1.0.0-SNAPSHOT
+-net.serenity-bdd:core:1.0.31
+-io.appium:java-client:2.1.0
+-org.seleniumhq.selenium:selenium-java:2.43.1
,
Dependency convergence error for com.google.guava:guava:18.0 paths to dependency are:
+-my-group-name:my-module-name:1.0.0-SNAPSHOT
+-net.serenity-bdd:core:1.0.31
+-com.google.guava:guava:18.0
and
+-my-group-name:my-module-name:1.0.0-SNAPSHOT
+-net.serenity-bdd:core:1.0.31
+-org.seleniumhq.selenium:selenium-server:2.44.0
+-org.seleniumhq.selenium:selenium-java:2.44.0
+-org.seleniumhq.selenium:selenium-chrome-driver:2.44.0
+-org.seleniumhq.selenium:selenium-remote-driver:2.44.0
+-org.seleniumhq.selenium:selenium-api:2.44.0
+-com.google.guava:guava:18.0
and
+-my-group-name:my-module-name:1.0.0-SNAPSHOT
+-net.serenity-bdd:core:1.0.31
+-org.seleniumhq.selenium:selenium-server:2.44.0
+-org.seleniumhq.selenium:selenium-java:2.44.0
+-org.seleniumhq.selenium:selenium-chrome-driver:2.44.0
+-org.seleniumhq.selenium:selenium-remote-driver:2.44.0
+-com.google.guava:guava:18.0
and
+-my-group-name:my-module-name:1.0.0-SNAPSHOT
+-net.serenity-bdd:core:1.0.31
+-io.appium:java-client:2.1.0
+-com.google.guava:guava:17.0
and
+-my-group-name:my-module-name:1.0.0-SNAPSHOT
+-net.serenity-bdd:serenity-junit:1.0.31
+-com.google.guava:guava:18.0
and
+-my-group-name:my-module-name:1.0.0-SNAPSHOT
+-net.serenity-bdd:serenity-report-resources:1.0.31
+-com.google.guava:guava:18.0
and
+-my-group-name:my-module-name:1.0.0-SNAPSHOT
+-net.serenity-bdd:serenity-spring:1.0.31
+-com.google.guava:guava:18.0
,
Dependency convergence error for commons-codec:commons-codec:1.9 paths to dependency are:
+-my-group-name:my-module-name:1.0.0-SNAPSHOT
+-net.serenity-bdd:core:1.0.31
+-commons-codec:commons-codec:1.9
and
+-my-group-name:my-module-name:1.0.0-SNAPSHOT
+-net.serenity-bdd:core:1.0.31
+-io.appium:java-client:2.1.0
+-org.apache.httpcomponents:httpclient:4.3.6
+-commons-codec:commons-codec:1.6
and
+-my-group-name:my-module-name:1.0.0-SNAPSHOT
+-net.serenity-bdd:serenity-junit:1.0.31
+-commons-codec:commons-codec:1.9
and
+-my-group-name:my-module-name:1.0.0-SNAPSHOT
+-net.serenity-bdd:serenity-report-resources:1.0.31
+-commons-codec:commons-codec:1.9
and
+-my-group-name:my-module-name:1.0.0-SNAPSHOT
+-net.serenity-bdd:serenity-spring:1.0.31
+-commons-codec:commons-codec:1.9
]
the maven command has the retry in case of failure/error
-Dsurefire.rerunFailingTestsCount=1 -Dmax.retries=1 -DJUNIT_RETRY_TESTS=true -Dwebdriver.timeouts.implicitlywait=31000 -Dwebdriver.wait.for.timeout=31000 -Dserenity.timeout=60
also running the tests in parallel with batch_number 1 2 and batch_count = 2 in jenkins as multi configuration project and aggregating the results by copying the 2 batch_number target folders into one target folder and then executing mvn serenity:aggregate
using serenity 1.0.45
If a test fails or has errors, it will repeat it (once as specified above) and if it's passed, then in the serenity test report summary, it will mark it as failed/error: 200 passed , 0 pending, 0 failed, 1 with errors, although second run shows pass.
In the report in the tests table, clicking on that test, it will show the two runs (just as there were two rows in a csv file), one with error, one with pass (which is ok). so can you make the summary take the pass result and not show it with error? so "201 passed , 0 pending, 0 failed, 0 with errors".
in thucydides 0.9.275, the report top summary showed correctly for the above scenario. in the test titles table it was marked as error, but in the top summary it stated all pass.
When I use Serenity with Selenium Grid, then WebDriverFactory ignore "platform" and "version" properties.
I investigated this issue and if I use parameter webdriver.remote.driver=iexplorer then WebDriverFactory will create real browser capabilities, because it creates remote capabilities only for Remote driver.
Original issue:
01:45:41 687015 [main] INFO net.thucydides.core.steps.StepInterceptor - STARTING STEP: saveButtonClick
09:40:52 29197772 [main] INFO net.thucydides.core.steps.StepInterceptor - STEP DONE: saveButtonClick
as you can see the tests hanged between 01:45 am and 09:40, so about 8 hours. there were no errors on the environment, OS, drivers, browsers, logs etc
So I looked for a way to resolve this, by adding the test timeout from junit 4.12.
added @test(timeout = x) and thucydides is marking the passed tests as skipped/pending
Test timeout (junit 4.12)
if test timeout is reached then it works
but in case of test pass, thucydides (0.9.275)/serenity(1.0.45) marks the test as skipped/pending, because it runs in another thread and doesn't reach the expected code.
I did not find other test timeouts options in thucydides. is there a workaround until this is fixed?
While my tests in Chrome run fine, I have sometimes problems with Firefox. Last time it happened between to test starts (I configured my Maven project with profiles to run only specific tests). After the end of the first test suite, the second started with:
org.openqa.selenium.remote.UnreachableBrowserException: Error communicating with the remote browser. It may have died.
Build info: version: '2.44.0', revision: '76d78cf323ce037c5f92db6c1bba601c2ac43ad8', time: '2014-10-23 13:11:40'
System info: host: 'WinGridTest', ip: '192.168.0.250', os.name: 'Windows 8.1', os.arch: 'amd64', os.version: '6.3', java.version: '1.8.0_05'
Driver info: driver.version: RemoteWebDriver
at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:593)
...
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
I'm not sure if this is just Selenium related, but it took exactly 3h until the Exception was thrown. I have a timeout configured for 180s (story.timeout.in.secs) ... could this have something to do with it and somehow been transformed to minutes?
This error has appeared in version 1.0.48 and reproduced in the current 1.0.50
INFO net.thucydides.core.screenshots.SingleThreadScreenshotProcessor - Processing screenshot: /tmp/screenshot1472783073628648310
INFO net.thucydides.core.steps.StepInterceptor - STEP DONE: simpleStep
WARN net.thucydides.core.screenshots.SingleThreadScreenshotProcessor - Failed to copy the screenshot to the destination directory: /tmp/screenshot1472783073628648310 -> target/site/serenity/fb8da3cb95894d7b08ae2402fb4b14a9_NONE.png: Invalid cross-device link
I can't find 1.0.52 release in any maven repo. Can you upload it somewhere please?
Could't find any easy way to switch to iframe in the page object
Copied from https://java.net/jira/browse/THUCYDIDES-264
Haven't debugged, it's just an fyi. There's a replica existing issue logged against chromedriver
Steps:
Concurrency config is parallel='classes', perCoreThreadCount=true, threadCount=2
, useUnlimitedThreads=false
Results :
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[WARNING] File encoding has not been set, using platform encoding Cp1251, i.e. b
uild is platform dependent!
[INFO]
[INFO] --- serenity-maven-plugin:1.0.23:aggregate (serenity-reports) @ webtests2
log4j:WARN No appenders could be found for logger (org.jboss.logging).
log4j:WARN Please initialize the log4j system properly.
[INFO] Reading requirements from net.thucydides.core.requirements.FileSystemRequ
irementsTagProvider@9c3057
[INFO] Reading requirements from net.thucydides.core.requirements.PackageAnnotat
ionBasedTagProvider@1941489
[INFO] Requirements found:[]
If you will run debug with jUnit then see that all works.
Hi
I have a simple basic project set up which uses Selenium, Cucumber, Java in a Page Object Model setup (see github repo https://github.com/markwinspear/SeleniumCucumberJavaBasicTest2). I noticed the flexibility and reporting capabilities of Serenity and am interested in seting up a project to try it out, however, my current project uses JUnit Rule to set up capabilities and interface with Sauce labs to execute and report on tests executed in the cloud.
Could you please advise how this could be done using Serenity please?
Note: I only have relatively basic developer skills (coming from a testing background)
Serenity exposes dependency convergence problems with transitive deps, making it difficult to keep a clean dep tree in client modules.
This gist is an example pom showing the issues.
https://gist.github.com/jeffjensen/729d64c56d5563dc9da2
Just tested the project build with Mozilla Firefox version 36 today (2015-02-25).
Test failure results in an issue creating a new Firefox instance.
org.openqa.selenium.firefox.NotConnectedException: Unable to connect to host 127.0.0.1 on port 7055 after 45000 ms. Firefox console output: , setting schema version preference to 16
...
ERROR n.t.core.webdriver.WebDriverFacade - FAILED TO CREATE NEW WEBDRIVER_DRIVER INSTANCE class org.openqa.selenium.firefox.FirefoxDriver: Could not instantiate class org.openqa.selenium.firefox.FirefoxDriver
net.thucydides.core.webdriver.UnsupportedDriverException: Could not instantiate class org.openqa.selenium.firefox.FirefoxDriver
Workaround: Use Firefox 35.x Portable and set the webdriver.firefox.bin
system property to the firefox.exe
file path.
I have a multi-module maven project. We are creating a specific module for BDD scenarios and test cases using Serenity and I think I found an issue:
Say I have a module called 'documentation' where I write my stories. If I type something like this:
mvn verify
or
mvn -pl documentation verify
Both commands will create a broken report site where some data will not be presented while the header menu will be incomplete (only "overall test results" and "requirements" are shown). But, if I type the following commands it works:
cd documentation
mvn verify
In that case, the data will be correctly presented and also the menu will be built in accordance.
I think the problem is something about the goal serenity:aggregate that, as far as I know, works grouping several data provided by jbehavior and build the reporting site.
When there are multiple page method calls within a step and one of them throws an exception there is little or no visibility as to exactly where where within the Step the failure occurred.
It would aid debugging if logging output (of calls to Page methods) was kept for the duration of the Step so that in event of an Exception being thrown then the output could be included in the report, (logging would be thrown away for Steps that pass).
I have a requirement to continue the scenario execution in case of step failure.
I am looking for a soft asset option.
I am using hamcrest for assetion and when ever a step fails, serenity jumps in the next step.
Could you please provide a solution to this problem?
Hey there,
I've tried the following:
FileSystemRequirementsTagProvider fileSystemRequirementsTagProvider =
new FileSystemRequirementsTagProvider("com/forrest/gump/integrationtest/cucumber/");
fileSystemRequirementsTagProvider.getRequirements();
and getRootDirectoryPaths() return the following path:
file:/usr/bin/apache-tomcat-7.0.53/webapps/ROOT/WEB-INF/lib/cucumber-demo-project-0.6.3.12-SNAPSHOT.jar!/com/forrest/gump/integrationtest/cucumber/
and yes, if new File(rootDirectoryPath).exists => then false, and failed to get the requirements.
Should it be treated as an enhancement of getting requirements from the JAR file? or is this an issue actually? Or there is another way of reading the requirements? Please adivise.
Andy.
With serenity-core 1.0.44 when you use serenity-cucumber with feature files in a different language than English, an error such as the following is reported:
Apr 13, 2015 1:01:37 AM com.google.common.util.concurrent.ExecutionList executeListener
SCHWERWIEGEND: RuntimeException while executing runnable net.thucydides.core.reports.ReportService$2@72674c7a with executor com.google.common.util.concurrent.MoreExecutors$DirectExecutorService@7658d6ee
gherkin.lexer.LexingError: Lexing error on line 2: 'Funktionalitรคt: Ich mรถchte testen, wie Serenity und Cucumber mit deutscher Sprache umgehen.
Szenariogrundriss: Google-Suche
Angenommen Ich habe Google offen.
Wenn Ich nach '<suchbegriff>' suche
Dann Sehe ich u.a. das Ergebnis '<ergebnis>'
Beispiele:
| suchbegriff | ergebnis |
| serenity selenium | The Serenity Reference Manual - Thucydides |
| Hรผpfburg | Hรผpfburg - eBay Kleinanzeigen |
%_FEATURE_END_%'. See http://wiki.github.com/cucumber/gherkin/lexingerror for more information.
at gherkin.lexer.En.scan(En.java:872)
at gherkin.lexer.I18nLexer.scan(I18nLexer.java:36)
at gherkin.parser.Parser.parse(Parser.java:62)
at net.thucydides.core.requirements.model.cucumber.CucumberParser.loadFeatureNarrative(CucumberParser.java:28)
at net.thucydides.core.requirements.FileSystemRequirementsTagProvider.loadFromFeatureFile(FileSystemRequirementsTagProvider.java:494)
at net.thucydides.core.requirements.FileSystemRequirementsTagProvider.readRequirementsFromStoryOrFeatureFile(FileSystemRequirementsTagProvider.java:477)
at net.thucydides.core.requirements.FileSystemRequirementsTagProvider.access$100(FileSystemRequirementsTagProvider.java:44)
at net.thucydides.core.requirements.FileSystemRequirementsTagProvider$2.convert(FileSystemRequirementsTagProvider.java:456)
at net.thucydides.core.requirements.FileSystemRequirementsTagProvider$2.convert(FileSystemRequirementsTagProvider.java:453)
at ch.lambdaj.function.convert.ConverterIterator.next(ConverterIterator.java:37)
at ch.lambdaj.Lambda.convert(Lambda.java:986)
at net.thucydides.core.requirements.FileSystemRequirementsTagProvider.loadStoriesFrom(FileSystemRequirementsTagProvider.java:440)
at net.thucydides.core.requirements.FileSystemRequirementsTagProvider.getRequirements(FileSystemRequirementsTagProvider.java:129)
at net.thucydides.core.requirements.RequirementsServiceImplementation.getRequirements(RequirementsServiceImplementation.java:55)
at net.thucydides.core.requirements.RequirementsServiceImplementation.getAllRequirements(RequirementsServiceImplementation.java:286)
at net.thucydides.core.requirements.RequirementsServiceImplementation.getRequirementTypes(RequirementsServiceImplementation.java:228)
at net.thucydides.core.reports.html.HtmlAcceptanceTestReporter.addTestOutcomeToContext(HtmlAcceptanceTestReporter.java:123)
at net.thucydides.core.reports.html.HtmlAcceptanceTestReporter.generateReportFor(HtmlAcceptanceTestReporter.java:88)
at net.thucydides.core.reports.ReportService.generateReportFor(ReportService.java:192)
at net.thucydides.core.reports.ReportService.generateQueuedReport(ReportService.java:141)
at net.thucydides.core.reports.ReportService.access$000(ReportService.java:24)
at net.thucydides.core.reports.ReportService$2.run(ReportService.java:115)
at com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:299)
at com.google.common.util.concurrent.ExecutionList.executeListener(ExecutionList.java:156)
at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:145)
at com.google.common.util.concurrent.ListenableFutureTask.done(ListenableFutureTask.java:91)
at java.util.concurrent.FutureTask.finishCompletion(FutureTask.java:384)
at java.util.concurrent.FutureTask.set(FutureTask.java:233)
at java.util.concurrent.FutureTask.run(FutureTask.java:274)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
The feature file for which this error was produced looks like this:
# language: de
Funktionalitรคt: Ich mรถchte testen, wie Serenity und Cucumber mit deutscher Sprache umgehen.
Szenariogrundriss: Google-Suche
Angenommen Ich habe Google offen.
Wenn Ich nach '<suchbegriff>' suche
Dann Sehe ich u.a. das Ergebnis '<ergebnis>'
Beispiele:
| suchbegriff | ergebnis |
| serenity selenium | The Serenity Reference Manual - Thucydides |
| Hรผpfburg | Hรผpfburg - eBay Kleinanzeigen |
The reason for this error seems to be that in line 40 of the CucumberParser, all comments are filtered out of the input which then is supposed to be parsed. Since the language and the encoding have to be set with comments such as # language: de
and # encoding: utf8
if they're supposed to differ from the default, this is a problem.
Since (as far as I can see) cucumber itself does not filter out comments like that and manages just fine, I'm not really sure whether there's any good reason to do this.
Various modules have differing versions and therefore conflicts. Yes, gradle will resolve them, but it presents convergence problems and unknown potential problems.
Perhaps move all dependency versions to top build config and inherit them as possible or use properties.
While in 1.0.28 the report was generated fine, there occured errors in the following versions up to 1.0.31. Several warnings and exception of the type are shown:
[WARNING] this file was not a valid JSON Serenity test report: 3e1acda910a5b01fa30ba70eca08dd97c60cf196a938c046d09530ffa77d22b8.json
java.lang.ClassNotFoundException: de.wemove.testing.junit.webmapClient.layer.External
I'm using jUnit tests. I saw that in 1.0.28 when generating a report the following output appeared in the beginning:
Feb 16, 2015 5:43:23 PM org.hibernate.validator.internal.util.Version
INFO: HV000001: Hibernate Validator 5.1.1.Final
which does not in the other versions.
When switching back to the old version and generating the report (from the 1.0.31 test run) it also seems, that there are some tests missing and warnings of the following type appeared:
[WARNING] this file was not a valid JSON Thucydides test report: 3fe441dc52bc84292010b8886cf1fd291cbc74aef13683de6d9130c715940425.json
Invalid UTF-8 middle byte 0x6e
at [Source: java.io.BufferedInputStream@5d51e129; line: 1, column: 1470](through reference chain: net.thucydides.core.model.TestOutcome["testSteps"]->java.util.ArrayList[4]->net.thucydides.core.model.TestStep["description"])
Hi John,
When I'm running parallel methods in single class file, the report which generates is completely empty.
can you please look into this issue
Thanks,
Vikram
I am using Serenity with jBehave and filter test scope via tags.
mvn clean integration-test -Dmetafilter=-manual serenity:aggregate
It works perfectly except one thing, in report whole scenario is marked as ignored and there are no steps related for it.
Is there a way to show all steps names for ignored scenario?
Has anyone been able to redirect output for either Thucydides or Serenity to a custom directory?
The default report dir is <$project root>/target/site/thucydides
Or
<$project root>/target/site/serenity
Depending on the version of thucydides/serenity you are using.
When I choose to customize the output directory using:
serenity.source=target/site/thucydides/
serenity.outputDirectory=target/site/thucydides/
I've also tried:
thucydides.outputDirectory=target/site/thucydides
The reporting pieces are correctly generated in target/site/thucydides/
But when the aggregate command is executed, the final index.html file is generated in the target/site/serenity directory and it is non-functional as the reporting files are split between the thucydides and serenity dirs.
My environment:
Serenity 1.0.47
MacOS 10.10
Firefox 38
Thanks,
Rob
Hello,
I am using serenity 1.0.38, JUnit runner and browsers FF/Chrome.
I discovered that @findby with xpath, className, linkText and may be others doesn't work. Element always has value "undefined", what is really weired.
Works well only with name and id.
As library I am using:
import net.serenitybdd.core.annotations.findby.FindBy;
Previously I used thucydides 0.9.275 where everything works as expected.
What is wrong or I am doing wrong?
Thank you.
As mentioned in the group:
https://groups.google.com/forum/#!topic/thucydides-users/b1gSdtkNXkY
I have a patch which fixes this bug i.e. it allows List<T extends WebElementFacade>
page object members and preserves lazy lookup behavior for the constituent elements. I'd like to submit a pull request, let me know if interested.
EDIT: I'll just submit it, since its already done :)
We are using Serenity with JBehave. We are facing issues when we have multiple scenario outlines with examples table in a single .feature file.
we have something like this in our .feature file.
Scenario Outline: title 1
Given
When
Then
Examples:
|data|
Scenario Outline: title 2
Given 2
When 2
Then 2
Examples:
|Data|
In this case after executing โExamplesโ of Scenario 1, it is considering Scenario outline 2 also as an input to scenario 1 example table, instead of treating it as a new scenario.
This is how the output looks like:
Scenario Outline: title 1
Given
When
Then
Examples:
|data|
|Scenario Outline: title 2|
|Given 2|
|When 2|
|Then 2|
|Examples:|
|Data|
Here are the versions of plugins:
jbehave.core.version - 3.9.5;
serenity.version - 1.0.47;
serenity.jbehave.version - 1.0.21
Can someone please help resolve this? We seem to be blocked by this issue.
Note: I have seen posts where people had same issues when using "scenario" with examples table rather than using "scenario outline" with examples. Here I am using "scenario outline" only, but still have the same issue.
After upgrading from 1.0.23 to 1.0.45, I no longer get any screenshots. I'm using the Chrome driver, configured via webdriver.driver
in serenity.properties
.
Stepping through the code, I can see that BaseStepListener.take()
gets hit, but BaseStepListener.browserIsOpen()
returns false, since proxiedWebDriver
is null in WebDriverFacade
so that no screenshot is taken.
I also observed two instances of WebDriverFacade being created, one for an empty/default driver (i.e. firefox) and another one for chrome, which might be the root cause - hope that helps.
I ran the following test:
package aa.aa;
import java.util.Arrays;
import java.util.Collection;
import net.serenitybdd.junit.runners.SerenityParameterizedRunner;
import net.thucydides.core.annotations.Managed;
import net.thucydides.core.annotations.Pending;
import net.thucydides.junit.annotations.TestData;
import org.junit.Assert;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.openqa.selenium.WebDriver;
@RunWith(SerenityParameterizedRunner.class)
public class LulzTest {
@Managed
WebDriver driver;
@TestData
public static Collection<Object[]> testData() {
return Arrays.asList(new Object[][] { { "cats" }, { "dogs" }, { "ferrets" }, { "rabbits" }, { "canaries" } });
}
public String searchTerm;
public LulzTest(String searchTerm) {
this.searchTerm = searchTerm;
}
@Pending
@Test
public void shouldInstantiatedPageObjectsForADataDrivenWebTest() {
}
@Test
public void shouldWhatever() {
Assert.assertTrue(true);
}
}
Then I got the following reports. 5 passed and 0 pending.
I've got a simple check if a certain element is present. For this check I have set the implicit timeout (per this blog post
I noticed that the resetImplicitTimeout does not reset the timeout value.
setImplicitTimeout(1000, TimeUnit.MILLISECONDS);
Boolean loggedIn = navbar.isPresent();
resetImplicitTimeout();
I checked it with some logging of getImplicitWaitTimeout() and I can see that the Implicit timeout is not reset after de resetImplicitTimeout() call.
before set: 2000 MILLISECONDS
after set: 1000 MILLISECONDS
after reset: 1000 MILLISECONDS
Hi John,
Besides of web tests (selenium), we can just create and NON-web tests (back-end, integration unit tests), and still have this wonderful Serenity Reporting capability.
But I noticed in the Serenity Report at the test outcome level that it's displayed the "Firefox" image, however no instance of browser has been opened.
Yes, it seems to be a cosmetic/ trivial issue that the image is displayed; So far now the default driver is set to "firefox", so due to that the image is displayed I believe. This kind of false positive output persuades to the wrong impression.
Would it be possible to hide this image for non-web tests?
I have TCs grouped by several tags.
When I specifying -Dtags="tag1:tag2" parameter in maven build all tests are skipped. :(
Serenity version: 1.0.35
Works in thucydides 0.9.275.
The screenshots in a report are cropped and I cannot see the full height of the images.
This method was working in thucydides but broken in serenity
org.openqa.selenium.ElementNotVisibleException: Timed out after 0 seconds waiting for net.serenitybdd.core.pages.WebElementFacadeImpl$1@6b01af7
Build info: version: '2.45.0', revision: '5017cb8e7ca8e37638dc3091b2440b90a1d8686f', time: '2015-02-27 09:10:26'
System info: host: 'N/A', ip: 'N/A', os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.9.5', java.version: '1.7.0_60'
Driver info: driver.version: unknown
Build info: version: '2.45.0', revision: '5017cb8e7ca8e37638dc3091b2440b90a1d8686f', time: '2015-02-27 09:10:26'
System info: host: 'N/A', ip: 'N/A', os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.9.5', java.version: '1.7.0_60'
Driver info: driver.version: unknown
at net.serenitybdd.core.pages.WebElementFacadeImpl.throwErrorWithCauseIfPresent(WebElementFacadeImpl.java:632)
at net.serenitybdd.core.pages.WebElementFacadeImpl.waitUntilVisible(WebElementFacadeImpl.java:609)
Please look into this
thanks & regards,
Vikram
I've got the following JUnit test which passes fine:
// Creates factory...
// RecursivePojo pojo = factory.manufacturePojo(RecursivePojo.class);
// Assert.assertNotNull("The recursive pojo cannot be null!", pojo);
//
// validateRecursiveHierarchyPojo(pojo);
However, when I use Serenity Steps, I get an overflow:
PodamFactory podamFactory = podamFactorySteps.givenAStandardPodamFactory();
RecursivePojo recursivePojo = podamInvocationSteps.whenIInvokeTheFactoryForClass(RecursivePojo.class, podamFactory);
podamValidationSteps.thePojoShouldNotBeNull(recursivePojo);
recursivePojoValidationSteps.allPojosInTheRecursiveStrategyShouldBeValid(recursivePojo);
The method creating the overflow is as follows:
/**
* Constructs a <code>String</code> with all attributes in name = value
* format.
*
* @return a <code>String</code> representation of this object.
*/
@Override
public String toString() {
final String TAB = " ";
StringBuilder retValue = new StringBuilder();
retValue.append("RecursivePojo ( ").append("intField = ")
.append(intField).append(TAB).append("parent = ")
.append(parent).append(TAB).append(" )");
return retValue.toString();
}
So while a normal JUnit test executes successfully the toString, with Serenity it gives overflow.
20:28:23.854 [main] INFO net.thucydides.core.steps.StepInterceptor - STARTING STEP: PodamFactoryBasicTypesTest.podamShouldFillRecursivePojos - givenAStandardPodamFactory
20:28:24.212 [main] INFO net.thucydides.core.steps.StepInterceptor - STEP DONE: givenAStandardPodamFactory
20:28:24.216 [main] INFO net.thucydides.core.steps.StepInterceptor - STARTING STEP: PodamFactoryBasicTypesTest.podamShouldFillRecursivePojos - whenIInvokeTheFactoryForClass
20:28:24.227 [main] INFO net.thucydides.core.steps.StepInterceptor - STEP DONE: whenIInvokeTheFactoryForClass
java.lang.StackOverflowError
at java.lang.StringBuilder.append(StringBuilder.java:208)
at uk.co.jemos.podam.test.dto.RecursivePojo.toString(RecursivePojo.java:87)
...
...
May 30, 2015 8:28:25 PM freemarker.log._JDK14LoggerFactory$JDK14Logger error
SEVERE: Error executing FreeMarker template
FreeMarker template error:
The following has evaluated to null or missing:
==> cause.message [in template "freemarker/default.ftl" at line 269, column 61]
----
Tip: It's the step after the last dot that caused this error, not those before it.
----
Tip: If the failing expression is known to be legally refer to something that's null or missing, either specify a default value like myOptionalVar!myDefault, or use <#if myOptionalVar??>when-present<#else>when-missing</#if>. (These only cover the last step of the expression; to cover the whole expression, use parenthesis: (myOptionalVar.foo)!myDefault, (myOptionalVar.foo)??
----
----
FTL stack trace ("~" means nesting-related):
- Failed at: ${cause.message} [in template "freemarker/default.ftl" in macro "stacktrace" at line 269, column 59]
- Reached through: @stacktrace cause=testOutcome.nestedT... [in template "freemarker/default.ftl" at line 406, column 37]
----
Java stack trace (for programmers):
----
freemarker.core.InvalidReferenceException: [... Exception message was already printed; see it above ...]
at freemarker.core.InvalidReferenceException.getInstance(InvalidReferenceException.java:108)
at freemarker.core.EvalUtil.coerceModelToString(EvalUtil.java:346)
at freemarker.core.Expression.evalAndCoerceToString(Expression.java:80)
at freemarker.core.DollarVariable.accept(DollarVariable.java:40)
at freemarker.core.Environment.visit(Environment.java:257)
at freemarker.core.MixedContent.accept(MixedContent.java:57)
at freemarker.core.Environment.visit(Environment.java:257)
at freemarker.core.Macro$Context.runMacro(Macro.java:173)
at freemarker.core.Environment.visit(Environment.java:686)
at freemarker.core.UnifiedCall.accept(UnifiedCall.java:80)
at freemarker.core.Environment.visit(Environment.java:257)
at freemarker.core.MixedContent.accept(MixedContent.java:57)
at freemarker.core.Environment.visitByHiddingParent(Environment.java:278)
at freemarker.core.ConditionalBlock.accept(ConditionalBlock.java:48)
at freemarker.core.Environment.visit(Environment.java:257)
at freemarker.core.MixedContent.accept(MixedContent.java:57)
at freemarker.core.Environment.visitByHiddingParent(Environment.java:278)
at freemarker.core.ConditionalBlock.accept(ConditionalBlock.java:48)
at freemarker.core.Environment.visit(Environment.java:257)
at freemarker.core.MixedContent.accept(MixedContent.java:57)
at freemarker.core.Environment.visitByHiddingParent(Environment.java:278)
at freemarker.core.ConditionalBlock.accept(ConditionalBlock.java:48)
at freemarker.core.Environment.visit(Environment.java:257)
at freemarker.core.MixedContent.accept(MixedContent.java:57)
at freemarker.core.Environment.visit(Environment.java:257)
at freemarker.core.Environment.process(Environment.java:235)
at freemarker.template.Template.process(Template.java:262)
at net.thucydides.core.reports.templates.FreemarkerReportTemplate.merge(FreemarkerReportTemplate.java:33)
at net.thucydides.core.reports.html.HtmlReporter$Merger.usingContext(HtmlReporter.java:175)
at net.thucydides.core.reports.html.HtmlAcceptanceTestReporter.generateReportFor(HtmlAcceptanceTestReporter.java:93)
at net.thucydides.core.reports.ReportService.generateReportFor(ReportService.java:210)
at net.thucydides.core.reports.ReportService.generateQueuedReport(ReportService.java:159)
at net.thucydides.core.reports.ReportService.access$000(ReportService.java:25)
at net.thucydides.core.reports.ReportService$2.run(ReportService.java:120)
at com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:299)
at com.google.common.util.concurrent.ExecutionList.executeListener(ExecutionList.java:156)
at com.google.common.util.concurrent.ExecutionList.add(ExecutionList.java:101)
at com.google.common.util.concurrent.ListenableFutureTask.addListener(ListenableFutureTask.java:83)
at net.thucydides.core.reports.ReportService.generateReportsFor(ReportService.java:117)
at net.thucydides.core.reports.ReportService.generateReportsFor(ReportService.java:97)
at net.serenitybdd.junit.runners.SerenityRunner.generateReportsFor(SerenityRunner.java:395)
at net.serenitybdd.junit.runners.SerenityRunner.generateReports(SerenityRunner.java:284)
at net.serenitybdd.junit.runners.SerenityRunner.run(SerenityRunner.java:254)
at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:78)
at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:212)
at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:68)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)
20:28:25.266 [main] ERROR net.thucydides.core.reports.templates.FreemarkerReportTemplate - Syntax error in report template: The following has evaluated to null or missing:
==> cause.message [in template "freemarker/default.ftl" at line 269, column 61]
----
Tip: It's the step after the last dot that caused this error, not those before it.
----
Tip: If the failing expression is known to be legally refer to something that's null or missing, either specify a default value like myOptionalVar!myDefault, or use <#if myOptionalVar??>when-present<#else>when-missing</#if>. (These only cover the last step of the expression; to cover the whole expression, use parenthesis: (myOptionalVar.foo)!myDefault, (myOptionalVar.foo)??
----
----
FTL stack trace ("~" means nesting-related):
- Failed at: ${cause.message} [in template "freemarker/default.ftl" in macro "stacktrace" at line 269, column 59]
- Reached through: @stacktrace cause=testOutcome.nestedT... [in template "freemarker/default.ftl" at line 406, column 37]
----
- Failed at: ${cause.message} [in template "freemarker/default.ftl" in macro "stacktrace" at line 269, column 59]
- Reached through: @stacktrace cause=testOutcome.nestedT... [in template "freemarker/default.ftl" at line 406, column 37]
May 30, 2015 8:28:25 PM com.google.common.util.concurrent.ExecutionList executeListener
SEVERE: RuntimeException while executing runnable net.thucydides.core.reports.ReportService$2@6edaa77a with executor com.google.common.util.concurrent.MoreExecutors$DirectExecutorService@1e63d216
java.lang.RuntimeException: Failed to merge template: Failed to process FreeMarker template
at net.thucydides.core.reports.html.HtmlReporter$Merger.usingContext(HtmlReporter.java:178)
at net.thucydides.core.reports.html.HtmlAcceptanceTestReporter.generateReportFor(HtmlAcceptanceTestReporter.java:93)
at net.thucydides.core.reports.ReportService.generateReportFor(ReportService.java:210)
at net.thucydides.core.reports.ReportService.generateQueuedReport(ReportService.java:159)
at net.thucydides.core.reports.ReportService.access$000(ReportService.java:25)
at net.thucydides.core.reports.ReportService$2.run(ReportService.java:120)
at com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:299)
at com.google.common.util.concurrent.ExecutionList.executeListener(ExecutionList.java:156)
at com.google.common.util.concurrent.ExecutionList.add(ExecutionList.java:101)
at com.google.common.util.concurrent.ListenableFutureTask.addListener(ListenableFutureTask.java:83)
at net.thucydides.core.reports.ReportService.generateReportsFor(ReportService.java:117)
at net.thucydides.core.reports.ReportService.generateReportsFor(ReportService.java:97)
at net.serenitybdd.junit.runners.SerenityRunner.generateReportsFor(SerenityRunner.java:395)
at net.serenitybdd.junit.runners.SerenityRunner.generateReports(SerenityRunner.java:284)
at net.serenitybdd.junit.runners.SerenityRunner.run(SerenityRunner.java:254)
at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:78)
at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:212)
at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:68)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)
Caused by: net.thucydides.core.reports.templates.TemplateMergeException: Failed to process FreeMarker template
at net.thucydides.core.reports.templates.FreemarkerReportTemplate.merge(FreemarkerReportTemplate.java:42)
at net.thucydides.core.reports.html.HtmlReporter$Merger.usingContext(HtmlReporter.java:175)
... 23 more
Caused by: freemarker.core.InvalidReferenceException: The following has evaluated to null or missing:
==> cause.message [in template "freemarker/default.ftl" at line 269, column 61]
----
Tip: It's the step after the last dot that caused this error, not those before it.
----
Tip: If the failing expression is known to be legally refer to something that's null or missing, either specify a default value like myOptionalVar!myDefault, or use <#if myOptionalVar??>when-present<#else>when-missing</#if>. (These only cover the last step of the expression; to cover the whole expression, use parenthesis: (myOptionalVar.foo)!myDefault, (myOptionalVar.foo)??
----
----
FTL stack trace ("~" means nesting-related):
- Failed at: ${cause.message} [in template "freemarker/default.ftl" in macro "stacktrace" at line 269, column 59]
- Reached through: @stacktrace cause=testOutcome.nestedT... [in template "freemarker/default.ftl" at line 406, column 37]
----
at freemarker.core.InvalidReferenceException.getInstance(InvalidReferenceException.java:108)
at freemarker.core.EvalUtil.coerceModelToString(EvalUtil.java:346)
at freemarker.core.Expression.evalAndCoerceToString(Expression.java:80)
at freemarker.core.DollarVariable.accept(DollarVariable.java:40)
at freemarker.core.Environment.visit(Environment.java:257)
at freemarker.core.MixedContent.accept(MixedContent.java:57)
at freemarker.core.Environment.visit(Environment.java:257)
at freemarker.core.Macro$Context.runMacro(Macro.java:173)
at freemarker.core.Environment.visit(Environment.java:686)
at freemarker.core.UnifiedCall.accept(UnifiedCall.java:80)
at freemarker.core.Environment.visit(Environment.java:257)
at freemarker.core.MixedContent.accept(MixedContent.java:57)
at freemarker.core.Environment.visitByHiddingParent(Environment.java:278)
at freemarker.core.ConditionalBlock.accept(ConditionalBlock.java:48)
at freemarker.core.Environment.visit(Environment.java:257)
at freemarker.core.MixedContent.accept(MixedContent.java:57)
at freemarker.core.Environment.visitByHiddingParent(Environment.java:278)
at freemarker.core.ConditionalBlock.accept(ConditionalBlock.java:48)
at freemarker.core.Environment.visit(Environment.java:257)
at freemarker.core.MixedContent.accept(MixedContent.java:57)
at freemarker.core.Environment.visitByHiddingParent(Environment.java:278)
at freemarker.core.ConditionalBlock.accept(ConditionalBlock.java:48)
at freemarker.core.Environment.visit(Environment.java:257)
at freemarker.core.MixedContent.accept(MixedContent.java:57)
at freemarker.core.Environment.visit(Environment.java:257)
at freemarker.core.Environment.process(Environment.java:235)
at freemarker.template.Template.process(Template.java:262)
at net.thucydides.core.reports.templates.FreemarkerReportTemplate.merge(FreemarkerReportTemplate.java:33)
... 24 more
20:28:25.278 [main] INFO net.thucydides.core.reports.ReportService - Report generated for Podam factory basic types test:podamShouldFillRecursivePojos
20:28:25.279 [main] INFO net.thucydides.core.reports.junit.JUnitXMLOutcomeReporter - GENERATING JUNIT REPORTS
20:28:25.279 [main] INFO net.thucydides.core.reports.junit.JUnitXMLOutcomeReporter - GENERATING JUNIT REPORT SERENITY-JUNIT-b6976dfce3c482bccc9dedd27b3bc4c425119d89600459fdbb5300a1ff1bcbe2.xml
20:28:25.302 [main] INFO net.thucydides.core.reports.ReportService - Reports generated in: 383
Process finished with exit code 255
If you want to check out the code, you can clone Podam on Github
For now I'll comment out the toString
/**
* Constructs a <code>String</code> with all attributes in name = value
* format.
*
* <p>Disabled for Serenity Issue: https://github.com/serenity-bdd/serenity-core/issues/66</p>
*
* @return a <code>String</code> representation of this object.
@Override
public String toString() {
final String TAB = " ";
StringBuilder retValue = new StringBuilder();
retValue.append("RecursivePojo ( ").append("intField = ")
.append(intField).append(TAB).append("parent = ")
.append(parent).append(TAB).append(" )");
return retValue.toString();
} */
But ideally I'd like to re-enable it
Hi, I've got the following JUnit test that passes:
@Test(expected = IllegalArgumentException.class)
@Title("Podam should throw an IllegalArgumentException if the @PodamIntValue annotation contains invalid format")
public void podamShouldThrowExceptionWhenThePodamIntValueAnnotationHasGotAnInvalidFormat() throws Exception {
PodamFactory podamFactory = podamFactorySteps.givenAStandardPodamFactory();
podamInvocationSteps.whenIInvokeTheFactoryForClass(IntegerValueWithErrorPojo.class, podamFactory);
}
However the reports show Errors. How does Serenity integrates with JUnit for (expected = ...)?
Currently, you can use a @rule of SpringIntegration to add support for spring method level annotations, such as @DirtiesContext. However, this does not support class level annotations, such as the class level @DirtiesContext. This is required when you have two classes that require fresh contexts, but the methods inside should all run with the dirty context.
For this, a companion to SpringIntegration is required that can be used with JUnit 4.9's @ClassRule, to run RunAfterTestClassCallbacks and RunAfterTestClassCallbacks.
If it would be helpful, I have written a class to do this (and a replacement for SpringIntegration, which extends a depracated class), as well as subclass of SerenityRunner that auto-adds both rules, and I could provide these, along with appropriate test classes.
I've recently updated our Serenity version from 1.0.9 to 1.0.47 and our sys ops person notified me that she is seeing the /tmp folder on various Jenkins slaves glutted with screenshot files.
Can someone confirm if this happens and why and if there is a workaround to perhaps override the temp folder?
Our setup is using:
Java + Junit
Jenkins + six slaves (linux)
Saucelabs (four configs per app test = firefox 35, chrome 38 + Mavericks, Yosemite)
Screenshots set to default behavior (before and after each step) though I'm now experimenting with FOR_FAILURES only to see if that reduces the /tmp storage.
Thanks
I am using Serenity with jBehave + jUnit.
During test run I store screenshots and adding to them custom overlay via third-party libs. Now I am trying to add generated images to report, but actually don't know how to handle it.
I look through method:
Serenity.takeScreenshot();
It takes screenshot and adds it to Step context. But there is no way to add some custom image to context.
Is there any way to do it? Please advise.
When tests fail due to an assertion in a Test class then the @after steps are executed before the results of the assertion failure are logged in the report output sequence.
When tests fail due to an exception being thrown anywhere ( Test/Step/Page ) then the @after steps do not execute and are marked as 'SKIPPED' in the report sequence. This means that cleanup tasks cannot be reliably carried out, i.e. there is no mechanism that allows for cleanup checks to be executed.
Would be nice to rename thucydides.properties to serenity.properties. Also the report folder within the site should be named serenity.
Seems that the reporting with maven is not working correctly: https://martinreinhardt-online.de/jenkins/job/serenity_maven_sample/
with serenity-core 1.0.44 :
$ gradle aggregate
...
:aggregate
Generating Serenity Reports for POS_REPORTS to directory /Users/pti/workspace/POS_REPORTS/target/site/serenity
this file was not a valid JSON Serenity test report: ca619da1feac2ae732e59d2e4590b237c6eb83cbb477f547a9fea10587ff2bed.json
null
BUILD SUCCESSFUL
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.