Comments (3)
ljacomet
commented on May 28, 2024
1
The issue is in the backlog of the relevant team and is prioritized by them.
from gradle.
itkhanz
commented on May 28, 2024
I have also tested this with disabling Gradle reports and specifying different path for TestNG reports as shown below in my build.gradle file:
test {
if (remote) {
reports {
html.required = false
junitXml.required = false
}
}
useTestNG() {
// To generate reports by TestNG library
useDefaultListeners = true
//set TestNG output dir
Provider<RegularFile> testngReportDir = layout.buildDirectory.file("testng")
outputDirectory = testngReportDir.get().asFile;
}
}
Although this disables generating Gradle HTML and XMLreports as it can also be verified from logs:
64 tests completed, 7 failed, 1 skipped
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':test'.
> There were failing tests
* Try:
> Run with --scan to get full insights.
BUILD FAILED in 1h 17m 25s
Test report disabled, omitting generation of the HTML test report.
but it still doesn't produce TestNG reports. It only produces following files in specified directory as indicated in logs:
##[debug] /__w/2/s/build/testng (directory)
##[debug] /__w/2/s/build/testng/Gradle suite (directory)
##[debug] /__w/2/s/build/testng/Gradle suite/Gradle test.html (file)
##[debug] /__w/2/s/build/testng/Gradle suite/Gradle test.xml (file)
Previously I though that TestNG reports were not getting generated because Gradle also generates report in same folder so it might be getting overwritten by Gradle so I have now also tested with specifying different paths for Gradle reports as well as even disabling the gradle reports but it does not solve the problem.
What is really strange is this issue only happens with a specific build and not others. Only difference between these builds is that the problematic build has different test groups to include for TestNG and has many tests to run.
from gradle.
itkhanz
commented on May 28, 2024
I was finally able to triage this issue and it turned out that Gradle was not able to execute tests because a NullPointerException thrown in the TestNG listener implementation class.
You can reproduce this issue by throwing a NullPointerException after the test execution in TestNG listener. I have intentionally declared the String as null in listener class to produce the NullPointerException. This causes the gradle test execution to fail.
Steps to reproduce:
DebugTest.class
import org.testng.Assert;
import org.testng.annotations.Listeners;
import org.testng.annotations.Test;
@Listeners({ com.dcs.appium.core.listeners.DebugListener.class })
public class DebugTest {
@Test
public void test_positive(){
System.out.println("this will pass");
}
@Test
public void test_negative(){
Assert.fail("this will fail");
}
}
DebugListener.class
import org.testng.ITestListener;
import org.testng.ITestResult;
public class DebugListener implements ITestListener {
private static final String car = null;
@Override
public void onTestSuccess(ITestResult result) {
System.out.println("Printing value of car on test success");
System.out.println(car.toUpperCase());
}
@Override
public void onTestFailure(ITestResult result) {
System.out.println("Printing value of car on test failure");
System.out.println(car.toUpperCase());
}
}
build.gradle
test {
useTestNG() {
useDefaultListeners = true
}
}
Run your tests with commandline as gradle clean test --tests 'DebugTest' --debug
It will cause the gradle test execution to fail. You can view the following output in console and also in the HTML reports produced by Gradle.
2024-05-14T12:47:12.482+0200 [DEBUG] [TestEventLogger]
2024-05-14T12:47:12.482+0200 [DEBUG] [TestEventLogger] Gradle suite > Gradle test > com.dcs.appium.dcsapps.Tests.DebugTest FAILED
2024-05-14T12:47:12.482+0200 [DEBUG] [TestEventLogger]
2024-05-14T12:47:12.482+0200 [DEBUG] [TestEventLogger] Gradle suite > Gradle test FAILED
2024-05-14T12:47:12.482+0200 [DEBUG] [TestEventLogger]
2024-05-14T12:47:12.482+0200 [DEBUG] [TestEventLogger] Gradle suite FAILED
2024-05-14T12:47:12.482+0200 [DEBUG] [TestEventLogger]
2024-05-14T12:47:12.482+0200 [DEBUG] [TestEventLogger] Gradle Test Executor 31 FAILED
2024-05-14T12:47:12.482+0200 [DEBUG] [TestEventLogger] org.gradle.api.internal.tasks.testing.TestSuiteExecutionException: Could not complete execution for Gradle Test Executor 31.
2024-05-14T12:47:12.482+0200 [DEBUG] [TestEventLogger] at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:64)
2024-05-14T12:47:12.482+0200 [DEBUG] [TestEventLogger] at [email protected]/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2024-05-14T12:47:12.482+0200 [DEBUG] [TestEventLogger] at [email protected]/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
2024-05-14T12:47:12.482+0200 [DEBUG] [TestEventLogger] at [email protected]/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2024-05-14T12:47:12.482+0200 [DEBUG] [TestEventLogger] at [email protected]/java.lang.reflect.Method.invoke(Method.java:568)
2024-05-14T12:47:12.482+0200 [DEBUG] [TestEventLogger] at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
2024-05-14T12:47:12.482+0200 [DEBUG] [TestEventLogger] at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
2024-05-14T12:47:12.483+0200 [DEBUG] [TestEventLogger] at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
2024-05-14T12:47:12.483+0200 [DEBUG] [TestEventLogger] at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
2024-05-14T12:47:12.483+0200 [DEBUG] [TestEventLogger] at jdk.proxy1/jdk.proxy1.$Proxy2.stop(Unknown Source)
2024-05-14T12:47:12.483+0200 [DEBUG] [TestEventLogger] at org.gradle.api.internal.tasks.testing.worker.TestWorker$3.run(TestWorker.java:193)
2024-05-14T12:47:12.483+0200 [DEBUG] [TestEventLogger] at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:129)
2024-05-14T12:47:12.483+0200 [DEBUG] [TestEventLogger] at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:100)
2024-05-14T12:47:12.483+0200 [DEBUG] [TestEventLogger] at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:60)
2024-05-14T12:47:12.483+0200 [DEBUG] [TestEventLogger] at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56)
2024-05-14T12:47:12.483+0200 [DEBUG] [TestEventLogger] at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:113)
2024-05-14T12:47:12.483+0200 [DEBUG] [TestEventLogger] at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:65)
2024-05-14T12:47:12.483+0200 [DEBUG] [TestEventLogger] at app//worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69)
2024-05-14T12:47:12.483+0200 [DEBUG] [TestEventLogger] at app//worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74)
2024-05-14T12:47:12.483+0200 [DEBUG] [TestEventLogger]
2024-05-14T12:47:12.483+0200 [DEBUG] [TestEventLogger] Caused by:
2024-05-14T12:47:12.483+0200 [DEBUG] [TestEventLogger] java.lang.NullPointerException: Cannot invoke "String.toUpperCase()" because "com.dcs.appium.core.listeners.DebugListener.car" is null
2024-05-14T12:47:12.483+0200 [DEBUG] [TestEventLogger] at com.dcs.appium.core.listeners.DebugListener.onTestFailure(DebugListener.java:19)
2024-05-14T12:47:12.483+0200 [DEBUG] [TestEventLogger] at org.testng.internal.TestListenerHelper.runTestListeners(TestListenerHelper.java:115)
2024-05-14T12:47:12.483+0200 [DEBUG] [TestEventLogger] at org.testng.internal.invokers.TestInvoker.runTestResultListener(TestInvoker.java:285)
2024-05-14T12:47:12.483+0200 [DEBUG] [TestEventLogger] at org.testng.internal.invokers.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:1010)
2024-05-14T12:47:12.483+0200 [DEBUG] [TestEventLogger] at org.testng.internal.invokers.TestInvoker.invokeTestMethods(TestInvoker.java:203)
2024-05-14T12:47:12.483+0200 [DEBUG] [TestEventLogger] at org.testng.internal.invokers.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:154)
2024-05-14T12:47:12.483+0200 [DEBUG] [TestEventLogger] at org.testng.internal.invokers.TestMethodWorker.run(TestMethodWorker.java:134)
2024-05-14T12:47:12.483+0200 [DEBUG] [TestEventLogger] at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
2024-05-14T12:47:12.483+0200 [DEBUG] [TestEventLogger] at org.testng.TestRunner.privateRun(TestRunner.java:739)
2024-05-14T12:47:12.483+0200 [DEBUG] [TestEventLogger] at org.testng.TestRunner.run(TestRunner.java:614)
2024-05-14T12:47:12.483+0200 [DEBUG] [TestEventLogger] at org.testng.SuiteRunner.runTest(SuiteRunner.java:421)
2024-05-14T12:47:12.483+0200 [DEBUG] [TestEventLogger] at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:413)
2024-05-14T12:47:12.483+0200 [DEBUG] [TestEventLogger] at org.testng.SuiteRunner.privateRun(SuiteRunner.java:373)
2024-05-14T12:47:12.483+0200 [DEBUG] [TestEventLogger] at org.testng.SuiteRunner.run(SuiteRunner.java:312)
2024-05-14T12:47:12.483+0200 [DEBUG] [TestEventLogger] at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
2024-05-14T12:47:12.483+0200 [DEBUG] [TestEventLogger] at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:95)
2024-05-14T12:47:12.483+0200 [DEBUG] [TestEventLogger] at org.testng.TestNG.runSuitesSequentially(TestNG.java:1274)
2024-05-14T12:47:12.483+0200 [DEBUG] [TestEventLogger] at org.testng.TestNG.runSuitesLocally(TestNG.java:1208)
2024-05-14T12:47:12.483+0200 [DEBUG] [TestEventLogger] at org.testng.TestNG.runSuites(TestNG.java:1112)
2024-05-14T12:47:12.483+0200 [DEBUG] [TestEventLogger] at org.testng.TestNG.run(TestNG.java:1079)
2024-05-14T12:47:12.483+0200 [DEBUG] [TestEventLogger] at org.gradle.api.internal.tasks.testing.testng.TestNGTestClassProcessor.runTests(TestNGTestClassProcessor.java:153)
2024-05-14T12:47:12.483+0200 [DEBUG] [TestEventLogger] at org.gradle.api.internal.tasks.testing.testng.TestNGTestClassProcessor.stop(TestNGTestClassProcessor.java:96)
2024-05-14T12:47:12.483+0200 [DEBUG] [TestEventLogger] at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:62)
2024-05-14T12:47:12.483+0200 [DEBUG] [TestEventLogger] ... 18 more
2024-05-14T12:47:12.497+0200 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Dispatch org.gradle.api.internal.tasks.testing.processors.RestartEveryNTestClassProcessor@3e28d049: released lock on worker lease
2024-05-14T12:47:12.497+0200 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker: acquired lock on state of build :
2024-05-14T12:47:12.497+0200 [DEBUG] [org.gradle.internal.resources.AbstractTrackedResourceLock] Execution worker: acquired lock on worker lease
2024-05-14T12:47:12.497+0200 [LIFECYCLE] [org.gradle.api.Task]
2024-05-14T12:47:12.498+0200 [LIFECYCLE] [org.gradle.api.Task] 0 passing (495ms)
2 failing
2024-05-14T12:47:12.499+0200 [QUIET] [system.out]
2024-05-14T12:47:12.499+0200 [QUIET] [system.out] ---------------------------------------------------------------
2024-05-14T12:47:12.499+0200 [QUIET] [system.out] | Results: FAILURE (2 tests, 0 passed, 2 failed, 0 skipped) |
2024-05-14T12:47:12.499+0200 [QUIET] [system.out] ---------------------------------------------------------------
2024-05-14T12:47:12.499+0200 [DEBUG] [TestEventLogger]
2024-05-14T12:47:12.499+0200 [DEBUG] [TestEventLogger] Gradle Test Run :test FAILED
2024-05-14T12:47:12.499+0200 [ERROR] [org.gradle.api.internal.tasks.testing.logging.TestCountLogger]
2 tests completed, 2 failed
i am not sure whether this behaviour is intended in Gradle. Even if the tests are failing, we should still be able to see the testng-results.xml reports getting properly generated but the only reports that are generated for TestNG are:
Gradle suite/ Gradle test.html
Gradle suite/ Gradle test.xml
This error does not occur if NullPointerException directly in test methods. In that case, gradle is still able to handle this gracefully and execute tests. The only time, gradle is unable to execute tests if this issue happens in listener and then we get this error:
org.gradle.api.internal.tasks.testing.TestSuiteExecutionException: Could not complete execution for Gradle Test Executor 30.
from gradle.
Related Issues (20)
- On starting `GradleRunner` with assertions enabled HOT 1
- Shadow plugin works incorrectly while depending on Nar packages HOT 1
- TAPI out of memory in Team city agent
- Compiler problem reporting causes `IllegalArgumentException` with `Corretto-8.412.08.1` HOT 2
- Classloader issue related to `java.xml.bind` in ArtifactTransform HOT 3
- Dependency of plugin gets ignored when a settings plugin has the same dependency HOT 1
- Support Scoop installed JDK discovery on Windows HOT 2
- Allow wrapper to use distribution credential from environment HOT 2
- Ability to inject custom comandline arguments to TestNG is missing HOT 2
- Document behavior of `Property` objects in relation to parallel task execution
- FileCollection.elements fail under load.
- Make `-x` Isolated Projects compatible
- Make `evaluationDependsOn` Isolated Projects compatible
- Using a software type with an empty configuration block has no effect in DCL
- Android Studio 2.4 Preview 5 April 7, 2017 Not working with radl HOT 1
- ConventionMapping doesn't work correctly for DirectoryProperty and RegularFileProperty after upgrade
- Introduce `@ToBeReplacedByLazyProperty` annotation
- Simplify testing the distribution (bust up DistributionIntegrationSpec)
- Introduce a replacement for `gradle.allprojects {...}` suitable for incremental adoption HOT 1
- Get rid of custom serialization when reporting task validation problems via the Problems API HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gradle.