radarsh / gradle-test-logger-plugin Goto Github PK

Publish the plugin to Gradle Central

If a theme is configured by using the testlogger DSL, it doesn't allow one to override the theme to plain by specifying --console plain or --console=plain.

Fix flakey tests caused by non-deterministic test execution order

With a config like this:

testlogger {
	showStandardStreams true
	
	showPassedStandardStreams false
	showSkippedStandardStreams false
	showFailedStandardStreams true
}

if the test suite run itself fails - i.e. not 'a test does not pass' but the test execution failing due to something else e.g. an @Sql setup script with an error in it - gradle fails and stderr says:

> Task :test FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':test'.

but there's no indication as to what the problem was. If I run again with --info I can see that the stdout explains the problem in detail, but that disables the above testlogger filters.

Could testlogger notice the suite failure and show the output please?

Unicode characters are displayed as ? on Windows (PowerShell, Command Prompt, Git Bash).

Upgrade plugin and tests to use Gradle 4.3

Test failures should print more detail by default. This should be the equivalent of configuring the standard Gradle test logging with the below settings.

testLogging {
    exceptionFormat "full"
    showExceptions true
    showCauses false
    showStackTraces true
}

Update documentation with usage instructions and GIFs for different platforms.

This plugin project should itself use the plugin during testing. The Circle CI config should be updated to run tests with --console=plain after this.

In order to allow different configurations at runtime support overriding configuration by using system properties.

Hi,
i really like your plugin and the well formatted output but the bright yellow of the test class name is really hard to read on bright backgrounds like jenkins has.
Would it be possible to exchange all bright yellows to another color?

Thanks

If a suite has no tests in it, it should not be printed. To reproduce, use Gradle test filtering and select a single test. Notice how all the other suites are also printed even though no tests within them are executed.

Erase ahead should be added to suite text

Refresh screenshot animations to demonstrate recent capabilities. Also consider adding screenshot images.

Would be great to have an option to show only failed tests in CI(Jenkins) build output. We have pretty big amount of tests and Jenkins output overflows several times, while at the same time in case if build failed on CI passed tests are not necessary things there. Smth. like

testLogger {
    showOnlyFailedInPlainOutputMode true
}

This will work only for --console=plain option enabled.
Also maybe legit if this flag is enabled by default for --console=plain

Add @CompileStatic to more classes to improve performance

I like the plugin but I ran into a couple things that would be nice to have (especially during first-time use):

1. Small but took a minute to find - I am used to camelCase with Java and Groovy names, so I started out with testLogger eventually realized the L is not capitalized. Maybe make both valid or just point out the difference in the documentation.
2. My projects use parallel tests which caused the theme to fail. It appears to know that I am using parallel testing - if this is the case, could you not switch to the parallel version of the same theme and provide a warning log that this change was made.

These are just suggestions - feel free to close this issue if they are not something you want to pursue.

Perform code coverage analysis using coveralls.io and display the status using a badge on the README.

Hi,

the plugin is great but I have a question regarding jUnit 5? Are you planning to support it on the future?

Thanks

I'm not sure if this is within this project's remit, or if there's a better way to do it, but:

When a test fails I wish I'd increased app (not test) logging verbosity. But when tests are good that logging is verbose and very cluttery. Could gradle-test-logger optionally capture the running program's output (I assume slf4j is outputting to stdout) and only output it if the test fails?

Hello there,

Thank you very much for this nice plugin! We really need user-friendly tools like yours in the Java community!

I was looking for a way to enable printing of stdout/stderr of my tests, and everywhere I googled I was pointed at gradle Test task's testLogging.showStandardStreams option.

But because I use gradle-test-logger-plugin, this wasn't working. After some time it dawned on me that the issue may come from the plugin, and I was able to confirm this and then go into the documentation and see that with this plugin, one should actually use testLogger.showStandardStreams instead.

Since this is a bit confusing, I would like to suggest that support for reading from testLogging.showStandardStreams also be added. I believe that would a lot more predictable for people with existing test configurations, and hopefully no other users would lose a few minutes banging their heads against this small issue.

Thanks for your time! 🙏

What I expect to see:
Test thisIsASlowTestThatFailed FAILED (9.2s)

What I see:
Test thisIsASlowTestThatFailed FAILED

Introduce a high level abstraction for collecting standard stream output which should hide the details of whether we're using a StringBuilder or anything else internally.

Add AppVeyor build badge to README

After completing a suite, a summary of some kind would be nice:

testlogger {
    showSummary true|false
}

The color in the current output is extremely useful for finding exactly which test failed while scrolling through, the summary could be an at-a-glance way to know what failed. I'm thinking the following pieces of information would be useful:

• Task status: SUCCESSFUL/FAILED/ERROR
• Number of tests
• Number passed
• Number skipped
• Number failed
• Number errored
• List of Package/Class/Method of failed and errored tests

I've been putting the following in our build.gradle files, but it would be nice not to have to repeat it everywhere:

afterSuite { desc, result ->
       if (!desc.parent) { // will match the outermost suite
           def output = "Results: ${result.resultType} (${result.testCount} tests, ${result.successfulTestCount} successes, ${result.failedTestCount} failures, ${result.skippedTestCount} skipped)"
           def startItem = '|  ', endItem = '  |'
           def repeatLength = startItem.length() + output.length() + endItem.length()
           println('\n' + ('-' * repeatLength) + '\n' + startItem + output + endItem + '\n' + ('-' * repeatLength))
       }
   }

Document system property overrides

Gradle 4.7 changes the meaning displayName and so has breaking changes for this plugin. Currently we should circumvent them as per advice received on gradle/gradle#5117.

Use consistent status badges from shields.io

Allow the definition of a slow threshold value using the config DSL.

In some of the projects I work on, they split tests up with additional source sets to allow the task to incrementally run (i.e. unitTest, integrationTest, systemTest). It would be nice if I could hook this into these other test tasks.

testlogger { testTasks ['test', 'integrationTest', 'systemTest'] }

Or optionally, hook into any Test type task by default:
tasks.withType(Test)

Dear Developers,

I tried to apply this plugin to my project, but unfortunately I got the following exception :

• Exception is:
  org.gradle.api.plugins.InvalidPluginException: An exception occurred applying plugin request [id: 'com.adarshr.test-logger', version: '1.1.2']
  at org.gradle.plugin.use.internal.DefaultPluginRequestApplicator.applyPlugin(DefaultPluginRequestApplicator.java:178)
  at org.gradle.plugin.use.internal.DefaultPluginRequestApplicator.applyPlugins(DefaultPluginRequestApplicator.java:130)
  at org.gradle.configuration.DefaultScriptPluginFactory$ScriptPluginImpl.apply(DefaultScriptPluginFactory.java:126)
  at org.gradle.configuration.project.BuildScriptProcessor.execute(BuildScriptProcessor.java:38)
  at org.gradle.configuration.project.BuildScriptProcessor.execute(BuildScriptProcessor.java:25)
  at org.gradle.configuration.project.ConfigureActionsProjectEvaluator.evaluate(ConfigureActionsProjectEvaluator.java:34)
  at org.gradle.configuration.project.LifecycleProjectEvaluator.evaluate(LifecycleProjectEvaluator.java:55)
  at org.gradle.api.internal.project.AbstractProject.evaluate(AbstractProject.java:510)
  at org.gradle.api.internal.project.AbstractProject.evaluate(AbstractProject.java:90)
  at org.gradle.execution.TaskPathProjectEvaluator.configureHierarchy(TaskPathProjectEvaluator.java:42)
  at org.gradle.configuration.DefaultBuildConfigurer.configure(DefaultBuildConfigurer.java:35)
  at org.gradle.initialization.DefaultGradleLauncher$2.run(DefaultGradleLauncher.java:125)
  at org.gradle.internal.Factories$1.create(Factories.java:22)
  at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:90)
  at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:52)
  at org.gradle.initialization.DefaultGradleLauncher.doBuildStages(DefaultGradleLauncher.java:122)
  at org.gradle.initialization.DefaultGradleLauncher.access$200(DefaultGradleLauncher.java:32)
  at org.gradle.initialization.DefaultGradleLauncher$1.create(DefaultGradleLauncher.java:99)
  at org.gradle.initialization.DefaultGradleLauncher$1.create(DefaultGradleLauncher.java:93)
  at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:90)
  at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:62)
  at org.gradle.initialization.DefaultGradleLauncher.doBuild(DefaultGradleLauncher.java:93)
  at org.gradle.initialization.DefaultGradleLauncher.run(DefaultGradleLauncher.java:82)
  at org.gradle.launcher.exec.InProcessBuildActionExecuter$DefaultBuildController.run(InProcessBuildActionExecuter.java:94)
  at org.gradle.tooling.internal.provider.ExecuteBuildActionRunner.run(ExecuteBuildActionRunner.java:28)
  at org.gradle.launcher.exec.ChainingBuildActionRunner.run(ChainingBuildActionRunner.java:35)
  at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:43)
  at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:28)
  at org.gradle.launcher.exec.ContinuousBuildActionExecuter.execute(ContinuousBuildActionExecuter.java:78)
  at org.gradle.launcher.exec.ContinuousBuildActionExecuter.execute(ContinuousBuildActionExecuter.java:48)
  at org.gradle.launcher.exec.DaemonUsageSuggestingBuildActionExecuter.execute(DaemonUsageSuggestingBuildActionExecuter.java:51)
  at org.gradle.launcher.exec.DaemonUsageSuggestingBuildActionExecuter.execute(DaemonUsageSuggestingBuildActionExecuter.java:28)
  at org.gradle.launcher.cli.RunBuildAction.run(RunBuildAction.java:43)
  at org.gradle.internal.Actions$RunnableActionAdapter.execute(Actions.java:170)
  at org.gradle.launcher.cli.CommandLineActionFactory$ParseAndBuildAction.execute(CommandLineActionFactory.java:237)
  at org.gradle.launcher.cli.CommandLineActionFactory$ParseAndBuildAction.execute(CommandLineActionFactory.java:210)
  at org.gradle.launcher.cli.JavaRuntimeValidationAction.execute(JavaRuntimeValidationAction.java:35)
  at org.gradle.launcher.cli.JavaRuntimeValidationAction.execute(JavaRuntimeValidationAction.java:24)
  at org.gradle.launcher.cli.CommandLineActionFactory$WithLogging.execute(CommandLineActionFactory.java:206)
  at org.gradle.launcher.cli.CommandLineActionFactory$WithLogging.execute(CommandLineActionFactory.java:169)
  at org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:33)
  at org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:22)
  at org.gradle.launcher.Main.doAction(Main.java:33)
  at org.gradle.launcher.bootstrap.EntryPoint.run(EntryPoint.java:45)
  at org.gradle.launcher.bootstrap.ProcessBootstrap.runNoExit(ProcessBootstrap.java:54)
  at org.gradle.launcher.bootstrap.ProcessBootstrap.run(ProcessBootstrap.java:35)
  at org.gradle.launcher.GradleMain.main(GradleMain.java:23)
  at org.gradle.wrapper.BootstrapMainStarter.start(BootstrapMainStarter.java:30)
  at org.gradle.wrapper.WrapperExecutor.execute(WrapperExecutor.java:129)
  at org.gradle.wrapper.GradleWrapperMain.main(GradleWrapperMain.java:61)
  Caused by: org.gradle.api.internal.plugins.PluginApplicationException: Failed to apply plugin [id 'com.adarshr.test-logger']
  at org.gradle.api.internal.plugins.DefaultPluginManager.doApply(DefaultPluginManager.java:160)
  at org.gradle.api.internal.plugins.DefaultPluginManager.apply(DefaultPluginManager.java:112)
  at org.gradle.plugin.use.internal.DefaultPluginRequestApplicator$5.run(DefaultPluginRequestApplicator.java:132)
  at org.gradle.plugin.use.internal.DefaultPluginRequestApplicator.applyPlugin(DefaultPluginRequestApplicator.java:166)
  ... 49 more
  Caused by: org.gradle.internal.reflect.ObjectInstantiationException: Could not create an instance of type com.adarshr.gradle.testlogger.TestLoggerExtension_Decorated.
  at org.gradle.internal.reflect.DirectInstantiator.newInstance(DirectInstantiator.java:51)
  at org.gradle.api.internal.ClassGeneratorBackedInstantiator.newInstance(ClassGeneratorBackedInstantiator.java:36)
  at org.gradle.api.internal.plugins.DefaultConvention.create(DefaultConvention.java:106)
  at com.adarshr.gradle.testlogger.TestLoggerPlugin.apply(TestLoggerPlugin.groovy:13)
  at com.adarshr.gradle.testlogger.TestLoggerPlugin.apply(TestLoggerPlugin.groovy)
  at org.gradle.api.internal.plugins.ImperativeOnlyPluginApplicator.applyImperative(ImperativeOnlyPluginApplicator.java:35)
  at org.gradle.api.internal.plugins.RuleBasedPluginApplicator.applyImperative(RuleBasedPluginApplicator.java:44)
  at org.gradle.api.internal.plugins.DefaultPluginManager.doApply(DefaultPluginManager.java:144)
  ... 52 more
  Caused by: java.lang.NoClassDefFoundError: Lorg/gradle/api/logging/configuration/ConsoleOutput;
  at com.adarshr.gradle.testlogger.TestLoggerExtension.$getStaticMetaClass(TestLoggerExtension.groovy)
  at com.adarshr.gradle.testlogger.TestLoggerExtension.(TestLoggerExtension.groovy)
  at com.adarshr.gradle.testlogger.TestLoggerExtension_Decorated.(Unknown Source)
  at org.gradle.internal.reflect.DirectInstantiator.newInstance(DirectInstantiator.java:49)
  ... 59 more
  Caused by: java.lang.ClassNotFoundException: org.gradle.api.logging.configuration.ConsoleOutput
  ... 63 more

Do you have any ideas about that ?

Fix CircleCI build instability

When using the plain theme, the output is fine:

com.toys.OperationTest

  Test testTimes() FAILED

  org.opentest4j.AssertionFailedError: TIMES should multiply 2 and 3 to be 6 ==> expected: <5.0> but was: <6.0>
      at com.toys.OperationTest.testTimes(OperationTest.java:20)

  Test testPlus() PASSED

FAILURE: Executed 2 tests in 853ms (1 failed)

However, when using standard or mocha the output is:

�[0K�[1mcom.toys.OperationTest�[m

�[0K�[1m  Test �[22mtestTimes()�[31m FAILED�[31m

  org.opentest4j.AssertionFailedError: TIMES should multiply 2 and 3 to be 6 ==> expected: <5.0> but was: <6.0>
      at com.toys.OperationTest.testTimes(OperationTest.java:20)
�[m
�[0K�[1m  Test �[22mtestPlus()�[32m PASSED�[m

�[0K�[1;31mFAILURE: �[39mExecuted 2 tests in 817ms (1 failed)�[m

The run configuration for the gradle test command specifies UTF-8 encoding.

Using Eclipse 4.9

Fix Gradle deprecation warnings

Currently if maxParallelForks is set to anything other than 1, the plugin throws an exception. This can be quite limiting.

Hi!

Here's our story: a teammate and I were trying to get your plugin working on our project. It took us a good amount of time, but we figured out why it wasn't working: the junit-platform-gradle-plugin was preventing most of the output from this plugin.

While there is a small blurb in the documentation about that plugin being deprecated, we still had it in our build.gradle because, I think, of some copy-pasta from older JUnit5 documentation.

It may be worth adding a note to your README that this might cause problems.

Regardless, this plugin works great! Thanks for the work you put in to it.

Upgrade to Gradle 4.9

Mocha theme should more accurately represent Mocha's spec reporter. See example below.

The time next to slow tests has 2 states - red if the test duration is greater than slow threshold and yellow if it's about half the slow threshold but below it.

Fix the broken coverage badge

Adding the plugin to my android project throws the following error:

Cannot cast object 'task ':app:test'' with class 'org.gradle.api.DefaultTask_Decorated' to class 'org.gradle.api.tasks.testing.Test'.

Any ideas?

Add TestNG based tests to expand the safety net.

Test logger should be configurable for each task (of type Test of course) independently. So if a project has multiple test tasks, they should be able define different values. A typical example:

testlogger {
    theme 'standard' // global
}

test {
    maxParallelForks = 8
    testlogger {
        theme 'standard-parallel' // use a parallel theme
    }
}

task functionalTest(type: Test) {
    maxParallelForks = 1 // no parallelisation
    testlogger {
        theme 'mocha'
        slowThreshold 10000 // as my functional tests are normally slow
    } 
    
    include 'com/adarshr/gradle/testlogger/functional/**'
}

Currently testlogger can only be configured globally.

Upgrade to Gradle 4.5.1

Upgrade to Gradle 4.10

Upgrade both the project and the tests to use Gradle 4.4

Encapsulate TestResult and TestDescriptor along with all their associated methods.

I like this plugin, but only wish it had an option to display the standard output stream.

Granted, I could add the standard Gradle-i or --info switch, but adds too much extra info. Is there a middle ground?

If there is a line on the console that's longer than what the summary would be, it isn't fully overwritten.

In the above case, the summary should've read

FAILURE: Executed 22 tests in 8.8s (2 failed, 1 skipped)

Not sure if this is possible with JUnit, but I'd like to see the list of tests before they're run. I have some long test suites and knowing that the tests I want to execute will be executed is very much useful to me.

I'm using gradle 4.6 and JUnit 5.

If I run tests using 8a70dcd I get the stacktrace below. If I go a few commits back to a9dfde0 it is fine.

No such property: displayName for class: org.gradle.api.internal.tasks.testing.DecoratingTestDescriptor
groovy.lang.MissingPropertyException: No such property: displayName for class: org.gradle.api.internal.tasks.testing.DecoratingTestDescriptor
        at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.unwrap(ScriptBytecodeAdapter.java:53)
        at org.codehaus.groovy.runtime.callsite.GetEffectivePojoPropertySite.getProperty(GetEffectivePojoPropertySite.java:66)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callGetProperty(AbstractCallSite.java:296)
        at com.adarshr.gradle.testlogger.theme.StandardTheme.testText(StandardTheme.groovy:20)
        at com.adarshr.gradle.testlogger.theme.Theme$testText$1.call(Unknown Source)
        at com.adarshr.gradle.testlogger.logger.TestEventLogger.afterTest(TestEventLogger.groovy:62)
        at sun.reflect.GeneratedMethodAccessor107.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
        at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
        at org.gradle.internal.event.AbstractBroadcastDispatch.dispatch(AbstractBroadcastDispatch.java:42)
        at org.gradle.internal.event.BroadcastDispatch$SingletonDispatch.dispatch(BroadcastDispatch.java:230)
        at org.gradle.internal.event.BroadcastDispatch$SingletonDispatch.dispatch(BroadcastDispatch.java:149)
        at org.gradle.internal.event.AbstractBroadcastDispatch.dispatch(AbstractBroadcastDispatch.java:58)
        at org.gradle.internal.event.BroadcastDispatch$CompositeDispatch.dispatch(BroadcastDispatch.java:324)
        at org.gradle.internal.event.BroadcastDispatch$CompositeDispatch.dispatch(BroadcastDispatch.java:234)
        at org.gradle.internal.event.ListenerBroadcast.dispatch(ListenerBroadcast.java:140)
        at org.gradle.internal.event.ListenerBroadcast.dispatch(ListenerBroadcast.java:37)
        at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93)
        at com.sun.proxy.$Proxy93.afterTest(Unknown Source)
        at org.gradle.api.internal.tasks.testing.results.TestListenerAdapter.completed(TestListenerAdapter.java:50)
        at sun.reflect.GeneratedMethodAccessor108.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
        at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
        at org.gradle.internal.event.AbstractBroadcastDispatch.dispatch(AbstractBroadcastDispatch.java:42)
        at org.gradle.internal.event.BroadcastDispatch$SingletonDispatch.dispatch(BroadcastDispatch.java:230)
        at org.gradle.internal.event.BroadcastDispatch$SingletonDispatch.dispatch(BroadcastDispatch.java:149)
        at org.gradle.internal.event.AbstractBroadcastDispatch.dispatch(AbstractBroadcastDispatch.java:58)
        at org.gradle.internal.event.BroadcastDispatch$CompositeDispatch.dispatch(BroadcastDispatch.java:324)
        at org.gradle.internal.event.BroadcastDispatch$CompositeDispatch.dispatch(BroadcastDispatch.java:234)
        at org.gradle.internal.event.ListenerBroadcast.dispatch(ListenerBroadcast.java:140)
        at org.gradle.internal.event.ListenerBroadcast.dispatch(ListenerBroadcast.java:37)
        at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93)
        at com.sun.proxy.$Proxy95.completed(Unknown Source)
        at org.gradle.api.internal.tasks.testing.results.StateTrackingTestResultProcessor.completed(StateTrackingTestResultProcessor.java:71)
        at org.gradle.api.internal.tasks.testing.results.AttachParentTestResultProcessor.completed(AttachParentTestResultProcessor.java:56)
        at sun.reflect.GeneratedMethodAccessor114.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
        at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
        at org.gradle.internal.dispatch.FailureHandlingDispatch.dispatch(FailureHandlingDispatch.java:29)
        at org.gradle.internal.dispatch.AsyncDispatch.dispatchMessages(AsyncDispatch.java:133)
        at org.gradle.internal.dispatch.AsyncDispatch.access$000(AsyncDispatch.java:34)
        at org.gradle.internal.dispatch.AsyncDispatch$1.run(AsyncDispatch.java:73)
        at org.gradle.internal.operations.BuildOperationIdentifierPreservingRunnable.run(BuildOperationIdentifierPreservingRunnable.java:39)
        at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63)
        at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:46)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:55)
        at java.lang.Thread.run(Thread.java:745)