Testsuite fatal error output not shown about gradle-test-logger-plugin HOT 8 CLOSED

Hmm, that's not a nice behaviour from the plugin. Let me see what I can do. Thanks for reporting this.

from gradle-test-logger-plugin.

I've investigated what's happening: System.exit(5) is being called.

I guess this is somewhat unexpected behaviour for JUnit tests but I have seen several StackOverflow posts suggest that as a way to abort the rest of the test suite so as to not waste when a fatal error has happened.

Would it be right for gradle-test-logger-plugin to catch the VM's non-zero exit code, treat that as a failed test and show the output?

Thanks.

from gradle-test-logger-plugin.

Hi @molexx, would you be able to provide an sscce for this so that I can provide an accurate fix? Thanks.

from gradle-test-logger-plugin.

Interesting. Would be good to find out what Gradle would do without the test logger plugin applied and with a configuration like this:

test {
    testLogging {
        showStandardStreams = true
    }
}

from gradle-test-logger-plugin.

Hi @molexx, could you share a bit more detail about this please? Where is System.exit(5) being called from? Is it in a setup method? Also what happens when you try this? Thanks. This info will be quite invaluable.

from gradle-test-logger-plugin.

Behaviour is the same for @Test and @Before methods which do some output before calling System.exit(5):

Without gradle-test-logger applied and without the above test{ testLogging { showStandardStreams=true}} the stdout/stderr messages are not shown.

Without gradle-test-logger applied but with the above test{ testLogging { showStandardStreams=true}} the stdout/stderr messages are shown.

With gradle-test-logger applied the test is reported as SKIPPED (in yellow), then the expected behaviour happens: if testLogger's showStandardStreams and showSkippedStandardStreams are true the messages are seen.
I don't think it should be detected as SKIPPED though, as the test suite's VM aborted without the test completing and with status 5 (which gradle notices and reports as in the first post).

@BeforeClass methods are similar without gradle-test-logger - with test{ testLogging { showStandardStreams=true}} the outputs show.

But with gradle-test-logger applied, even with all options set to true:

testlogger {
	showStandardStreams true
	
	showPassedStandardStreams true
	showSkippedStandardStreams true
	showFailedStandardStreams true
}

the System.out and System.err in the @BeforeClass method are not seen.

from gradle-test-logger-plugin.

I don't think it should be detected as SKIPPED though, as the test suite's VM aborted without the test completing and with status 5 (which gradle notices and reports as in the first post).

Unfortunately, there is nothing the plugin can do about this as this is Gradle's internal behaviour. I have fixed the missing system out issue though. It was a genuine bug, thanks for reporting it and the detailed analysis 👍

from gradle-test-logger-plugin.

Ah ok seeing the messages is the main thing, thanks! :-)

from gradle-test-logger-plugin.