Fail in the TestMain not recognized about go-junit-report HOT 9 CLOSED

Thanks for the example, I can reproduce it now. This is definitely a problem and I can see where it goes wrong. Once I find some time I'll see if I can fix it.

from go-junit-report.

@jstemmer hey, sorry, got a bit distracted and forgot about this issue. I will try to reproduce it next week and share the results with you.

from go-junit-report.

You didn't include the generated report.xml, what did it look like?

Running your output through go-junit-report results in an XML that reports my_package/tests as having a failure (it even includes the correct failure message) and the exit code is 1.

$ cat /tmp/output
?       my_pacakge/file1        [no test files]
?       my_pacakge/file2        [no test files]
time="2018-03-28T12:47:03Z" level=fatal msg="cannot connect to mysql: ..."
FAIL    my_packege/tests        0.156s
=== RUN   Test1
--- PASS: Test1 (0.00s)
=== RUN   Test2
--- PASS: Test2 (0.00s)

$ cat /tmp/output | go-junit-report --set-exit-code ; echo $?
<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
        <testsuite tests="1" failures="1" time="0.156" name="my_packege/tests">
                <properties>
                        <property name="go.version" value="go1.10"></property>
                </properties>
                <testcase classname="tests" name="Failure" time="0.000">
                        <failure message="Failed" type="">time=&#34;2018-03-28T12:47:03Z&#34; level=fatal msg=&#34;cannot connect to mysql: ...&#34;</failure>
                </testcase>
        </testsuite>
        <testsuite tests="2" failures="0" time="0.000" name="">
                <properties>
                        <property name="go.version" value="go1.10"></property>
                </properties>
                <testcase classname="" name="Test1" time="0.000"></testcase>
                <testcase classname="" name="Test2" time="0.000"></testcase>
        </testsuite>
</testsuites>
1

Are you using the latest version of go-junit-report? And is the output you pasted accurate?

from go-junit-report.

@jstemmer hey, I was able to reproduce the issue.
The example I posted above worked fine, because I shortened, and apparently removed the pattern :(

So here is the example that not being detected as problem:

=== RUN   Test0
--- PASS: Test0 (0.00s)
PASS
?   	my_pacakge/file1	[no test files]
?   	my_pacakge/file2	[no test files]
time="2018-03-28T12:47:03Z" level=fatal msg="cannot connect to mysql: ..."
FAIL	my_packege/tests	0.156s
=== RUN   Test1
--- PASS: Test1 (0.00s)
=== RUN   Test2
--- PASS: Test2 (0.00s)

It is completely the same, just added:

=== RUN   Test0
--- PASS: Test0 (0.00s)
PASS

at the beginning

from go-junit-report.

I see, this is indeed not detected as a problem currently. It looks like the output you posted does not contain a summary line (example), and I suspect that is why the failure is not captured at the moment. Any idea why this is missing?

I haven't been able to reproduce this myself, can you post a minimal example that produces this output when running go test -v ./...?

from go-junit-report.

Hey, there is summary actually, I guess I just forgot it while was sanitizing the output.
Here is one more example:

=== RUN   Test0
--- PASS: Test0 (0.00s)
PASS
ok  	test/0	0.030s
?   	test/1	[no test files]
?   	test/2	[no test files]
time="2018-04-23T19:01:04+03:00" level=fatal msg="cannot connect to mysql: dial tcp: lookup mysql: no such host"
FAIL	test/3	0.061s
=== RUN   Test1
--- PASS: Test1 (0.00s)
=== RUN   Test2
--- PASS: Test2 (0.00s)
=== RUN   Test3
--- PASS: Test3 (0.00s)
=== RUN   Test4
--- PASS: Test4 (0.00s)
PASS
ok  	test/3	0.060s
?   	test/4	[no test files]
?   	test/5	[no test files]

Also, I will try to prepare the project with the failing tests for you.

from go-junit-report.

@jstemmer here you go: https://github.com/Trane9991/go-junit-report-issue

from go-junit-report.

Just ran into this.
Output with prints from the tests stripped out:

--- PASS: foo (0.86s)
PASS
FAIL	bar	222.957s

go-junit-report reported 0 failures and 0 errors.

The way I understand it is that os.exit(non-zero) was called at the end of the test suite and that caused a failure on the module level (FAIL bar), but there was no failing test, so go-junit-report didn't report failing tests.

I guess a solution would be to say <testsuite errors="1"> ?

Do you have an ETA on the fix ?;]

from go-junit-report.

This has been fixed in v2.0.0-beta1.

from go-junit-report.