Code Monkey home page Code Monkey logo

jtec's Introduction

JTeC: Java Testwise Coverage

CI

JTeC is a tool that enables researchers to obtain testwise coverage for programs that run in the JVM. It targets research in regression testing, such as test selection, prioritization, or flaky test analysis. To analyze tests beyond code coverage, JTeC also supports instrumenting APIs that open external files (e.g., java.io and java.nio), connect to sockets (e.g., java.net.socket), or spawn threads or processes (e.g., java.lang). Notably, JTeC thus doesn't rely on the Java SecurityManager, which is outdated and will be removed in newer JDKs.

Acknowledgments & Contributors

The project is developed as part of the BMBF funded project Q-Soft by CQSE and TUM. JTeC is mainly developed by Raphael Noemmer and Daniel Elsner.

Usage

JTeC is built to be used with Maven Surefire and Failsafe. Therefore, the simplest way to use JTeC in a Maven project is through the JTeC Maven plugin:

<build>
    <plugins>
        <plugin>
            <groupId>edu.tum.sse</groupId>
            <artifactId>jtec-maven-plugin</artifactId>
            <version>0.0.5-SNAPSHOT</version>
            <executions>
                <execution>
                    <goals>
                        <goal>jtec
                        </goal>  <!-- Instrument all JVMs spawned by Maven Surefire/Failsafe that execute tests -->
                        <goal>report
                        </goal>  <!-- Create reports for each Maven project the reactor after test execution -->
                    </goals>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>

Then, execute your tests as you would normally do, by optionally providing arguments to JTeC:

$ mvn clean verify -fn -Djtec.opts="test.trace,sys.trace,cov.trace"

After the command has finished, you'll find a JSON test report in each target/jtec directory, as well as the raw logs created by JTeC. If you want to have a single aggregated test report, e.g., in a Maven multi-module project, simply add the jtec:report-aggregate goal to your command line:

$ mvn clean verify -fn -Djtec.opts="test.trace,sys.trace,cov.trace" jtec:report-aggregate

Advanced JUnit Usage

If you do not want to use JTeC's default instrumentation for JUnit (4/5) test cases, you may pass the option test.instr=false, which prevents bytecode transformation for JUnit classes. Instead, for JUnit5 projects, the service loader mechanism is used to dynamically register a TestExecutionListener . For JUnit4 projects, you can register a JTeC RunListener with Maven in your pom.xml as follows:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId> <!-- same for failsafe -->
    <version>3.0.0-M5</version>
    <configuration>
        <properties>
            <property>
                <name>listener</name>
                <value>edu.tum.sse.jtec.instrumentation.testevent.JUnitTestEventListener</value>
            </property>
        </properties>
    </configuration>
</plugin>

Structure

├── jtec-agent              <- The JTeC agent is a simple Java agent that can be parameterized and attached to a JVM process.
├── jtec-core               <- The JTeC core package contains instrumentation and auxiliary code.
├── jtec-instrumentation    <- The JTeC instrumentation package contains instrumentation code injected via the JTeC agent at runtime.
├── jtec-maven-plugin       <- The JTeC maven plugin provides a utility to attach the agent to the tests of a Maven project.
└── jtec-sample-project     <- A sample project that demonstrates how to use JTeC in a Maven multi-module project.

Setup

To build JTeC simply run:

$ mvn clean install 

This will build the code for all JTeC projects, run all tests, and install the JARs to your local Maven repository.

jtec's People

Contributors

delsner avatar dependabot[bot] avatar raphael-n avatar rollix avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

Forkers

raseshshah

jtec's Issues

[Feature] Add spring test execution listener support

We need to properly implement test execution listener instrumentation and registering (without instrumentation, test.instr=false) for Spring TestExecutionListener.

Registering should be straightforward and similar to the JUnit5 service loader mechanism:

The spring-test module declares all core default listeners under the org.springframework.test.context.TestExecutionListener key in its META-INF/spring.factories properties file. Similarly, we can register our custom listener by using the above key in our own META-INF/spring.factories properties file:

org.springframework.test.context.TestExecutionListener=\
edu.tum.sse.jtec.instrumentation.testevent.SpringTestExecutionListener

[Bugfix] Activating all tracers breaks instrumentation

Calling the jtec maven plugin with -Djtec.opts="test.trace,sys.trace,cov.trace" leads to the following output on macOS (same with cov.instr):

[ERROR] Attaching JTeC agent with args: jtec.out=/workspace/JTeC/jtec-sample-project/junit5-project/target/jtec,test.trace=true,sys.trace=true,cov.trace=true,cov.instr=false,cov.level=CLASS,cov.includes=.*junit5.*,cov.excludes=(sun|java|jdk|com.sun|edu.tum.sse.jtec|net.bytebuddy|org.apache.maven).*
[ERROR] [Byte Buddy] BEFORE_INSTALL net.bytebuddy.shaded.agent.builder.AgentBuilder$Default$ExecutingTransformer$ByteBuddy$ModuleSupport@23941fb4 on sun.instrument.InstrumentationImpl@7486b455
[ERROR] [Byte Buddy] REDEFINE COMPLETE 0 batch(es) containing 0 types [0 failed batch(es)]
[ERROR] [Byte Buddy] INSTALL net.bytebuddy.shaded.agent.builder.AgentBuilder$Default$ExecutingTransformer$ByteBuddy$ModuleSupport@23941fb4 on sun.instrument.InstrumentationImpl@7486b455
[ERROR] [Byte Buddy] BEFORE_INSTALL net.bytebuddy.shaded.agent.builder.AgentBuilder$Default$ExecutingTransformer$ByteBuddy$ModuleSupport@6dd7b5a3 on sun.instrument.InstrumentationImpl@7486b455
[ERROR] [Byte Buddy] REDEFINE BATCH #0 [7 of 7 type(s)]
[ERROR] [Byte Buddy] TRANSFORM java.io.RandomAccessFile [null, module java.base, Thread[main,5,main], loaded=true]
[ERROR] [Byte Buddy] TRANSFORM sun.nio.fs.UnixFileSystemProvider [null, module java.base, Thread[main,5,main], loaded=true]
[ERROR] [Byte Buddy] TRANSFORM java.nio.file.spi.FileSystemProvider [null, module java.base, Thread[main,5,main], loaded=true]
[ERROR] [Byte Buddy] TRANSFORM java.io.FileOutputStream [null, module java.base, Thread[main,5,main], loaded=true]
[ERROR] [Byte Buddy] TRANSFORM java.io.FileInputStream [null, module java.base, Thread[main,5,main], loaded=true]
[ERROR] [Byte Buddy] TRANSFORM java.lang.Thread [null, module java.base, Thread[main,5,main], loaded=true]
[ERROR] [Byte Buddy] TRANSFORM java.lang.ClassLoader [null, module java.base, Thread[main,5,main], loaded=true]
[ERROR] [Byte Buddy] REDEFINE COMPLETE 1 batch(es) containing 7 types [0 failed batch(es)]
[ERROR] [Byte Buddy] INSTALL net.bytebuddy.shaded.agent.builder.AgentBuilder$Default$ExecutingTransformer$ByteBuddy$ModuleSupport@6dd7b5a3 on sun.instrument.InstrumentationImpl@7486b455
[ERROR] [Byte Buddy] ERROR java.nio.channels.Channels$1 [null, module java.base, Thread[main,5,main], loaded=false]
[ERROR] java.lang.ClassCircularityError: java/nio/channels/Channels$1
[ERROR] 	at java.base/java.nio.channels.Channels.newOutputStream(Channels.java:141)
[ERROR] 	at java.base/java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:482)
[ERROR] 	at java.base/java.nio.file.Files.newOutputStream(Files.java:219)
[ERROR] 	at java.base/java.nio.file.Files.write(Files.java:3422)
[ERROR] 	at java.base/java.lang.ClassLoader.getResource(ClassLoader.java:1392)
[ERROR] 	at java.base/java.net.URLClassLoader.getResourceAsStream(URLClassLoader.java:322)
[ERROR] 	at net.bytebuddy.shaded.dynamic.ClassFileLocator$ForClassLoader.locate(ClassFileLocator.java:453)
[ERROR] 	at net.bytebuddy.shaded.dynamic.ClassFileLocator$ForClassLoader.locate(ClassFileLocator.java:434)
[ERROR] 	at net.bytebuddy.shaded.dynamic.ClassFileLocator$Compound.locate(ClassFileLocator.java:1891)
[ERROR] 	at net.bytebuddy.shaded.pool.TypePool$Default.doDescribe(TypePool.java:839)
[ERROR] 	at net.bytebuddy.shaded.pool.TypePool$Default$WithLazyResolution.access$001(TypePool.java:921)
[ERROR] 	at net.bytebuddy.shaded.pool.TypePool$Default$WithLazyResolution.doResolve(TypePool.java:1019)
[ERROR] 	at net.bytebuddy.shaded.pool.TypePool$Default$WithLazyResolution$LazyTypeDescription.delegate(TypePool.java:1088)
[ERROR] 	at net.bytebuddy.shaded.description.type.TypeDescription$AbstractBase$OfSimpleType$WithDelegation.getSuperClass(TypeDescription.java:8351)
[ERROR] 	at net.bytebuddy.shaded.description.type.TypeDescription$Generic$OfNonGenericType.getSuperClass(TypeDescription.java:3602)
[ERROR] 	at net.bytebuddy.shaded.description.type.TypeDefinition$SuperClassIterator.next(TypeDefinition.java:391)
[ERROR] 	at net.bytebuddy.shaded.description.type.TypeDefinition$SuperClassIterator.next(TypeDefinition.java:357)
[ERROR] 	at net.bytebuddy.shaded.matcher.HasSuperTypeMatcher.doMatch(HasSuperTypeMatcher.java:54)
[ERROR] 	at net.bytebuddy.shaded.matcher.HasSuperTypeMatcher.doMatch(HasSuperTypeMatcher.java:32)
[ERROR] 	at net.bytebuddy.shaded.matcher.ElementMatcher$Junction$ForNonNullValues.matches(ElementMatcher.java:249)
[ERROR] 	at net.bytebuddy.shaded.matcher.ElementMatcher$Junction$Disjunction.matches(ElementMatcher.java:214)
[ERROR] 	at net.bytebuddy.shaded.agent.builder.AgentBuilder$RawMatcher$ForElementMatchers.matches(AgentBuilder.java:1833)
[ERROR] 	at net.bytebuddy.shaded.agent.builder.AgentBuilder$Default$ExecutingTransformer.doTransform(AgentBuilder.java:11870)
[ERROR] 	at net.bytebuddy.shaded.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:11828)
[ERROR] 	at net.bytebuddy.shaded.agent.builder.AgentBuilder$Default$ExecutingTransformer.access$1700(AgentBuilder.java:11545)
[ERROR] 	at net.bytebuddy.shaded.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:12308)
[ERROR] 	at net.bytebuddy.shaded.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:12240)
[ERROR] 	at java.base/java.security.AccessController.doPrivileged(Native Method)
[ERROR] 	at net.bytebuddy.shaded.agent.builder.AgentBuilder$Default$ExecutingTransformer.doPrivileged(AgentBuilder.java)
[ERROR] 	at net.bytebuddy.shaded.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:11771)
[ERROR] 	at net.bytebuddy.shaded.agent.builder.AgentBuilder$Default$ExecutingTransformer$ByteBuddy$ModuleSupport.transform(Unknown Source)
[ERROR] 	at java.instrument/sun.instrument.TransformerManager.transform(TransformerManager.java:188)
[ERROR] 	at java.instrument/sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:563)
[ERROR] 	at java.base/java.nio.channels.Channels.newOutputStream(Channels.java:141)
[ERROR] 	at java.base/java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:482)
[ERROR] 	at java.base/java.nio.file.Files.newOutputStream(Files.java:219)
[ERROR] 	at java.base/java.nio.file.Files.write(Files.java:3422)
[ERROR] 	at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:209)
[ERROR] 	at java.base/java.nio.file.Files.newByteChannel(Files.java:370)
[ERROR] 	at java.base/java.nio.file.Files.createFile(Files.java:647)
[ERROR] 	at edu.tum.sse.jtec.util.IOUtils.createFileAndEnclosingDir(IOUtils.java:20)
[ERROR] 	at edu.tum.sse.jtec.agent.Tracer.<init>(Tracer.java:39)
[ERROR] 	at edu.tum.sse.jtec.agent.JTeCAgent.premain(JTeCAgent.java:13)
[ERROR] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[ERROR] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[ERROR] 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[ERROR] 	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
[ERROR] 	at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:513)
[ERROR] 	at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:525)
[ERROR] [Byte Buddy] ERROR java.nio.channels.SelectableChannel [null, module java.base, Thread[main,5,main], loaded=false]
[ERROR] java.lang.ClassCircularityError: java/nio/channels/SelectableChannel
[ERROR] 	at java.base/java.nio.channels.Channels.writeFully(Channels.java:89)
[ERROR] 	at java.base/java.nio.channels.Channels$1.write(Channels.java:172)
[ERROR] 	at java.base/java.nio.file.Files.write(Files.java:3427)
[ERROR] 	at java.base/java.lang.ClassLoader.getResource(ClassLoader.java:1392)
[ERROR] 	at java.base/java.net.URLClassLoader.getResourceAsStream(URLClassLoader.java:322)
[ERROR] 	at net.bytebuddy.shaded.dynamic.ClassFileLocator$ForClassLoader.locate(ClassFileLocator.java:453)
[ERROR] 	at net.bytebuddy.shaded.dynamic.ClassFileLocator$ForClassLoader.locate(ClassFileLocator.java:434)
[ERROR] 	at net.bytebuddy.shaded.dynamic.ClassFileLocator$Compound.locate(ClassFileLocator.java:1891)
[ERROR] 	at net.bytebuddy.shaded.pool.TypePool$Default.doDescribe(TypePool.java:839)
[ERROR] 	at net.bytebuddy.shaded.pool.TypePool$Default$WithLazyResolution.access$001(TypePool.java:921)
[ERROR] 	at net.bytebuddy.shaded.pool.TypePool$Default$WithLazyResolution.doResolve(TypePool.java:1019)
[ERROR] 	at net.bytebuddy.shaded.pool.TypePool$Default$WithLazyResolution$LazyTypeDescription.delegate(TypePool.java:1088)
[ERROR] 	at net.bytebuddy.shaded.description.type.TypeDescription$AbstractBase$OfSimpleType$WithDelegation.getInterfaces(TypeDescription.java:8358)
[ERROR] 	at net.bytebuddy.shaded.description.type.TypeDescription$Generic$OfNonGenericType.getInterfaces(TypeDescription.java:3619)
[ERROR] 	at net.bytebuddy.shaded.matcher.HasSuperTypeMatcher.doMatch(HasSuperTypeMatcher.java:67)
[ERROR] 	at net.bytebuddy.shaded.matcher.HasSuperTypeMatcher.doMatch(HasSuperTypeMatcher.java:32)
[ERROR] 	at net.bytebuddy.shaded.matcher.ElementMatcher$Junction$ForNonNullValues.matches(ElementMatcher.java:249)
[ERROR] 	at net.bytebuddy.shaded.matcher.ElementMatcher$Junction$Disjunction.matches(ElementMatcher.java:214)
[ERROR] 	at net.bytebuddy.shaded.agent.builder.AgentBuilder$RawMatcher$ForElementMatchers.matches(AgentBuilder.java:1833)
[ERROR] 	at net.bytebuddy.shaded.agent.builder.AgentBuilder$Default$ExecutingTransformer.doTransform(AgentBuilder.java:11870)
[ERROR] 	at net.bytebuddy.shaded.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:11828)
[ERROR] 	at net.bytebuddy.shaded.agent.builder.AgentBuilder$Default$ExecutingTransformer.access$1700(AgentBuilder.java:11545)
[ERROR] 	at net.bytebuddy.shaded.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:12308)
[ERROR] 	at net.bytebuddy.shaded.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:12240)
[ERROR] 	at java.base/java.security.AccessController.doPrivileged(Native Method)
[ERROR] 	at net.bytebuddy.shaded.agent.builder.AgentBuilder$Default$ExecutingTransformer.doPrivileged(AgentBuilder.java)
[ERROR] 	at net.bytebuddy.shaded.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:11771)
[ERROR] 	at net.bytebuddy.shaded.agent.builder.AgentBuilder$Default$ExecutingTransformer$ByteBuddy$ModuleSupport.transform(Unknown Source)
[ERROR] 	at java.instrument/sun.instrument.TransformerManager.transform(TransformerManager.java:188)
[ERROR] 	at java.instrument/sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:563)
[ERROR] 	at java.base/java.nio.channels.Channels.writeFully(Channels.java:89)
[ERROR] 	at java.base/java.nio.channels.Channels$1.write(Channels.java:172)
[ERROR] 	at java.base/java.nio.file.Files.write(Files.java:3427)
[ERROR] 	at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:209)
[ERROR] 	at java.base/java.nio.file.Files.newByteChannel(Files.java:370)
[ERROR] 	at java.base/java.nio.file.Files.createFile(Files.java:647)
[ERROR] 	at edu.tum.sse.jtec.util.IOUtils.createFileAndEnclosingDir(IOUtils.java:20)
[ERROR] 	at edu.tum.sse.jtec.agent.Tracer.<init>(Tracer.java:39)
[ERROR] 	at edu.tum.sse.jtec.agent.JTeCAgent.premain(JTeCAgent.java:13)
[ERROR] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[ERROR] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[ERROR] 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[ERROR] 	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
[ERROR] 	at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:513)
[ERROR] 	at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:525)
[ERROR] Exception, printedName is: jdk/internal/misc/TerminatingThreadLocal.class
[ERROR] java.lang.NullPointerException
[ERROR] 	at java.base/sun.nio.ch.Util.getTemporaryDirectBuffer(Util.java:230)
[ERROR] 	at java.base/sun.nio.ch.IOUtil.write(IOUtil.java:71)
[ERROR] 	at java.base/sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:280)
[ERROR] 	at java.base/java.nio.channels.Channels.writeFullyImpl(Channels.java:74)
[ERROR] 	at java.base/java.nio.channels.Channels.writeFully(Channels.java:97)
[ERROR] 	at java.base/java.nio.channels.Channels$1.write(Channels.java:172)
[ERROR] 	at java.base/java.nio.file.Files.write(Files.java:3427)
[ERROR] 	at java.base/java.lang.ClassLoader.getResource(ClassLoader.java:1392)
[ERROR] 	at java.base/java.net.URLClassLoader.getResourceAsStream(URLClassLoader.java:322)
[ERROR] 	at net.bytebuddy.shaded.dynamic.ClassFileLocator$ForClassLoader.locate(ClassFileLocator.java:453)
[ERROR] 	at net.bytebuddy.shaded.dynamic.ClassFileLocator$ForClassLoader.locate(ClassFileLocator.java:434)
[ERROR] 	at net.bytebuddy.shaded.dynamic.ClassFileLocator$Compound.locate(ClassFileLocator.java:1891)
[ERROR] 	at net.bytebuddy.shaded.pool.TypePool$Default.doDescribe(TypePool.java:839)
[ERROR] 	at net.bytebuddy.shaded.pool.TypePool$Default$WithLazyResolution.access$001(TypePool.java:921)
[ERROR] 	at net.bytebuddy.shaded.pool.TypePool$Default$WithLazyResolution.doResolve(TypePool.java:1019)
[ERROR] 	at net.bytebuddy.shaded.pool.TypePool$Default$WithLazyResolution$LazyTypeDescription.delegate(TypePool.java:1088)
[ERROR] 	at net.bytebuddy.shaded.description.type.TypeDescription$AbstractBase$OfSimpleType$WithDelegation.getSuperClass(TypeDescription.java:8351)
[ERROR] 	at net.bytebuddy.shaded.description.type.TypeDescription$Generic$OfParameterizedType.getSuperClass(TypeDescription.java:4917)
[ERROR] 	at net.bytebuddy.shaded.description.type.TypeDescription$Generic$LazyProjection$WithEagerNavigation.getSuperClass(TypeDescription.java:6495)
[ERROR] 	at net.bytebuddy.shaded.description.type.TypeDefinition$SuperClassIterator.next(TypeDefinition.java:391)
[ERROR] 	at net.bytebuddy.shaded.description.type.TypeDefinition$SuperClassIterator.next(TypeDefinition.java:357)
[ERROR] 	at net.bytebuddy.shaded.matcher.HasSuperTypeMatcher.doMatch(HasSuperTypeMatcher.java:54)
[ERROR] 	at net.bytebuddy.shaded.matcher.HasSuperTypeMatcher.doMatch(HasSuperTypeMatcher.java:32)
[ERROR] 	at net.bytebuddy.shaded.matcher.ElementMatcher$Junction$ForNonNullValues.matches(ElementMatcher.java:249)
[ERROR] 	at net.bytebuddy.shaded.matcher.ElementMatcher$Junction$Disjunction.matches(ElementMatcher.java:214)
[ERROR] 	at net.bytebuddy.shaded.agent.builder.AgentBuilder$RawMatcher$ForElementMatchers.matches(AgentBuilder.java:1833)
[ERROR] 	at net.bytebuddy.shaded.agent.builder.AgentBuilder$Default$ExecutingTransformer.doTransform(AgentBuilder.java:11870)
[ERROR] 	at net.bytebuddy.shaded.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:11828)
[ERROR] 	at net.bytebuddy.shaded.agent.builder.AgentBuilder$Default$ExecutingTransformer.access$1700(AgentBuilder.java:11545)
[ERROR] 	at net.bytebuddy.shaded.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:12308)
[ERROR] 	at net.bytebuddy.shaded.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:12240)
[ERROR] 	at java.base/java.security.AccessController.doPrivileged(Native Method)
[ERROR] 	at net.bytebuddy.shaded.agent.builder.AgentBuilder$Default$ExecutingTransformer.doPrivileged(AgentBuilder.java)
[ERROR] 	at net.bytebuddy.shaded.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:11771)
[ERROR] 	at net.bytebuddy.shaded.agent.builder.AgentBuilder$Default$ExecutingTransformer$ByteBuddy$ModuleSupport.transform(Unknown Source)
[ERROR] 	at java.instrument/sun.instrument.TransformerManager.transform(TransformerManager.java:188)
[ERROR] 	at java.instrument/sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:563)
[ERROR] 	at java.base/sun.nio.ch.Util.<clinit>(Util.java:55)
[ERROR] 	at java.base/sun.nio.ch.IOUtil.write(IOUtil.java:71)
[ERROR] 	at java.base/sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:280)
[ERROR] 	at java.base/java.nio.channels.Channels.writeFullyImpl(Channels.java:74)
[ERROR] 	at java.base/java.nio.channels.Channels.writeFully(Channels.java:97)
[ERROR] 	at java.base/java.nio.channels.Channels$1.write(Channels.java:172)
[ERROR] 	at java.base/java.nio.file.Files.write(Files.java:3427)
[ERROR] 	at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:209)
[ERROR] 	at java.base/java.nio.file.Files.newByteChannel(Files.java:370)
[ERROR] 	at java.base/java.nio.file.Files.createFile(Files.java:647)
[ERROR] 	at edu.tum.sse.jtec.util.IOUtils.createFileAndEnclosingDir(IOUtils.java:20)
[ERROR] 	at edu.tum.sse.jtec.agent.Tracer.<init>(Tracer.java:39)
[ERROR] 	at edu.tum.sse.jtec.agent.JTeCAgent.premain(JTeCAgent.java:13)
[ERROR] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[ERROR] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[ERROR] 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[ERROR] 	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
[ERROR] 	at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:513)
[ERROR] 	at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:525)
[ERROR] Exception, printedName is: java/lang/ThreadLocal.class
[ERROR] java.lang.NullPointerException
[ERROR] 	at java.base/sun.nio.ch.Util.getTemporaryDirectBuffer(Util.java:230)
[ERROR] 	at java.base/sun.nio.ch.IOUtil.write(IOUtil.java:71)
[ERROR] 	at java.base/sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:280)
[ERROR] 	at java.base/java.nio.channels.Channels.writeFullyImpl(Channels.java:74)
[ERROR] 	at java.base/java.nio.channels.Channels.writeFully(Channels.java:97)
[ERROR] 	at java.base/java.nio.channels.Channels$1.write(Channels.java:172)
[ERROR] 	at java.base/java.nio.file.Files.write(Files.java:3427)
[ERROR] 	at java.base/java.lang.ClassLoader.getResource(ClassLoader.java:1392)
[ERROR] 	at java.base/java.net.URLClassLoader.getResourceAsStream(URLClassLoader.java:322)
[ERROR] 	at net.bytebuddy.shaded.dynamic.ClassFileLocator$ForClassLoader.locate(ClassFileLocator.java:453)
[ERROR] 	at net.bytebuddy.shaded.dynamic.ClassFileLocator$ForClassLoader.locate(ClassFileLocator.java:434)
[ERROR] 	at net.bytebuddy.shaded.dynamic.ClassFileLocator$Compound.locate(ClassFileLocator.java:1891)
[ERROR] 	at net.bytebuddy.shaded.pool.TypePool$Default.doDescribe(TypePool.java:839)
[ERROR] 	at net.bytebuddy.shaded.pool.TypePool$Default$WithLazyResolution.access$001(TypePool.java:921)
[ERROR] 	at net.bytebuddy.shaded.pool.TypePool$Default$WithLazyResolution.doResolve(TypePool.java:1019)
[ERROR] 	at net.bytebuddy.shaded.pool.TypePool$Default$WithLazyResolution$LazyTypeDescription.delegate(TypePool.java:1088)
[ERROR] 	at net.bytebuddy.shaded.description.type.TypeDescription$AbstractBase$OfSimpleType$WithDelegation.getEnclosingType(TypeDescription.java:8396)
[ERROR] 	at net.bytebuddy.shaded.pool.TypePool$Default$LazyTypeDescription$GenericTypeToken$ForParameterizedType$LazyParameterizedType.getOwnerType(TypePool.java:5409)
[ERROR] 	at net.bytebuddy.shaded.description.type.TypeDescription$Generic$Visitor$Substitutor.onParameterizedType(TypeDescription.java:1876)
[ERROR] 	at net.bytebuddy.shaded.description.type.TypeDescription$Generic$Visitor$Substitutor$WithoutTypeSubstitution.onParameterizedType(TypeDescription.java:1925)
[ERROR] 	at net.bytebuddy.shaded.description.type.TypeDescription$Generic$OfParameterizedType.accept(TypeDescription.java:5057)
[ERROR] 	at net.bytebuddy.shaded.description.type.TypeDescription$Generic$LazyProjection.accept(TypeDescription.java:6220)
[ERROR] 	at net.bytebuddy.shaded.description.type.TypeDescription$Generic$LazyProjection$WithResolvedErasure.resolve(TypeDescription.java:6872)
[ERROR] 	at net.bytebuddy.shaded.description.type.TypeDescription$Generic$LazyProjection$WithEagerNavigation.getSuperClass(TypeDescription.java:6495)
[ERROR] 	at net.bytebuddy.shaded.description.type.TypeDefinition$SuperClassIterator.next(TypeDefinition.java:391)
[ERROR] 	at net.bytebuddy.shaded.description.type.TypeDefinition$SuperClassIterator.next(TypeDefinition.java:357)
[ERROR] 	at net.bytebuddy.shaded.matcher.HasSuperTypeMatcher.doMatch(HasSuperTypeMatcher.java:54)
[ERROR] 	at net.bytebuddy.shaded.matcher.HasSuperTypeMatcher.doMatch(HasSuperTypeMatcher.java:32)
[ERROR] 	at net.bytebuddy.shaded.matcher.ElementMatcher$Junction$ForNonNullValues.matches(ElementMatcher.java:249)
[ERROR] 	at net.bytebuddy.shaded.matcher.ElementMatcher$Junction$Disjunction.matches(ElementMatcher.java:214)
[ERROR] 	at net.bytebuddy.shaded.agent.builder.AgentBuilder$RawMatcher$ForElementMatchers.matches(AgentBuilder.java:1833)
[ERROR] 	at net.bytebuddy.shaded.agent.builder.AgentBuilder$Default$ExecutingTransformer.doTransform(AgentBuilder.java:11870)
[ERROR] 	at net.bytebuddy.shaded.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:11828)
[ERROR] 	at net.bytebuddy.shaded.agent.builder.AgentBuilder$Default$ExecutingTransformer.access$1700(AgentBuilder.java:11545)
[ERROR] 	at net.bytebuddy.shaded.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:12308)
[ERROR] 	at net.bytebuddy.shaded.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:12240)
[ERROR] 	at java.base/java.security.AccessController.doPrivileged(Native Method)
[ERROR] 	at net.bytebuddy.shaded.agent.builder.AgentBuilder$Default$ExecutingTransformer.doPrivileged(AgentBuilder.java)
[ERROR] 	at net.bytebuddy.shaded.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:11771)
[ERROR] 	at net.bytebuddy.shaded.agent.builder.AgentBuilder$Default$ExecutingTransformer$ByteBuddy$ModuleSupport.transform(Unknown Source)
[ERROR] 	at java.instrument/sun.instrument.TransformerManager.transform(TransformerManager.java:188)
[ERROR] 	at java.instrument/sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:563)
[ERROR] 	at java.base/sun.nio.ch.Util.<clinit>(Util.java:55)
[ERROR] 	at java.base/sun.nio.ch.IOUtil.write(IOUtil.java:71)
[ERROR] 	at java.base/sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:280)
[ERROR] 	at java.base/java.nio.channels.Channels.writeFullyImpl(Channels.java:74)
[ERROR] 	at java.base/java.nio.channels.Channels.writeFully(Channels.java:97)
[ERROR] 	at java.base/java.nio.channels.Channels$1.write(Channels.java:172)
[ERROR] 	at java.base/java.nio.file.Files.write(Files.java:3427)
[ERROR] 	at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:209)
[ERROR] 	at java.base/java.nio.file.Files.newByteChannel(Files.java:370)
[ERROR] 	at java.base/java.nio.file.Files.createFile(Files.java:647)
[ERROR] 	at edu.tum.sse.jtec.util.IOUtils.createFileAndEnclosingDir(IOUtils.java:20)
[ERROR] 	at edu.tum.sse.jtec.agent.Tracer.<init>(Tracer.java:39)
[ERROR] 	at edu.tum.sse.jtec.agent.JTeCAgent.premain(JTeCAgent.java:13)
[ERROR] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[ERROR] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[ERROR] 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[ERROR] 	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
[ERROR] 	at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:513)
[ERROR] 	at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:525)
[ERROR] [Byte Buddy] ERROR java.nio.DirectByteBuffer$Deallocator [null, module java.base, Thread[main,5,main], loaded=false]
[ERROR] java.lang.ClassCircularityError: java/nio/DirectByteBuffer$Deallocator
[ERROR] 	at java.base/java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:134)
[ERROR] 	at java.base/java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:317)
[ERROR] 	at java.base/sun.nio.ch.Util.getTemporaryDirectBuffer(Util.java:242)
[ERROR] 	at java.base/sun.nio.ch.IOUtil.write(IOUtil.java:71)
[ERROR] 	at java.base/sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:280)
[ERROR] 	at java.base/java.nio.channels.Channels.writeFullyImpl(Channels.java:74)
[ERROR] 	at java.base/java.nio.channels.Channels.writeFully(Channels.java:97)
[ERROR] 	at java.base/java.nio.channels.Channels$1.write(Channels.java:172)
[ERROR] 	at java.base/java.nio.file.Files.write(Files.java:3427)
[ERROR] 	at java.base/java.lang.ClassLoader.getResource(ClassLoader.java:1392)
[ERROR] 	at java.base/java.net.URLClassLoader.getResourceAsStream(URLClassLoader.java:322)
[ERROR] 	at net.bytebuddy.shaded.dynamic.ClassFileLocator$ForClassLoader.locate(ClassFileLocator.java:453)
[ERROR] 	at net.bytebuddy.shaded.dynamic.ClassFileLocator$ForClassLoader.locate(ClassFileLocator.java:434)
[ERROR] 	at net.bytebuddy.shaded.dynamic.ClassFileLocator$Compound.locate(ClassFileLocator.java:1891)
[ERROR] 	at net.bytebuddy.shaded.pool.TypePool$Default.doDescribe(TypePool.java:839)
[ERROR] 	at net.bytebuddy.shaded.pool.TypePool$Default$WithLazyResolution.access$001(TypePool.java:921)
[ERROR] 	at net.bytebuddy.shaded.pool.TypePool$Default$WithLazyResolution.doResolve(TypePool.java:1019)
[ERROR] 	at net.bytebuddy.shaded.pool.TypePool$Default$WithLazyResolution$LazyTypeDescription.delegate(TypePool.java:1088)
[ERROR] 	at net.bytebuddy.shaded.description.type.TypeDescription$AbstractBase$OfSimpleType$WithDelegation.getInterfaces(TypeDescription.java:8358)
[ERROR] 	at net.bytebuddy.shaded.description.type.TypeDescription$Generic$OfNonGenericType.getInterfaces(TypeDescription.java:3619)
[ERROR] 	at net.bytebuddy.shaded.matcher.HasSuperTypeMatcher.doMatch(HasSuperTypeMatcher.java:67)
[ERROR] 	at net.bytebuddy.shaded.matcher.HasSuperTypeMatcher.doMatch(HasSuperTypeMatcher.java:32)
[ERROR] 	at net.bytebuddy.shaded.matcher.ElementMatcher$Junction$ForNonNullValues.matches(ElementMatcher.java:249)
[ERROR] 	at net.bytebuddy.shaded.matcher.ElementMatcher$Junction$Disjunction.matches(ElementMatcher.java:214)
[ERROR] 	at net.bytebuddy.shaded.agent.builder.AgentBuilder$RawMatcher$ForElementMatchers.matches(AgentBuilder.java:1833)
[ERROR] 	at net.bytebuddy.shaded.agent.builder.AgentBuilder$Default$ExecutingTransformer.doTransform(AgentBuilder.java:11870)
[ERROR] 	at net.bytebuddy.shaded.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:11828)
[ERROR] 	at net.bytebuddy.shaded.agent.builder.AgentBuilder$Default$ExecutingTransformer.access$1700(AgentBuilder.java:11545)
[ERROR] 	at net.bytebuddy.shaded.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:12308)
[ERROR] 	at net.bytebuddy.shaded.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:12240)
[ERROR] 	at java.base/java.security.AccessController.doPrivileged(Native Method)
[ERROR] 	at net.bytebuddy.shaded.agent.builder.AgentBuilder$Default$ExecutingTransformer.doPrivileged(AgentBuilder.java)
[ERROR] 	at net.bytebuddy.shaded.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:11771)
[ERROR] 	at net.bytebuddy.shaded.agent.builder.AgentBuilder$Default$ExecutingTransformer$ByteBuddy$ModuleSupport.transform(Unknown Source)
[ERROR] 	at java.instrument/sun.instrument.TransformerManager.transform(TransformerManager.java:188)
[ERROR] 	at java.instrument/sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:563)
[ERROR] 	at java.base/java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:134)
[ERROR] 	at java.base/java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:317)
[ERROR] 	at java.base/sun.nio.ch.Util.getTemporaryDirectBuffer(Util.java:242)
[ERROR] 	at java.base/sun.nio.ch.IOUtil.write(IOUtil.java:71)
[ERROR] 	at java.base/sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:280)
[ERROR] 	at java.base/java.nio.channels.Channels.writeFullyImpl(Channels.java:74)
[ERROR] 	at java.base/java.nio.channels.Channels.writeFully(Channels.java:97)
[ERROR] 	at java.base/java.nio.channels.Channels$1.write(Channels.java:172)
[ERROR] 	at java.base/java.nio.file.Files.write(Files.java:3427)
[ERROR] 	at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:209)
[ERROR] 	at java.base/java.nio.file.Files.newByteChannel(Files.java:370)
[ERROR] 	at java.base/java.nio.file.Files.createFile(Files.java:647)
[ERROR] 	at edu.tum.sse.jtec.util.IOUtils.createFileAndEnclosingDir(IOUtils.java:20)
[ERROR] 	at edu.tum.sse.jtec.agent.Tracer.<init>(Tracer.java:39)
[ERROR] 	at edu.tum.sse.jtec.agent.JTeCAgent.premain(JTeCAgent.java:13)
[ERROR] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[ERROR] 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[ERROR] 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[ERROR] 	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
[ERROR] 	at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:513)
[ERROR] 	at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:525)

[Feature] Add coverage instrumentation (class- + method-level)

To collect per-test coverage, we need to be able to either monitor loaded class files (for forked test execution, where 1 JVM == 1 test suite) or instrument class files to track constructor/method calls.

There should be class and method level as two possible instrumentation methods, where method will naturally only work if instrumenting class files.

[Chore] Allow beforeTest hook that blocks until IPC (e.g., file sync) signals test continuation

Currently, the pre-test hook will be executed as a background process and the JVM process continues running without interruption. There might be cases where we want to also sync the pre-test hook with the JVM process, to wait until the pre-test hook has been completely executed (e.g., if attaching with instrumentation tool to JVM process).

The simplest way to achieve synchronization between the JVM process and the pre-test hook process would be through file synchronization, where the JVM process waits for a file to be available (by polling) and the pre-test hook process creates the file upon successful initialization.

[Feature] Add Maven report Mojo for single test report

The report should be in JSON format and roughly look as:

{
  "id": "test-report1",
  "timestamp": "...",
  "suites": [
    {
      "id": "FooTest",
      "result": "FAILED",
      "timestamp": "...",
      "openedFiles": ["/foo.txt"],
      "connectedSockets": ["localhost:1234"],
      "spawnedThreads": ["main"],
      "spawnedProcesses": ["123"],
      "coverage": ["Foo:bar()"],
    }
  ]
}

[Feature] Add configuration option for dumping after Tests or Test Suites

Currently, the only time when a coverage dump is created is when the JVM shuts down.
This makes sense for test suites that have setup that is used for multiple tests or test suites as we would miss this setup code for later tests if we dumped after every test.

However, for test suites that don't have a global setup, this unnecessarily reduces the granularity at which we gather testwise coverage. An option to configure dumps after Tests or Test suites would allow for more fine grained testwise coverage where possible.

[Feature] Automatically generate `includes` regex

Grep filepaths from compiled .class files in Maven target directory to find package names.
Add Maven plugin option -Djtec.autoExtractInclude (or similar) than automatically generates the overarching includes regex.

[Bug] Missing test suite start events in non-forking mode

If in non-forking mode, the test suite start event is missing in the *_test.log.
We need to either fix the instrumentation or the TestEventInterceptorUtility to handle the case when a new test suite name appears without a prior test suite start event.

image

Incorrect result with junit parallel thread execution

It won't work with below property which allow the junit test cases to run in fork join thread models. Mostly, needs to use Inherit thread local and capture the test name with each test start.

junit.jupiter.execution.parallel.enabled = true
junit.jupiter.execution.parallel.config.fixed.parallelism = 8

[Chore] (Too) extensive JUnit instrumentation

It seems as if the junit instrumentation is a bit too coarse grained:

[ERROR] [Byte Buddy] INSTALL net.bytebuddy.shaded.agent.builder.AgentBuilder$Default$ExecutingTransformer@2f4948e4 on sun.instrument.InstrumentationImpl@dc24521
[ERROR] [Byte Buddy] TRANSFORM org.junit.platform.launcher.TestExecutionListener [sun.misc.Launcher$AppClassLoader@18b4aac2, null, Thread[main,5,main], loaded=false]
[ERROR] [Byte Buddy] TRANSFORM org.apache.maven.surefire.junitplatform.RunListenerAdapter [sun.misc.Launcher$AppClassLoader@18b4aac2, null, Thread[main,5,main], loaded=false]
[ERROR] [Byte Buddy] TRANSFORM org.junit.platform.launcher.core.TestExecutionListenerRegistry$EagerTestExecutionListener [sun.misc.Launcher$AppClassLoader@18b4aac2, null, Thread[main,5,main], loaded=false]
[ERROR] [Byte Buddy] TRANSFORM org.junit.platform.launcher.core.TestExecutionListenerRegistry$CompositeTestExecutionListener [sun.misc.Launcher$AppClassLoader@18b4aac2, null, Thread[main,5,main], loaded=false]
[ERROR] [Byte Buddy] TRANSFORM org.junit.platform.launcher.core.StreamInterceptingTestExecutionListener [sun.misc.Launcher$AppClassLoader@18b4aac2, null, Thread[main,5,main], loaded=false]

Maybe we can further narrow this down to the most relevant classes.

[Feature] Distributed code coverage

For future use cases in distributed systems, JTeC should support collecting distributed coverage using OpenTelemetry.

The extension point is in jtec-core in the class edu.tum.sse.jtec.instrumentation.coverage.CoverageInstrumentation:

    private AgentBuilder.Transformer getCoverageTransformer() {
        // TODO: We can extend JTeC here in the future, e.g., to support distributed coverage,
        //  by simply creating a transformer with different parameters here.
        return CoverageTransformer.create(coverageLevel);
    }

    private CoverageProbeFactory createCoverageProbeFactory() {
        // TODO: We can extend JTeC here in the future, e.g., to support distributed coverage,
        //  by simply adding new coverage probe factories.
        return new ProcessCoverageProbeFactory();
    }

[Bug] Method-level Coverage Tracing misses Lambda Functions

This is due to the fact that ByteBuddy (and the JVM in general) does provide good capabilities to instrument lambdas (see here).

Example:

import java.util.ArrayList;
import java.util.function.Consumer;

public class App {
    
    private static Consumer<Integer> method = (n) -> { System.out.println(n); };

    public static void main(String[] args) {
        ArrayList<Integer> numbers = new ArrayList<Integer>();
        numbers.add(5);
        numbers.forEach( method );
    }
}
$ javac App.java
$ java -javaagent:~/.m2/repository/edu/tum/sse/jtec-agent/0.0.3/jtec-agent-0.0.3.jar=cov.trace=true,cov.level=method App
$ cat 71150_1671108967249_cov.log
{"71150":["App#main([Ljava.lang.String;)void"]}

[Bug] Test suite finished event occurs after each test in JUnit5 project

Sample test log:

1655627234951 18776 SUITE_STARTED
1655627234993 18776 TEST_STARTED some.TestSuite test_abc
1655627235162 18776 TEST_FINISHED some.TestSuite test_abc
1655627235163 18776 SUITE_FINISHED some.TestSuite 1 0 0
1655627235167 18776 TEST_STARTED some.TestSuite test_def
1655627235173 18776 TEST_FINISHED some.TestSuite test_def
1655627235174 18776 SUITE_FINISHED some.TestSuite 2 0 0

Seen on JDK 11.0.7 (AdoptOpenJDK, Windows Server 2019) with JUnit junit-jupiter-api 5.8.2 and junit-platform-engine 1.8.2

Instrumentation log:

[ERROR] [Byte Buddy] TRANSFORM org.junit.platform.launcher.core.CompositeTestExecutionListener [jdk.internal.loader.ClassLoaders$AppClassLoader@2f333739, unnamed module @2b76ff4e, Thread[main,5,main], loaded=false]
[ERROR] [Byte Buddy] TRANSFORM org.junit.platform.launcher.TestExecutionListener [jdk.internal.loader.ClassLoaders$AppClassLoader@2f333739, unnamed module @2b76ff4e, Thread[main,5,main], loaded=false]
[ERROR] [Byte Buddy] TRANSFORM org.junit.platform.launcher.listeners.UniqueIdTrackingListener [jdk.internal.loader.ClassLoaders$AppClassLoader@2f333739, unnamed module @2b76ff4e, Thread[main,5,main], loaded=false]
[ERROR] [Byte Buddy] TRANSFORM org.junit.platform.launcher.core.CompositeTestExecutionListener [jdk.internal.loader.ClassLoaders$AppClassLoader@2f333739, unnamed module @7b420819, Thread[main,5,main], loaded=false]
[ERROR] [Byte Buddy] TRANSFORM org.junit.platform.launcher.TestExecutionListener [jdk.internal.loader.ClassLoaders$AppClassLoader@2f333739, unnamed module @7b420819, Thread[main,5,main], loaded=false]
[ERROR] [Byte Buddy] TRANSFORM org.junit.platform.launcher.listeners.UniqueIdTrackingListener [jdk.internal.loader.ClassLoaders$AppClassLoader@2f333739, unnamed module @7b420819, Thread[main,5,main], loaded=false]
[ERROR] [Byte Buddy] TRANSFORM org.apache.maven.surefire.junitplatform.RunListenerAdapter [jdk.internal.loader.ClassLoaders$AppClassLoader@2f333739, unnamed module @2b76ff4e, Thread[main,5,main], loaded=false]
[ERROR] [Byte Buddy] TRANSFORM org.apache.maven.surefire.junitplatform.RunListenerAdapter [jdk.internal.loader.ClassLoaders$AppClassLoader@2f333739, unnamed module @7b420819, Thread[main,5,main], loaded=false]
[ERROR] [Byte Buddy] TRANSFORM org.junit.platform.launcher.core.CompositeTestExecutionListener$EagerTestExecutionListener [jdk.internal.loader.ClassLoaders$AppClassLoader@2f333739, unnamed module @2b76ff4e, Thread[main,5,main], loaded=false]
[ERROR] [Byte Buddy] TRANSFORM org.junit.platform.launcher.core.StreamInterceptingTestExecutionListener [jdk.internal.loader.ClassLoaders$AppClassLoader@2f333739, unnamed module @2b76ff4e, Thread[main,5,main], loaded=false]
[ERROR] [Byte Buddy] TRANSFORM org.junit.platform.launcher.core.CompositeTestExecutionListener$EagerTestExecutionListener [jdk.internal.loader.ClassLoaders$AppClassLoader@2f333739, unnamed module @7b420819, Thread[main,5,main], loaded=false]
[ERROR] [Byte Buddy] TRANSFORM org.junit.platform.launcher.core.StreamInterceptingTestExecutionListener [jdk.internal.loader.ClassLoaders$AppClassLoader@2f333739, unnamed module @7b420819, Thread[main,5,main], loaded=false]

[Bug] Instrumentation breaks test suites that use inheritance

When instrumenting with sys.trace,test.trace, the test instrumentation breaks test cases that inherit from an (abstract) super test class:

[ERROR] [Byte Buddy] ERROR some.TestSuiteImpl [jdk.internal.loader.ClassLoaders$AppClassLoader@2f333739, unnamed module @535779e4, Thread[main,5,main], loaded=false]
[ERROR] net.bytebuddy.shaded.pool.TypePool$Resolution$NoSuchTypeException: Cannot resolve type description for some.AbstractTestSuite

[Bugfix] Wrong number of test cases in SUITE_FINISHED event

There seems to be an issue with the number of passed/failed/ignored tests per test suite. They are +1 of what they should be. Maybe this is due to (1) an issue with multiple occurrences of SUITE_FINISHED events or (2) wrong initialization (or increment) of the counters.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.