pragmatists / junitparams Goto Github PK

When many test cases are run, the JUnitParamRunner degrades greatly compared with the default Paremeterized Junit 4 runner. To see the problem create an empty test that takes a string argument and generate enough data to cause the problem. On my old laptop, 1000 test cases take around 1 minutes versus a few seconds on the regular Junit 4 runner. A faster laptop, might need to test with 2k to 4k test cases. A quick profiling of the problem points towards Utils.stringify method. There is a lot of string concatenation. Using a StringBuilder should help here. Also the method tryFindingOverridenToString is part of the problem. Having an internal cache so the lookp is not done multiple times should alleviate that problem. I have a test case written since I need to fix this problem. Let me know if you would like any help.

Following code snippet used to work on 1.0.2 and fails on 1.0.4

    private Object nullArray() {
        return new Object[] {/*is null*/  /* array */
                new Object[] { false,     new String[] { "1", "2" } },
                new Object[] { true ,     null },
        };
    }

    @Test
    @Parameters(method = "nullArray")
    public void testNullArraysArePassedIn(boolean isNullArray, String[] array) throws Exception {
        Assert.assertEquals(isNullArray, array == null);
    }

Error message is the following:

[junit]      java.lang.NullPointerException
[junit]          at junitparams.internal.InvokeParameterisedMethod.createArrayOfTypesOf(InvokeParameterisedMethod.java:90)
[junit]          at junitparams.internal.InvokeParameterisedMethod.castParamsFromObjects(InvokeParameterisedMethod.java:62)
[junit]          at junitparams.internal.InvokeParameterisedMethod.<init>(InvokeParameterisedMethod.java:34)
[junit]          at junitparams.internal.ParameterisedTestClassRunner.buildMethodInvoker(ParameterisedTestClassRunner.java:125)
[junit]          at junitparams.internal.ParameterisedTestClassRunner.parameterisedMethodInvoker(ParameterisedTestClassRunner.java:119)
[junit]          at junitparams.JUnitParamsRunner.methodInvoker(JUnitParamsRunner.java:434)
[junit]          at junitparams.JUnitParamsRunner.runChild(JUnitParamsRunner.java:414)
[junit]          at junitparams.JUnitParamsRunner.runChild(JUnitParamsRunner.java:385) 

Reported by mkucko, Dec 12, 2013
Writing my tests I noticed a lot of new int[] {...} statements.
I think more shortcuts could be useful.

#6 mkucko

Checkout https://github.com/mucaho/JArrayLiterals .

First of all, thanks for sharing your great work!
Please choose a (permissive? ;) license and add a license file so that I can use it at work too.

@Paramtrized accepts value and method. 'Value' is used as a data source, 'method' is ignored.
I would expect that an exception would be thrown when one defines multiple data sources or that all defined data sources are used in the test.

Could we add a feature for the runner to parallelize test execution (using an executor service based on the available core or whatever policy you deem appropriate)?

I've about 1000 parametrized tests in a single module and while intellij is showing me a total runtime of about 4s, in reality it takes more like 40-50s (i suspect the startup/shutdown harness work, class loading, etc, are at play). These are pure unit tests and do not access outside resources so they are definitely CPU bound and CPU utliziation is only about 30% while running. Suggestions very welcome. Thanks.

@RunWith(JUnitParamsRunner.class) is currently required but this means if there is already a @RunWith it won't work. There is no simple way. See http://stackoverflow.com/questions/24431427

Suggesting either:

1. Find a way to avoid requiring a @RunWith altogether,
2. A @Rule that does same thing
3. Some code to put in the class to initialize it

The javadoc for junitparams.Parameters#value says that

Each element in the array is a full parameter set, comma-separated.

However, pipe symbol (|) is also treated as a parameter value separator (junitparams.internal.InvokeParameterisedMethod#splitAtCommaOrPipe). This should be clearly documented.

Also, the error message in junitparams.internal.InvokeParameterisedMethod#castParamsFromString reads

Cannot parse parameters. Did you use , as column separator?

so it could be enhanced to mention the possibility of using a pipe.

JUnit 4.12 supports Hierarchical context runner so it would be great if we can provide that support in JUnitParamsRunner

@Runwith(JunitParamsHeirarchicalRunner.class)
public class ParentContext {

   // some tests go here

   @Runwith(JunitParamsHeirarchicalRunner.class)
   public class ChildContext {
        //some other tests go here
        ...
   }
}

Hi there!

Right now, parameter providers can return Iterable<Object> for single-parameter test methods or Iterable<Object[]> for multi-parameter test methods. I'd love to see support for Iterable<Iterable<Object>> for multi-parameter test methods as well. It seems reasonable to guess that people who want to use Iterables probably want to use them at both levels. For example, this would enable the following use case, which would result in 18 (3 * 3 * 2) calls:

@Test
@Parameters
public void parametersInCollection(String p1, int p2, PersonType p3) { }
public Set<List<Object>> parametersForParametersInCollection() {
    return Sets.cartesianProduct(
        ImmutableSet.<Object>of("a", "b", "c"),
        ImmutableSet.<Object>of(1, 2, 3),
        ImmutableSet.<Object>copyOf(PersonType.values()),
    );
}

It looks like this would require relatively minor changes here:

Thanks!

I accidentally created a test method that takes one parameter.
(Test methods are supposed to take zero parameters.)
The error message tells me that something is wrong, but neither the error message nor the stack trace tell me which test has the problem!

My test method:

    @Test
    public void testAddressMulti_parseValue_invalid(String input) {
        fail();
    }

The error message:

    [junit] Testcase: testAddressMulti_parseValue_invalid(org.thevoterguide.vgt.model.ballot.BioFieldTypeTest): Caused an ERROR
    [junit] wrong number of arguments
    [junit] java.lang.IllegalArgumentException: wrong number of arguments
    [junit]     at junitparams.JUnitParamsRunner.runChild(JUnitParamsRunner.java:416)
    [junit]     at junitparams.JUnitParamsRunner.runChild(JUnitParamsRunner.java:385)

I think the IllegalArgumentException message should include the name of the test that couldn't be run.

Do I have to know the actual parameter values ahead of time or can they be generated on the fly automatically? For example this parameter could be a seed to a RNG and I would pre-generate the seeds to run the test N different times with each test getting a different seed value. Only I would use a function to generate the seeds so I would not have to type them or save them to a file. Thanks.

If I annotate a parameter with some annotation that is not known to JUnitParams I get the following exception:

Caused by: java.lang.RuntimeException: Only @ConvertParam annotation is allowed on parameters!

This makes it impossible to use annotations that are used by other tools. Additional annotations on parameters should just be ignored by the JUnitParams runner.

Just porting the existing issue from GoogleCode since appears that I am running into it as well. I don't see it as a closed issue in GHE nor did I see it in the release notes for 1.0.4.

https://code.google.com/archive/p/junitparams/issues/60

I am getting the error:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test (default-test) on project metrics-aggregator-daemon: Execution default-test of goal org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test failed: There was an error in the forked process

And my stack trace:

[ERROR] org.apache.maven.surefire.testset.TestSetFailedException: java.lang.NullPointerException
[ERROR] at org.apache.maven.surefire.common.junit4.JUnit4RunListener.rethrowAnyTestMechanismFailures(JUnit4RunListener.java:213)
[ERROR] at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.createRequestAndRun(JUnitCoreWrapper.java:109)
[ERROR] at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.executeEager(JUnitCoreWrapper.java:78)
[ERROR] at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:54)
[ERROR] at org.apache.maven.surefire.junitcore.JUnitCoreProvider.invoke(JUnitCoreProvider.java:144)
[ERROR] at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203)
[ERROR] at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155)
[ERROR] at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
[ERROR] Caused by: java.lang.NullPointerException
[ERROR] at org.apache.maven.surefire.common.junit4.JUnit4RunListener.extractClassName(JUnit4RunListener.java:182)
[ERROR] at org.apache.maven.surefire.common.junit4.JUnit4RunListener.getClassName(JUnit4RunListener.java:158)
[ERROR] at org.apache.maven.surefire.common.junit4.JUnit4RunListener.createReportEntry(JUnit4RunListener.java:153)
[ERROR] at org.apache.maven.surefire.common.junit4.JUnit4RunListener.testStarted(JUnit4RunListener.java:91)
[ERROR] at org.junit.runner.notification.SynchronizedRunListener.testStarted(SynchronizedRunListener.java:49)
[ERROR] at org.junit.runner.notification.RunNotifier$3.notifyListener(RunNotifier.java:121)
[ERROR] at org.junit.runner.notification.RunNotifier$SafeNotifier.run(RunNotifier.java:72)
[ERROR] at org.junit.runner.notification.RunNotifier.fireTestStarted(RunNotifier.java:118)
[ERROR] at org.junit.internal.runners.model.EachTestNotifier.fireTestStarted(EachTestNotifier.java:42)
[ERROR] at junitparams.internal.ParameterisedTestMethodRunner.runMethodInvoker(ParameterisedTestMethodRunner.java:45)
[ERROR] at junitparams.internal.ParameterisedTestMethodRunner.runTestMethod(ParameterisedTestMethodRunner.java:40)
[ERROR] at junitparams.internal.ParameterisedTestClassRunner.runParameterisedTest(ParameterisedTestClassRunner.java:147)
[ERROR] at junitparams.JUnitParamsRunner.runChild(JUnitParamsRunner.java:414)
[ERROR] at junitparams.JUnitParamsRunner.runChild(JUnitParamsRunner.java:385)
[ERROR] at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
[ERROR] at org.apache.maven.surefire.junitcore.pc.Scheduler$1.run(Scheduler.java:387)
[ERROR] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[ERROR] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[ERROR] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[ERROR] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[ERROR] at java.lang.Thread.run(Thread.java:745)

JUnit and JUnitParams versions:

    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>4.12</version>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>pl.pragmatists</groupId>
      <artifactId>JUnitParams</artifactId>
      <version>1.0.4</version>
      <scope>test</scope>
    </dependency>

Surefire configuration:

        <plugin>
          <artifactId>maven-surefire-plugin</artifactId>
          <version>2.18.1</version>
          <executions>
            <execution>
              <id>default-test</id>
              <phase>test</phase>
              <goals>
                <goal>test</goal>
              </goals>
              <configuration>
                <skipTests>false</skipTests>
                <parallel>methods</parallel>
                <threadCount>4</threadCount>
                <perCoreThreadCount>true</perCoreThreadCount>
                <argLine>${argLine} -Xms512m -ea -Duser.timezone="UTC"</argLine>
                <includes>
                  <include>**/Test*.java</include>
                  <include>**/*Test.java</include>
                  <include>**/*TestCase.java</include>
                  <include>**/Test*.kt</include>
                  <include>**/*Test.kt</include>
                  <include>**/*TestCase.kt</include>
                </includes>
              </configuration>
            </execution>
          </executions>
        </plugin>

There were some ideas in the original report on how to fix this. Let me know if there is anything else I can provide to help track this down; if I manage to narrow down the problem I will post an update.

If you use @FileParameters to pull in .csv data, you can use a varargs method to collect an arbitrary number of remaining items.
An inconsistency in the csv varargs handling causes problems, though, when some cases should have no values.

For example:

@Test
@FileParameters("File.csv")
public void testLegalOptions(String iMode, String... iValues) {

1. If the .csv file line is "Regular" (with no trailing comma) then an error is signalled:
   java.lang.IllegalArgumentException: Cannot parse parameters. Did you use ',' or '|' as column separator?
2. If the line has a trailing comma ("Regular," ... or "Regular, " or "Regular, "), then iValues is populated with an length-one array containing an empty string.

I believe that number 2 is correct behavior, and that the error-handling in number 1 should be fixed to provide an empty array when there is no trailing comma.

Just download latest version from master and got 'IllegalArgumentException' for following code:

/**
 * Test data
 * @return
 */
@SuppressWarnings("unused")
private Object[] partitioning_data() {
    return $(
            $(new int[]{38, 23, 9, 19, 113, 5, 42, 85, 71, 112}, 3)
    );
}

/**
 * Test method for {@link codeabbey.QuickSort#partitioning(int[])}.
 */
@Test
@Parameters(method = "partitioning_data")
public void testPartitioning(int[] array_to_partition, int final_position) {
    int first_element = array_to_partition[0];
    assertTrue("List of partitions has been calculated incorrectly", first_element == QuickSort.partitioning(array_to_partition));
}

java.lang.IllegalArgumentException: Number of parameters inside @parameters annotation doesn't match the number of test method parameters.
There are 1 parameters in annotation, while there's 2 parameters in the testPartitioning method.
at junitparams.internal.InvokeParameterisedMethod.verifySameSizeOfArrays(InvokeParameterisedMethod.java:228)
at junitparams.internal.InvokeParameterisedMethod.castParamsUsingConverters(InvokeParameterisedMethod.java:104)
at junitparams.internal.InvokeParameterisedMethod.castParamsFromObjects(InvokeParameterisedMethod.java:58)
at junitparams.internal.InvokeParameterisedMethod.(InvokeParameterisedMethod.java:35)

Using jUnit 4.11 and jUnitParams 1.0.4

java.lang.RuntimeException: Cannot find invoker for the parameterised method. Using wrong JUnit version?
    at junitparams.internal.ParameterisedTestMethodRunner.findParameterisedMethodInvokerInChain(ParameterisedTestMethodRunner.java:71)
    at junitparams.internal.ParameterisedTestMethodRunner.findChildForParams(ParameterisedTestMethodRunner.java:58)
    at junitparams.internal.ParameterisedTestMethodRunner.runTestMethod(ParameterisedTestMethodRunner.java:39)
    at junitparams.internal.ParameterisedTestClassRunner.runParameterisedTest(ParameterisedTestClassRunner.java:143)
    at junitparams.JUnitParamsRunner.runChild(JUnitParamsRunner.java:405)
    at junitparams.JUnitParamsRunner.runChild(JUnitParamsRunner.java:383)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
    at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)

Current Scenario: I initialize some variables in BeforeClass method and keep them as members. Since, I do some processing on files, to optimize tests, I just do them once in the @BeforeClass method, and have asserts in different tests.

Requirement: I want to test multiple files, and hence was thinking may be there is some way I can call BeforeClass with each file and call all of the tests again with the new members.

Is there any way to do this? Maybe some class level params?

JUnit has Parameterize now: https://github.com/junit-team/junit/wiki/Parameterized-tests

but I like this JUnitParams code better, using an annotation. (You can look at the data and the implementation in 1 screenful. I already liked the similar feature in py.test, the Python test framework which uses decorators to parameterize.)

Is there a plan to get JUnitParams into JUnit, or will this remain as a independent plugin?

When I run only one test class, one of test classes with @parameter method is always instantiated.
If I remove parameter annotation, the only one test is executed.

How can I prevent test class with @parameter not to be instantiated?

please remove unused import 'org.assertj' from imported-packages in MANIFEST.MF.
besides that, the correct import package would need to import 'org.assertj.core.api' or something like that.

see MANIFEST:MF file of

org.assertj
assertj-core
1.7.1

When I try to run an individual test case with parameters in Eclipse, I get an initializationError from the runner. I'm always forced to run the whole class, i.e. all test cases.
Any way to do this?

JUnitParams supports most of the literals - numeric, enums, strings, but class names are not automatically converted:

    @Test
    @Parameters({"java.lang.Exception"})
    public void testException(Class<?> e) {...}

fails with

java.lang.IllegalArgumentException: Parameter type (java.lang.Class) cannot be handled! Only primitive types and Strings can be used.
    at junitparams.internal.InvokeParameterisedMethod.castParameterDirectly(InvokeParameterisedMethod.java:192)
    at junitparams.internal.InvokeParameterisedMethod.castAllParametersToProperTypes(InvokeParameterisedMethod.java:143)
    at junitparams.internal.InvokeParameterisedMethod.castParamsUsingConverters(InvokeParameterisedMethod.java:104)
    at junitparams.internal.InvokeParameterisedMethod.castParamsFromString(InvokeParameterisedMethod.java:45)

It would be nice to see that working :)

How do I access the current test parameters in the TestWatcher class? I have custom logic in the failed and succeeded methods of my own TestWatcher class and I need to access the current test parameter values ideally from the Description object but I am unable to locate them. I also tried using the @TestCaseName annotation on my test method but that didn't produce any better data in the TestWatcher class methods.

It's very strange. We have link Jenkins with Jazz Server(RTC) so that the latest code can be reached. And build toolkit also configured.
When I run the job with parameter "PostalErrorMsgValidation.xml", it build success. But when I change it to "maventest.xml", it alway failed with the following error. I don't know why was that, just change a parameter.

But it can be success in the command window with the command: mvn test -DsuiteXmlFile=src/com/ibm/testautomation/TestPlan/maventest.xml -Djenkins_env=DST2_LIVE -Djenkins_productType=WATSON_ANALYTICS -Djenkins_locale=en_US -Djenkins_browser=firefox

Results :

Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

[JENKINS] Recording test results

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 30.231 s
[INFO] Finished at: 2015-05-08T10:35:12+08:00
[INFO] Final Memory: 27M/75M
[INFO] ------------------------------------------------------------------------
Waiting for Jenkins to finish collecting data
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.17:test (default-test) on project SCWTestAutomation: Execution default-test of goal org.apache.maven.plugins:maven-surefire-plugin:2.17:test failed: There was an error in the forked process
[ERROR] java.lang.RuntimeException: java.lang.NullPointerException
[ERROR] at org.testng.internal.MethodInvocationHelper.invokeDataProvider(MethodInvocationHelper.java:161)
[ERROR] at org.testng.internal.Parameters.handleParameters(Parameters.java:429)
[ERROR] at org.testng.internal.FactoryMethod.invoke(FactoryMethod.java:63)
[ERROR] at org.testng.internal.TestNGClassFinder.(TestNGClassFinder.java:140)
[ERROR] at org.testng.TestRunner.initMethods(TestRunner.java:409)
[ERROR] at org.testng.TestRunner.init(TestRunner.java:235)
[ERROR] at org.testng.TestRunner.init(TestRunner.java:205)
[ERROR] at org.testng.TestRunner.(TestRunner.java:153)
[ERROR] at org.testng.SuiteRunner$DefaultTestRunnerFactory.newTestRunner(SuiteRunner.java:522)
[ERROR] at org.testng.SuiteRunner.init(SuiteRunner.java:157)
[ERROR] at org.testng.SuiteRunner.(SuiteRunner.java:111)
[ERROR] at org.testng.TestNG.createSuiteRunner(TestNG.java:1299)
[ERROR] at org.testng.TestNG.createSuiteRunners(TestNG.java:1286)
[ERROR] at org.testng.TestNG.runSuitesLocally(TestNG.java:1140)
[ERROR] at org.testng.TestNG.run(TestNG.java:1057)
[ERROR] at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:293)
[ERROR] at org.apache.maven.surefire.testng.TestNGXmlTestSuite.execute(TestNGXmlTestSuite.java:84)
[ERROR] at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:91)
[ERROR] at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:200)

Hello,

When using JUnitParams 1.0.4 with JUnit 4.10 I run into this exception when I run my unit tests:

java.lang.NoSuchMethodError: org/junit/runner/Description.createTestDescription(Ljava/lang/String;Ljava/lang/String;Ljava/io/Serializable;)Lorg/junit/runner/Description;
at junitparams.internal.TestMethod.describe(TestMethod.java:106)
at junitparams.internal.ParameterisedTestClassRunner.describeParameterisedMethod(ParameterisedTestClassRunner.java:162)
at junitparams.JUnitParamsRunner.describeMethod(JUnitParamsRunner.java:460)
at junitparams.internal.ParametrizedTestMethodsFilter.filteredMethods(ParametrizedTestMethodsFilter.java:30)
at junitparams.JUnitParamsRunner.getListOfMethods(JUnitParamsRunner.java:456)
at junitparams.JUnitParamsRunner.getDescription(JUnitParamsRunner.java:445)
at org.junit.runner.Runner.testCount(Runner.java:38)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestClassReference.countTestCases(JUnit4TestClassReference.java:30)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.countTests(RemoteTestRunner.java:487)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:455)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)

I tried using JUnitParams 1.0.0 and it worked successfully.

Regards,

Walter

Hi,

the paramtrization implementation of JUnit offers the nice feature to self-describe the parameters like this:
@Parameters(name = "{index}: add({0}+{1})={2}")

I would really like to see this feature in JUnitParams. The test results in case of an error are much more understandable this way as you can see here: http://www.mkyong.com/wp-content/uploads/2009/05/junit-parameter-example-1.png

Any chance an active contributor might add it?

Regards
-René

upgrade readme and runner JavaDoc

I introduced this with PR #60 (sorry!). Will fix

Contents of an array passed as one of test parameters are currently not being displayed in the test's description generated by @TestCaseName annotation.

Example:

private static final File JAVA_DIRECTORY = get("src/main/java/").toFile();
private static final String[] NOT_ACCEPTABLE_IN_JAVA_DIRECTORY = {"groovy", "css", "html"};

public Object[] filesNotAcceptedInDirectories() {
    return $(
            $(JAVA_DIRECTORY, NOT_ACCEPTABLE_IN_JAVA_DIRECTORY)
    );
}

@Test
@Parameters(method = "filesNotAcceptedInDirectories")
@TestCaseName("Verify that no files with extensions {1} are present in directory {0}")
public void verifyFilesArePlacedInAppropriateDirectories(File directory, String[] unacceptableFiles) {
    assertThat(listFiles(directory, unacceptableFiles, true)).isEmpty();
}

should display as:

Verify that no files with extensions [groovy, css, html] are present in directory src\main\java

but what we currently get is:
Verify that no files with extensions String[] are present in directory src\main\java

This is a feature-request.

I would like to propose a @CombinedParameters, the reason is that I dont want all my parameters to be in just one provider.

A quick example:

public @interface CombinedParameters {
Parameters[] value();
}

@test
@CombinedParameters(
@parameters(source = ProviderA.class),
@parameters(source = ProviderB.class)
)
public void test(String fromA, String fromB) {
...
}

public class ProviderA {
public static Object[] provideStrings() {
return $(
"A1",
"A2"
);
}
}

public class ProviderB {
public static Object[] provideAString() {
return $(
"B1"
);
}

public static Object[] provideAnotherString() {
    return $(
                "B2"
           );
}

}

In this example, my test should be executed four times with the combines parameters like this:

test("A1", "B1")
test("A1", "B2")
test("A2", "B1")
test("A2", "B2")

#1 [email protected]

I thought about this again, at CombinedParameters should possibly extend Parameters, which would allow nesting CombinedParameters.

#3 jim.newsham

I just discovered junitparams yesterday, and I think this project is great. I had the same idea as this feature request, and I also feel that this feature should also be made available not only as "providers" but using the other mechanisms that junitparams supports such as providing method names. For example:

@test
@parameters(method = "versions, flushTypes")
public void rangeValidity(Version version, FlushType) throws Exception {
...
}

I know that the current junitparams implementation would use versions() and flushTypes() to generate a single list of parameters, but I would like to use something like the above to produce a cross-product of the provided parameters as described earlier in this issue.

I realize this feature will not be implemented overnight, and since I need something of this sort immediately, I whipped up an interim hack which I would like to share here. The following method produces a cross-product of parameters:

private static Object[] product(Object[]... paramLists) {
if (paramLists.length == 0) {
return new Object[0];
}

int count = 1;
for (int i = 0; i < paramLists.length; i++) {
  int listLength = paramLists[i].length;
  if (listLength == 0) {
    return new Object[0];
  }
  count *= listLength;
}

int[] mods = new int[paramLists.length + 1];
mods[paramLists.length] = 1;
for (int i = paramLists.length - 1; i >= 0; i--) {
  mods[i] = mods[i + 1] * paramLists[i].length; 
}

Object[] result = new Object[count];
for (int i = 0; i < count; i++) {
  Object[] record = new Object[paramLists.length];
  Object[] indices = new Object[paramLists.length];
  for (int j = 0; j < paramLists.length; j++) {
    int index = (i / mods[j + 1]) % paramLists[j].length;
    indices[j] = index;
    record[j] = paramLists[j][index];
  }
  result[i] = record; 
}
return result;

}

And here is how I would use it (a little cumbersome, but it does the job):

private static Object[] versions() {
...
}

private static Object[] flushTypes() {
...
}

private static Object[] versions_FlushTypes() {
return product(versions(), flushTypes());
}

@test
@parameters(method = "versions_flushTypes")
public void rangeValidity(Version version, FlushType) throws Exception {
...
}

The org.junit.rules.TestName rule returs NULL for the TestName.getMethodName()

Small sample test

import static junitparams.JUnitParamsRunner.$;
import static org.junit.Assert.assertEquals;

import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.TestName;
import org.junit.runner.RunWith;


@RunWith(JUnitParamsRunner.class)
public class TestSampleJunit4JUnitParams {

  @Rule
  public TestName mName = new TestName();

  public String getName() {
    return mName.getMethodName();
  }

  @Test
  @Parameters(method = "parametersForIsAdult")
  public void isAdult(PersonX person, boolean valid) throws Exception {
   // mName is not null 
    System.out.println(mName); 
// but mName.getMethodName() returns null
    System.out.println("in isAdult " + getName() + " using thread "
        + Thread.currentThread().getName() + " person:" + person);
    assertEquals(valid, person.isAdult());
  }

  @SuppressWarnings("unused")
  private Object[] parametersForIsAdult() {
    return $(
                 $(new PersonX(13), false),
                 $(new PersonX(22), true)
            );
  }

}

class PersonX {
  private int age;

  public PersonX(int age) {
    this.age = age;
  }

  public boolean isAdult() {
    return age >= 18;
  }

  @Override
  public String toString() {
    return "Person of age: " + age;
  }

}

Cheers
Lucas

I was trying to use JUnitParams to help me parameterize a test for various types of line breaks. The tests were failing and upon further inspection, I came to realize that of the standard line breaks the parameter was an empty string.

Digging through the JUnitParams source, I believe the problem lies in Utils.splitAtCommaOrPipe(). However, I'm not entirely sure what the correct behavior should be. Whitespace should be trimmed in most cases, but it's feasible that you'd want whitespace to be passed as a parameter.

It seems that the workaround is simply to use a method or provider class to supply the values, so this may be intended behavior.

Hi all,

According to the release notes for 1.0.5

Utility method $ was deprecated. It was causing too much problems and we decided not to support it any more. If you wish to keep using it, implement it in your own codebase.

What exactly are the problems? As this method is quite commonly used in our codebase we are considering implementing our own version and it'd be good to know the possible pitfalls.

Best,
Maciej

Is Jenkins able to interpret the JUnit XML file of JUnitParams today? Looks like a layer of reporting below the test name would be needed.

In this kind of report, what should show up as the test name in Jenkins? The name inside parens?

<testsuite name="junitparams.usage.Samples_of_Usage_Test" time="0.005" tests="39" errors="0" skipped="0" failures="0">
<testcase name="[0] SOME_VALUE (enums_as_params_in_annotation)" classname="junitparams.usage.Samples_of_Usage_Test" time="0.001"/>
<testcase name="[1] OTHER_VALUE (enums_as_params_in_annotation)" classname="junitparams.usage.Samples_of_Usage_Test" time="0"/>
</testsuite>

I can see users like myself wanting Jenkins to track the trend both at the test level and at the parameter level. Is that where JUnitParams is going overall?

Example: run mvn test in JUnitParams project with following configuration:

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.18.1</version>

                <configuration>
                    <runOrder>filesystem</runOrder>
                    <parallel>classesAndMethods</parallel>
                    <perCoreThreadCount>false</perCoreThreadCount>
                    <threadCount>16</threadCount>
                    <useUnlimitedThreads>false</useUnlimitedThreads>
                    <threadCountSuites>0</threadCountSuites>
                    <threadCountClasses>0</threadCountClasses>
                    <threadCountMethods>0</threadCountMethods>
                    <parallelOptimized>true</parallelOptimized>
                    <includes>
                        <include>**/SubclassTest.java</include>
                        <include>**/Samples_*</include>
                    </includes>
                </configuration>
            </plugin>

This should give total of 44 tests as of today, but often it will give less than that. Contact me for details about research done so far.
Suspicious places:

• org.apache.maven.surefire.junitcore.JUnitCoreRunListener#fillTestCountMap
• org.apache.maven.surefire.junitcore.ClassesParallelRunListener#checkIfTestSetCanBeReported
• org.apache.maven.surefire.junitcore.TestSet#setAllScheduled

Currently following values of surefire "parallel" option should not be used:

• "classesAndMethods"
• "methods"
• "both"

For now the order is looking like this be default:
parameterProvider() -> BeforeClass -> Test(parameters)
Can I switch the parameterProvider() and BeforeClass to make it like this?
BeforeClass -> parameterProvider() -> Test(parameters)

Thanks,
-feng

I have an @Parameters("foo", "123", " ") for a certain testcase. However the " " is trimmed to an empty string. This makes my testcase fail.

I don't thing the parameter values should be trimmed.

This could be a potential performance problem for people using @FileParameters
Please +1 if anyone finds this problematic. Let us know if this is an important issue for you before we invest time in some caching mechanism implementation.

Hi!

I am also doing JUnit extension/runner and I discovered your nice lib so I wanted to test my stuff with yours.

But filters doesn't seems to work as it should. Also the rule for test name seems to be broken. I file another issue for that. JUnits own parameter support works with filters.

I have some small test that you can run.
Here is the test (sorry I used your package name):

package junitparams;
import static org.junit.Assert.assertEquals;
import org.junit.Test;
import org.junit.runner.Description;
import org.junit.runner.JUnitCore;
import org.junit.runner.Request;
import org.junit.runner.Result;
import org.junit.runner.manipulation.Filter;

public class FilterTest {
  public static JUnitCore sJUnitCore = new JUnitCore();

  @Test
  public void testFilter() throws Throwable {
    // just so that you can see what the "names" would be
    Description description = createDescription(TestSampleJunit4JUnitParams.class);
    for (Description childDesc : description.getChildren()) {
      System.out.println(childDesc.getDisplayName());
      for (Description childOfChild : childDesc.getChildren()) {
        System.out.println(childOfChild.getMethodName());
      }
    }

    Result result = runTest(TestSampleJunit4JUnitParams.class, "[0] Person of age: 13, false (isAdult)");
    System.out.println("result:" +  result.getFailures());
    assertEquals(0, result.getFailureCount());
    assertEquals(1, result.getRunCount());
  }

  public Result runTest(Class<?> testClass, String methodName) throws Throwable {
    Filter filter = Filter.matchMethodDescription(Description.createTestDescription(testClass, methodName));
    Request request = Request.aClass(testClass);
    request = request.filterWith(filter);
    Result result = sJUnitCore.run(request);
    return result;
  }  

  protected Description createDescription(Class<?> testClass) throws ClassNotFoundException {
    Request request = Request.aClass(testClass);
    Description description = request.getRunner().getDescription();
    return description;
  }
}

And here is the sample test that uses the runner:

package junitparams;

import static junitparams.JUnitParamsRunner.$;
import static org.junit.Assert.assertEquals;

import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.TestName;
import org.junit.runner.RunWith;


@RunWith(JUnitParamsRunner.class)
public class TestSampleJunit4JUnitParams {

  @Rule
  public TestName mName = new TestName();

  public String getName() {
    return mName.getMethodName();
  }

  @Test
  @Parameters(method = "parametersForIsAdult")
  public void isAdult(PersonX person, boolean valid) throws Exception {
    System.out.println(mName);
    System.out.println("in isAdult " + getName() + " using thread "
        + Thread.currentThread().getName() + " person:" + person);
    assertEquals(valid, person.isAdult());
  }

  @SuppressWarnings("unused")
  private Object[] parametersForIsAdult() {
    return $(
                 $(new PersonX(13), false),
                 $(new PersonX(22), true)
            );
  }

}

class PersonX {
  private int age;

  public PersonX(int age) {
    this.age = age;
  }

  public boolean isAdult() {
    return age >= 18;
  }

  @Override
  public String toString() {
    return "Person of age: " + age;
  }

}

The error I got is like this:

result:[initializationError(org.junit.runner.manipulation.Filter): No tests found matching Method [0] Person of age: 13, false (isAdult)(junitparams.TestSampleJunit4JUnitParams) from org.junit.internal.requests.ClassRequest@1a86f2f1]

If you happen to use Eclipse then you can see that filters doesn't work to.
First select any test class that uses the runner and do "Run as JUnit". It will run fine. But when you want to just re-run one of the parameterized tests (one of the "children") then it will not work. (in the JUnit runner view)

I actually tried to see if I could fix it but alas I got a bit confused about the concepts in JUnit4 ;-)
I suspect that the root cause is that the JUnitParamsRunner extends BlockJUnit4ClassRunner which in turn is a runner for FrameworkMethod(s). In JUnitParamsRunner instance for the above test there will only be one (1) FrameworkMethod, isAdult(junitparams.TestSampleJunit4JUnitParams) but the filter is for [0] Person of age: 13, false (isAdult)(junitparams.TestSampleJunit4JUnitParams).
First I thought that you faked one FrameworkMethod for each combination of method and parameters but then I saw that was nto the case. But maybe that could be a solution.

Cheers
Lucas

I tried to pass in some parameters to a test like so, forgetting to include the @parameters annotation:

@Test
public void test(String testParameter) {..}

Generates the following stack trace:

java.lang.IllegalArgumentException: wrong number of arguments
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
    at junitparams.JUnitParamsRunner.runChild(JUnitParamsRunner.java:416)
    at junitparams.JUnitParamsRunner.runChild(JUnitParamsRunner.java:385)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
    at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:78)
    at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:212)
    at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:68)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)

Took me a sec to figure it out. Might be worthwhile to generate a more relevant exception message for this case

I'm trying to use surefire feature -Dsurefire.rerunFailingTestsCount on failing test. If count=1, it's supposed to be run again one time, but it doesn't:

Running Test3
Tests run: 2, Failures: 1, Errors: 1, Skipped: 0, Time elapsed: 0.055 sec <<< FAILURE! - in Test3
[0] AAA,1 (testTestTest)(Test3) Time elapsed: 0.005 sec <<< FAILURE!
java.lang.AssertionError
at Test3.testTestTest(Test3.java:55)

initializationError(org.junit.runner.manipulation.Filter) Time elapsed: 0 sec <<< ERROR!
java.lang.Exception: No tests found matching Method [0] AAA,1 (testTestTest)(Test3) from org.junit.internal.requests.ClassRequest@27ddd392

Results :

Failed tests:
Test3.testTestTest:55
Tests in error:
Filter.initializationError » No tests found matching Method [0] AAA,1 (testTe...

Tests run: 2, Failures: 1, Errors: 1, Skipped: 0

I found bugreport for surefire which was fixed in 2.19. I took sample project from this report and run it as-is with surefire 2.19.1, junit 4.12 (it worked), then replaced default runner with JUnitParamsRunner, @parameters annotation (to fit replaced runner) and got an error.

import org.junit.Assert;
import org.junit.Test;
import org.junit.runner.RunWith;
import junitparams.JUnitParamsRunner;
import junitparams.Parameters;

@RunWith(JUnitParamsRunner.class)
public class Test3 {

    private static boolean test;

    static {
        test = true;
    }

    @Test
    @Parameters({"AAA,1"})
    public void testTestTest(String p1, int p2) {
        boolean current = test;
        test = !test;
        Assert.assertFalse(current);
    }
}

I also tried @TestCaseName("{method}[{index}]") annotation, but it didn't help.
Could it be a bug in runner?

Reported by juxeiier, Feb 5, 2014
Please see this question with a working example:
http://stackoverflow.com/questions/21582378/junitparams-not-working-with-string-array

Please review the safelyCastParamsToArray method.

Given this example test:

@RunWith(JUnitParamsRunner.class)
public class QuickTest{
    @Parameters
    @Test
    public void test(int i) {
        assertEquals(i, i);
    }
    private Object[] parametersForTest() { return $($(0), $(1), $(2), $(3));}
}

Running this within Eclipse Luna works fine when I run it as a JUnit test using Java 8 however when I try to run it as part of a compiled JAR file on an embeded linux arm chip using an embedded version of Java 8 I receive the following exception:

Starting Tests
JUnit version 4.11
The following tests were loaded:
    QuickTest

Failure List: 
initializationError(edu.wpi.first.wpilibj.test.QuickTest)
java.lang.annotation.AnnotationFormatError: Invalid default: public abstract java.lang.Class junitparams.Parameters.source()
    at java.lang.reflect.Method.getDefaultValue(Method.java:597)
    at sun.reflect.annotation.AnnotationType.<init>(AnnotationType.java:128)
    at sun.reflect.annotation.AnnotationType.getInstance(AnnotationType.java:85)
    at sun.reflect.annotation.AnnotationParser.parseAnnotation2(AnnotationParser.java:266)
    at sun.reflect.annotation.AnnotationParser.parseAnnotations2(AnnotationParser.java:120)
    at sun.reflect.annotation.AnnotationParser.parseAnnotations(AnnotationParser.java:72)
    at java.lang.reflect.Executable.declaredAnnotations(Executable.java:546)
    at java.lang.reflect.Executable.getDeclaredAnnotations(Executable.java:539)
    at java.lang.reflect.Method.getDeclaredAnnotations(Method.java:615)
    at java.lang.reflect.AccessibleObject.getAnnotations(AccessibleObject.java:206)
    at org.junit.runners.model.FrameworkMethod.getAnnotations(FrameworkMethod.java:189)
    at org.junit.runners.model.TestClass.addToAnnotationLists(TestClass.java:58)
    at org.junit.runners.model.TestClass.<init>(TestClass.java:46)
    at org.junit.runners.ParentRunner.<init>(ParentRunner.java:75)
    at org.junit.runners.BlockJUnit4ClassRunner.<init>(BlockJUnit4ClassRunner.java:57)
    at junitparams.JUnitParamsRunner.<init>(JUnitParamsRunner.java:377)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
    at org.junit.internal.builders.AnnotatedBuilder.buildRunner(AnnotatedBuilder.java:29)
    at org.junit.internal.builders.AnnotatedBuilder.runnerForClass(AnnotatedBuilder.java:21)
    at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:59)
    at org.junit.internal.builders.AllDefaultPossibilitiesBuilder.runnerForClass(AllDefaultPossibilitiesBuilder.java:26)
    at org.junit.runner.Computer.getRunner(Computer.java:40)
    at org.junit.runner.Computer$1.runnerForClass(Computer.java:31)
    at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:59)
    at org.junit.runners.model.RunnerBuilder.runners(RunnerBuilder.java:101)
    at org.junit.runners.model.RunnerBuilder.runners(RunnerBuilder.java:87)
    at org.junit.runners.Suite.<init>(Suite.java:80)
    at org.junit.runner.Computer.getSuite(Computer.java:28)
    at org.junit.runner.Request.classes(Request.java:75)
    at org.junit.runner.JUnitCore.run(JUnitCore.java:128)
    at org.junit.runner.JUnitCore.runClasses(JUnitCore.java:73)
    at edu.wpi.first.wpilibj.test.TestSuite.parseArgsRunAndGetResult(TestSuite.java:180)
    at edu.wpi.first.wpilibj.test.TestSuite.main(TestSuite.java:323)

I have double checked to ensure that maven is correctly adding your library to the .jar file before I copy it over to the arm chip and attempt to run it.

Since 4874ae9
java.lang.IllegalArgumentException: Number of parameters inside @parameters annotation doesn't match the number of test method parameters.

junitparams.usage.Samples_of_Usage_Test#wrap_params_with_constructor
junitparams.usage.Samples_of_Usage_Test#load_params_from_any_file

Reported by feilimb, Apr 17, 2013
What steps will reproduce the problem?

1. Run any JUnit4+ test with JUnitParamsRunner

What is the expected output? What do you see instead?
Proposal for flexibility of the runtime TestName produced.

What version of the product are you using? On what operating system?
1.0.1

Please provide any additional information below.
I would like to propose that the test case name which gets generated at runtime can be more flexible with format specifiers. For example a developer could choose to omit the index which is prefixed to every test case name at present (eg. "[0] Person of age: 25, true" could be just "Person of age: 25").

The introduction of an annotation to specify a template for the Test case name could be one approach.

Thanks, Feilim

#2 nico.mommaerts

Also, as now a toString seems to be done on the arguments, the generated test names can be really long. This gives a problem with CI systems, for example in Bamboo the test names can only be 4000chars, resulting in a build failure.

#3 neo.kldo.test

Im having the same problem with overlong method/classnames leading to Errors in Sonar "Failed to execute goal org.codehaus.mojo:sonar-maven-plugin:2.2:sonar (default-cli): Can not execute SonarQube analysis: Resource key is too long, max is 400 characters. Got: ..."

JUnitParams 1.0.4

@Test
@Parameters(method = "primitiveArray")
public void testPrimitiveArray(int[] ints) {
    assertThat(ints).containsExactly(0, 1);
}

@Test
@Parameters(method = "objectArray")
public void testObjectArray(Integer[] integers) {
    assertThat(integers).containsExactly(0, 1);
}

public Object[] primitiveArray() {
    return $($(new int[]{0, 1}));
}

public Object[] objectArray() {
    return $($(new Integer[]{0, 1}));
}

testPrimitiveArray() passes, testObjectArray() fails with

java.lang.AssertionError: 
Actual and expected should have same size but actual size was:
  <1>
while expected size was:
  <2>
Actual was:
  <[0]>
Expected was:
  <[0, 1]>

It seems a primitive array parameter is OK, but an object array is not.

See also http://stackoverflow.com/questions/21582378/junitparams-not-working-with-string-array

I am planning to use

1. Spring MVC - For Integration testing
2. NestedRunner - For nesting test cases
3. JUnitParams - For DDT
4. Assumes - for specifying test dependencies
5. Mockito - For mocking
6. PowerMockito - For any mocking abuse I may need to do (I mean class/static method mocking)
7. DBUnit - For fiddling with DB data

Spring MVC in the current milestone version (4.2.0.RC1 or > 4.2 whenever the release happens) uses rules instead of runner to support integration with other runners. PowerMock currently supports Rules. Mockito supports Rule in the current Beta version (2.0.17-beta)

However, AFAIK, all others does not support rules. It would be better if this library can implement the runner using Junit Rule instead of Runner so that it can be compatible with all other runners to make a full blown testing stack. Else, people need to hack around to get most out of Java test frameworks.

Reference:

1. Creating a custom runner for mockito
2. Nested Runners Example

Hi guys,

once in a while (pretty much randomly, might be race condition) Surefire throws this:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test (default-test) on project trio-services: Execution default-test of goal org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test failed: There was an error in the forked process
[ERROR] org.apache.maven.surefire.testset.TestSetFailedException: java.lang.NullPointerException
[ERROR] at org.apache.maven.surefire.common.junit4.JUnit4RunListener.rethrowAnyTestMechanismFailures(JUnit4RunListener.java:213)
[ERROR] at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.createRequestAndRun(JUnitCoreWrapper.java:109)
[ERROR] at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.executeEager(JUnitCoreWrapper.java:78)
[ERROR] at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:54)
[ERROR] at org.apache.maven.surefire.junitcore.JUnitCoreProvider.invoke(JUnitCoreProvider.java:144)
[ERROR] at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203)
[ERROR] at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155)
[ERROR] at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
[ERROR] Caused by: java.lang.NullPointerException
[ERROR] at org.apache.maven.surefire.common.junit4.JUnit4RunListener.extractClassName(JUnit4RunListener.java:182)
[ERROR] at org.apache.maven.surefire.common.junit4.JUnit4RunListener.getClassName(JUnit4RunListener.java:158)
[ERROR] at org.apache.maven.surefire.common.junit4.JUnit4RunListener.createReportEntry(JUnit4RunListener.java:153)
[ERROR] at org.apache.maven.surefire.common.junit4.JUnit4RunListener.testStarted(JUnit4RunListener.java:91)
[ERROR] at org.junit.runner.notification.SynchronizedRunListener.testStarted(SynchronizedRunListener.java:49)
[ERROR] at org.junit.runner.notification.RunNotifier$3.notifyListener(RunNotifier.java:121)
[ERROR] at org.junit.runner.notification.RunNotifier$SafeNotifier.run(RunNotifier.java:72)
[ERROR] at org.junit.runner.notification.RunNotifier.fireTestStarted(RunNotifier.java:118)
[ERROR] at org.junit.internal.runners.model.EachTestNotifier.fireTestStarted(EachTestNotifier.java:42)
[ERROR] at junitparams.internal.ParameterisedTestMethodRunner.runMethodInvoker(ParameterisedTestMethodRunner.java:45)
[ERROR] at junitparams.internal.ParameterisedTestMethodRunner.runTestMethod(ParameterisedTestMethodRunner.java:40)
[ERROR] at junitparams.internal.ParameterisedTestClassRunner.runParameterisedTest(ParameterisedTestClassRunner.java:147)
[ERROR] at junitparams.JUnitParamsRunner.runChild(JUnitParamsRunner.java:414)
[ERROR] at junitparams.JUnitParamsRunner.runChild(JUnitParamsRunner.java:385)
[ERROR] at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
[ERROR] at org.apache.maven.surefire.junitcore.pc.Scheduler$1.run(Scheduler.java:387)
[ERROR] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
[ERROR] at java.util.concurrent.FutureTask.run(FutureTask.java:262)
[ERROR] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[ERROR] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[ERROR] at java.lang.Thread.run(Thread.java:724)
[ERROR] -> [Help 1]

Here are my environment details:

Apache Maven 3.2.3 (33f8c3e1027c3ddde99d3cdebad2656a31e8fdf4; 2014-08-11T22:58:10+02:00)
Java version: 1.7.0_40, vendor: Oracle Corporation
Java home: /usr/lib/jvm/jdk1.7.0_40/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "3.13.0-37-generic", arch: "amd64", family: "unix"

• JUnit 4.12
• Surefire 2.18.1
• JUnitParams 1.0.4

On top of that I run Surefire tests in parallel with following settings:

• parallel='classesAndMethods'
• perCoreThreadCount=false
• threadCount=8
• useUnlimitedThreads=false
• threadCountSuites=0
• threadCountClasses=0
• threadCountMethods=0
• parallelOptimized=true

Any ideas what might be the problem?

Note: this problem was also present with

• JUnit 4.11
• Surefire 2.18
• JUnitParams 1.0.3