shazam / fork Goto Github PK

View Code? Open in Web Editor NEW

389.0 389.0 144.0 70.43 MB

A library that offers the fastest way to execute Android instrumentation tests.

License: Other

Shell 0.21% Java 93.20% HTML 2.18% CSS 0.51% Groovy 3.01% Less 0.90%

fork's People

Contributors

Stargazers

Watchers

Forkers

jbaginski cookbrite morrisoncole rcjavier tmelz estefaniaduque mmrahman1 xjlswd vikingden saltchuck matrixy ammadghani deepanshumadan haterman fbetetarivero fpezzato calebboakye traviswestbrook oscar84922tw rooted1 q315099997 bavly eddieringle dmarin lreuven arpita184 ericjee banno mederic ijunes metova potamus2017 waseefakhtar itcasth rkochhar martynhaigh androidmi louisnguyen90 kittsville alonlevih veskoi agoda-com hibiscusss tarcv houzz utb-best jaishree cfirmo33 latheesh123 chgohlke ajostergaard jenzz cnxtech anton46 qautomatron nand-kishore-rajput andrushsha maxidebello sboishtyan eticzon hemantyadavat1994 vpeter86 nanemar bersh davidalbers jignesh-vyas tinkoff-mobile-tech tubbz-alt callas1 karrosse hs-11 monikanagy92 hongdongni larack8 jalanexplorer innamora69 ortallll mostafahosseen33 carbonlabs119 good288 ackvincent arunlodhi timi4444 ets-android5 christchristded lizhengdao safurh justnikee plezunto ahmadrizkiawanpratama tayebi23 huyka10 fredybombi mahabataka rrkumarvishwakarma dastageer777222 abdollahhassanzade thistian vitalijcelitan kadibado

fork's Issues

Allow "android.test.classes" per pool

I would like to be able to have a different android.test.classes for each pool.

For example, I might want to run a separate set of tests on my API 18+ devices.

Maybe android.test.classes.$pool_name$

IllegalArgumentException while creating html report

Version : 1.1.1-SNAPSHOT

java.lang.IllegalArgumentException: Array is empty    
        at org.apache.commons.lang3.BooleanUtils.and(BooleanUtils.java:944)
        at com.shazam.fork.summary.OutcomeAggregator.and(OutcomeAggregator.java:62)
        at com.shazam.fork.summary.OutcomeAggregator.access$100(OutcomeAggregator.java:27)
        at com.shazam.fork.summary.OutcomeAggregator$1.apply(OutcomeAggregator.java:46)
        at com.shazam.fork.summary.OutcomeAggregator$1.apply(OutcomeAggregator.java:40)
        at com.google.common.collect.Iterators$8.transform(Iterators.java:794)
        at com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
        at java.util.AbstractCollection.toArray(AbstractCollection.java:195)
        at com.shazam.fork.summary.OutcomeAggregator.and(OutcomeAggregator.java:62)
        at com.shazam.fork.summary.OutcomeAggregator.aggregate(OutcomeAggregator.java:36)
        at com.shazam.fork.summary.HtmlConverters.toHtmlSummary(HtmlConverters.java:36)
        at com.shazam.fork.summary.HtmlSummaryPrinter.print(HtmlSummaryPrinter.java:68)
        at com.shazam.fork.summary.CompositeSummaryPrinter.print(CompositeSummaryPrinter.java:14)
        at com.shazam.fork.summary.Summarizer.summarize(Summarizer.java:33)
        at com.shazam.fork.summary.SummaryGeneratorHook.defineOutcome(SummaryGeneratorHook.java:63)
        at com.shazam.fork.ForkRunner.run(ForkRunner.java:75)
        at com.shazam.fork.Fork.run(Fork.java:50)     
        at com.shazam.fork.Fork$run.call(Unknown Source)
        at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:112)
        at com.shazam.fork.gradle.ForkRunTask.runFork(ForkRunTask.groovy:74)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:75)
        at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.doExecute(AnnotationProcessingTaskFactory.java:226)
        at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.execute(AnnotationProcessingTaskFactory.java:219)
        at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.execute(AnnotationProcessingTaskFactory.java:208)
        at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:589)
        at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:572)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:80)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:61)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:46)
        at org.gradle.api.internal.tasks.execution.PostExecutionAnalysisTaskExecuter.execute(PostExecutionAnalysisTaskExecuter.java:35)
        at org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter.execute(SkipUpToDateTaskExecuter.java:64)
        at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:58)
        at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:42)
        at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:52)
        at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:53)
        at org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter.execute(ExecuteAtMostOnceTaskExecuter.java:43)
        at org.gradle.api.internal.AbstractTask.executeWithoutThrowingTaskFailure(AbstractTask.java:310)
        at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.executeTask(AbstractTaskPlanExecutor.java:79)
        at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.processTask(AbstractTaskPlanExecutor.java:63)
        at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.run(AbstractTaskPlanExecutor.java:51)
        at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor.process(DefaultTaskPlanExecutor.java:23)
        at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter.execute(DefaultTaskGraphExecuter.java:88)
        at org.gradle.execution.SelectedTaskExecutionAction.execute(SelectedTaskExecutionAction.java:37)
        at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:62)
        at org.gradle.execution.DefaultBuildExecuter.access$200(DefaultBuildExecuter.java:23)
        at org.gradle.execution.DefaultBuildExecuter$2.proceed(DefaultBuildExecuter.java:68)
        at org.gradle.execution.DryRunBuildExecutionAction.execute(DryRunBuildExecutionAction.java:32)
        at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:62)
        at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:55)
        at org.gradle.initialization.DefaultGradleLauncher.doBuildStages(DefaultGradleLauncher.java:149)
        at org.gradle.initialization.DefaultGradleLauncher.doBuild(DefaultGradleLauncher.java:106)
        at org.gradle.initialization.DefaultGradleLauncher.run(DefaultGradleLauncher.java:86)
        at org.gradle.launcher.exec.InProcessBuildActionExecuter$DefaultBuildController.run(InProcessBuildActionExecuter.java:90)
        at org.gradle.tooling.internal.provider.ExecuteBuildActionRunner.run(ExecuteBuildActionRunner.java:28)
        at org.gradle.launcher.exec.ChainingBuildActionRunner.run(ChainingBuildActionRunner.java:35)
        at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:41)
        at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:28)
        at org.gradle.launcher.daemon.server.exec.ExecuteBuild.doBuild(ExecuteBuild.java:49)
        at org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:36)
        at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
        at org.gradle.launcher.daemon.server.exec.WatchForDisconnection.execute(WatchForDisconnection.java:37)
        at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
        at org.gradle.launcher.daemon.server.exec.ResetDeprecationLogger.execute(ResetDeprecationLogger.java:26)
        at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
        at org.gradle.launcher.daemon.server.exec.RequestStopIfSingleUsedDaemon.execute(RequestStopIfSingleUsedDaemon.java:34)
        at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
        at org.gradle.launcher.daemon.server.exec.ForwardClientInput$2.call(ForwardClientInput.java:74)
        at org.gradle.launcher.daemon.server.exec.ForwardClientInput$2.call(ForwardClientInput.java:72)
        at org.gradle.util.Swapper.swap(Swapper.java:38)
        at org.gradle.launcher.daemon.server.exec.ForwardClientInput.execute(ForwardClientInput.java:72)
        at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
        at org.gradle.launcher.daemon.server.health.DaemonHealthTracker.execute(DaemonHealthTracker.java:47)
        at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
        at org.gradle.launcher.daemon.server.exec.LogToClient.doBuild(LogToClient.java:66)
        at org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:36)
        at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
        at org.gradle.launcher.daemon.server.exec.EstablishBuildEnvironment.doBuild(EstablishBuildEnvironment.java:71)
        at org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:36)
        at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
        at org.gradle.launcher.daemon.server.health.HintGCAfterBuild.execute(HintGCAfterBuild.java:41)
        at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
        at org.gradle.launcher.daemon.server.exec.StartBuildOrRespondWithBusy$1.run(StartBuildOrRespondWithBusy.java:50)
        at org.gradle.launcher.daemon.server.DaemonStateCoordinator$1.run(DaemonStateCoordinator.java:246)
        at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:54)
        at org.gradle.internal.concurrent.StoppableExecutorImpl$1.run(StoppableExecutorImpl.java:40)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)

This might be related to #55

Document the swimlane output

What do . | a b, etc mean in the swimlane output?

com/github/mustachejava/MustacheFactory : Unsupported major.minor version 52.0

Recently I started getting this error:

* What went wrong:
Execution failed for task ':app:forkDebug'.
> com/github/mustachejava/MustacheFactory : Unsupported major.minor version 52.0

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Does fork now require Java 8?

Config option to disable fork screenshots

A config option to disable fork screenshots would be helpful ( and a way to configuration that from the gradle plugin ).

Not available on maven central?

http://search.maven.org/#search%7Cga%7C1%7Cg%3A%22com.shazam%22

In gradle I get "Could not find" errors for the fork-gradle-plugin, and I don't see it through searching maven-central (above).

Cannot run Fork

I followed the instructions in the README.md file, and when running

./gradlew forkDebug

I get:

Configuring devices and pools failed
com.shazam.fork.pooling.NoDevicesForPoolException: No devices found.

Is there any extra configuration needed to make fork run? I am just trying to run UATs on two different emulators to save time.

Support for latest ddmlib

Fork has an issue with the ddmlib version used by the Android gradle plugin v0.14.x
In practice, this makes it look like all the tests pass ( failures don't get reported ).

Fork needs a similar fix:

square/spoon#210

Exception in thread "DeviceExecutor-1" java.lang.AbstractMethodError: com.shazam.fork.TestRunActivityWatchdog.testFailed(Lcom/android/ddmlib/testrunner/TestIdentifier;Ljava/lang/String;)V
    at com.android.ddmlib.testrunner.InstrumentationResultParser.reportResult(InstrumentationResultParser.java:461)
    at com.android.ddmlib.testrunner.InstrumentationResultParser.parseStatusCode(InstrumentationResultParser.java:415)
    at com.android.ddmlib.testrunner.InstrumentationResultParser.parse(InstrumentationResultParser.java:276)
    at com.android.ddmlib.testrunner.InstrumentationResultParser.processNewLines(InstrumentationResultParser.java:247)
    at com.android.ddmlib.MultiLineReceiver.addOutput(MultiLineReceiver.java:94)
    at com.android.ddmlib.AdbHelper.executeRemoteCommand(AdbHelper.java:521)
    at com.android.ddmlib.AdbHelper.executeRemoteCommand(AdbHelper.java:387)
    at com.android.ddmlib.Device.executeShellCommand(Device.java:565)
    at com.android.ddmlib.testrunner.RemoteAndroidTestRunner.run(RemoteAndroidTestRunner.java:263)
    at com.android.ddmlib.testrunner.RemoteAndroidTestRunner.run(RemoteAndroidTestRunner.java:248)
    at com.shazam.fork.TestRun.execute(TestRun.java:54)
    at com.shazam.fork.TestSuiteRunnerTask.runIndividualTestClass(TestSuiteRunnerTask.java:90)
    at com.shazam.fork.TestSuiteRunnerTask.run(TestSuiteRunnerTask.java:59)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)

Filter on test annotation @SmallTest, @MediumTest, @LargeTest

Spoon provides this functionality so I assumed it would be available here; am I missing something or is it not supported?

Build Failed (with gradle plugin)

build failed (with fork plugin) using the command:

./gradlew fork

Also the test report output was useless (see screenies) I got this stacktrace from the command line.
(geny motion emu)

Build passes with:

./gradlew connectedAndroidTest

:app:preBuild                                                                                            
:app:compileDebugNdk UP-TO-DATE     
:app:preDebugBuild                
:app:checkDebugManifest                
:app:prepareDebugDependencies                
:app:compileDebugAidl UP-TO-DATE     
:app:compileDebugRenderscript UP-TO-DATE      
:app:generateDebugBuildConfig UP-TO-DATE      
:app:generateDebugAssets UP-TO-DATE      
:app:mergeDebugAssets UP-TO-DATE      
:app:generateDebugResValues UP-TO-DATE      
:app:generateDebugResources UP-TO-DATE      
:app:mergeDebugResources UP-TO-DATE      
:app:processDebugManifest UP-TO-DATE      
:app:processDebugResources UP-TO-DATE      
:app:generateDebugSources UP-TO-DATE      
:core:compileJava UP-TO-DATE                                               
:core:processResources UP-TO-DATE      
:core:classes UP-TO-DATE      
:core:jar UP-TO-DATE      
:app:compileDebugJava UP-TO-DATE      
:app:unzipJacocoAgent UP-TO-DATE      
:app:instrumentDebug UP-TO-DATE                                                       
:app:preDexDebug UP-TO-DATE      
:app:dexDebug UP-TO-DATE      
:app:processDebugJavaRes UP-TO-DATE      
:app:validateDebugSigning                 
:app:packageDebug UP-TO-DATE      
:app:zipalignDebug UP-TO-DATE      
:app:assembleDebug UP-TO-DATE      
:app:compileDebugTestNdk UP-TO-DATE      
:app:preDebugTestBuild                 
:app:prepareDebugTestDependencies                 
:app:compileDebugTestAidl UP-TO-DATE      
:app:processDebugTestManifest UP-TO-DATE      
:app:compileDebugTestRenderscript UP-TO-DATE      
:app:generateDebugTestBuildConfig UP-TO-DATE      
:app:generateDebugTestAssets UP-TO-DATE      
:app:mergeDebugTestAssets UP-TO-DATE      
:app:generateDebugTestResValues UP-TO-DATE      
:app:generateDebugTestResources UP-TO-DATE      
:app:mergeDebugTestResources UP-TO-DATE      
:app:processDebugTestResources UP-TO-DATE      
:app:generateDebugTestSources UP-TO-DATE      
:app:compileDebugTestJava UP-TO-DATE      
:app:preDexDebugTest UP-TO-DATE      
:app:dexDebugTest UP-TO-DATE      
:app:processDebugTestJavaRes UP-TO-DATE      
:app:packageDebugTest UP-TO-DATE      
:app:assembleDebugTest UP-TO-DATE      
:app:forkDebug                 
Use -Dandroid.test.classes=REGEX to specify a pattern for the classes/packages to run
Use -Dfork.tablet=(true|false) to configure pools depending on their manufacturer's 'tablet' flag (ro.build.characteristics)
Use -Dfork.pool.POOL_NAME=(Serial','?)* to add devices with a given serial to a pool with given name,e.g. hdpi=01234567,abcdefgh
Use -Dfork.computed.STRATEGY=(PoolName=LowerBound','?)* to automatically create pools based on device characteristics
STRATEGY:=sw - by smallest width, e.g.phablet=0,tablet=720
STRATEGY:=api - by api, e.g. gingerbread_and_earlier=0,honeycomb_and_later=11)
Use -Dfork.eachdevice=(true|false) to create a pool per device (a.k.a. Spoon mode). This is the default behaviour.
Use -Dfork.excluded.serial=(Serial','?)* to exclude specific devices from running any tests
Use -Dfork.report.title=Title to specify a title for the generated report
Use -Dfork.report.subtitle=Subitle to specify a subtitle for the generated report
Error while Fork runner was executing
java.lang.IllegalArgumentException: Array is empty
        at org.apache.commons.lang3.BooleanUtils.and(BooleanUtils.java:929)
        at com.shazam.fork.summary.OutcomeAggregator.and(OutcomeAggregator.java:59)
        at com.shazam.fork.summary.OutcomeAggregator.access$100(OutcomeAggregator.java:27)
        at com.shazam.fork.summary.OutcomeAggregator$1.apply(OutcomeAggregator.java:42)
        at com.shazam.fork.summary.OutcomeAggregator$1.apply(OutcomeAggregator.java:36)
        at com.google.common.collect.Iterators$8.transform(Iterators.java:794)
        at com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
        at java.util.AbstractCollection.toArray(AbstractCollection.java:195)
        at com.shazam.fork.summary.OutcomeAggregator.and(OutcomeAggregator.java:59)
        at com.shazam.fork.summary.OutcomeAggregator.aggregate(OutcomeAggregator.java:32)
        at com.shazam.fork.ForkRunner.run(ForkRunner.java:115)
        at com.shazam.fork.Fork.run(Fork.java:46)
        at com.shazam.fork.Fork$run.call(Unknown Source)
        at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:112)
        at com.shazam.fork.gradle.ForkRunTask.runFork(ForkRunTask.groovy:62)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:63)
        at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.doExecute(AnnotationProcessingTaskFactory.java:218)
        at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.execute(AnnotationProcessingTaskFactory.java:211)
        at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.execute(AnnotationProcessingTaskFactory.java:200)
        at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:579)
        at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:562)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:80)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:61)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:46)
        at org.gradle.api.internal.tasks.execution.PostExecutionAnalysisTaskExecuter.execute(PostExecutionAnalysisTaskExecuter.java:35)
        at org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter.execute(SkipUpToDateTaskExecuter.java:64)
        at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:58)
        at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:42)
        at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:52)
        at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:53)
        at org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter.execute(ExecuteAtMostOnceTaskExecuter.java:43)
        at org.gradle.api.internal.AbstractTask.executeWithoutThrowingTaskFailure(AbstractTask.java:305)
        at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.executeTask(AbstractTaskPlanExecutor.java:79)
        at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.processTask(AbstractTaskPlanExecutor.java:63)
        at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.run(AbstractTaskPlanExecutor.java:51)
        at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor.process(DefaultTaskPlanExecutor.java:23)
        at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter.execute(DefaultTaskGraphExecuter.java:88)
        at org.gradle.execution.SelectedTaskExecutionAction.execute(SelectedTaskExecutionAction.java:29)
        at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:62)
        at org.gradle.execution.DefaultBuildExecuter.access$200(DefaultBuildExecuter.java:23)
        at org.gradle.execution.DefaultBuildExecuter$2.proceed(DefaultBuildExecuter.java:68)
        at org.gradle.execution.DryRunBuildExecutionAction.execute(DryRunBuildExecutionAction.java:32)
        at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:62)
        at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:55)
        at org.gradle.initialization.DefaultGradleLauncher.doBuildStages(DefaultGradleLauncher.java:149)
        at org.gradle.initialization.DefaultGradleLauncher.doBuild(DefaultGradleLauncher.java:106)
        at org.gradle.initialization.DefaultGradleLauncher.run(DefaultGradleLauncher.java:86)
        at org.gradle.launcher.exec.InProcessBuildActionExecuter$DefaultBuildController.run(InProcessBuildActionExecuter.java:80)
        at org.gradle.launcher.cli.ExecuteBuildAction.run(ExecuteBuildAction.java:33)
        at org.gradle.launcher.cli.ExecuteBuildAction.run(ExecuteBuildAction.java:24)
        at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:36)
        at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:26)
        at org.gradle.launcher.daemon.server.exec.ExecuteBuild.doBuild(ExecuteBuild.java:47)
        at org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:34)
        at org.gradle.launcher.daemon.server.exec.DaemonCommandExecution.proceed(DaemonCommandExecution.java:119)
        at org.gradle.launcher.daemon.server.exec.WatchForDisconnection.execute(WatchForDisconnection.java:35)
        at org.gradle.launcher.daemon.server.exec.DaemonCommandExecution.proceed(DaemonCommandExecution.java:119)
        at org.gradle.launcher.daemon.server.exec.ResetDeprecationLogger.execute(ResetDeprecationLogger.java:24)
        at org.gradle.launcher.daemon.server.exec.DaemonCommandExecution.proceed(DaemonCommandExecution.java:119)
        at org.gradle.launcher.daemon.server.exec.StartStopIfBuildAndStop.execute(StartStopIfBuildAndStop.java:33)
        at org.gradle.launcher.daemon.server.exec.DaemonCommandExecution.proceed(DaemonCommandExecution.java:119)
        at org.gradle.launcher.daemon.server.exec.ForwardClientInput$2.call(ForwardClientInput.java:71)
        at org.gradle.launcher.daemon.server.exec.ForwardClientInput$2.call(ForwardClientInput.java:69)
        at org.gradle.util.Swapper.swap(Swapper.java:38)
        at org.gradle.launcher.daemon.server.exec.ForwardClientInput.execute(ForwardClientInput.java:69)
        at org.gradle.launcher.daemon.server.exec.DaemonCommandExecution.proceed(DaemonCommandExecution.java:119)
        at org.gradle.launcher.daemon.server.exec.LogToClient.doBuild(LogToClient.java:60)
        at org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:34)
        at org.gradle.launcher.daemon.server.exec.DaemonCommandExecution.proceed(DaemonCommandExecution.java:119)
        at org.gradle.launcher.daemon.server.exec.EstablishBuildEnvironment.doBuild(EstablishBuildEnvironment.java:70)
        at org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:34)
        at org.gradle.launcher.daemon.server.exec.DaemonCommandExecution.proceed(DaemonCommandExecution.java:119)
        at org.gradle.launcher.daemon.server.exec.DaemonHygieneAction.execute(DaemonHygieneAction.java:39)
        at org.gradle.launcher.daemon.server.exec.DaemonCommandExecution.proceed(DaemonCommandExecution.java:119)
        at org.gradle.launcher.daemon.server.exec.StartBuildOrRespondWithBusy$1.run(StartBuildOrRespondWithBusy.java:46)
        at org.gradle.launcher.daemon.server.DaemonStateCoordinator$1.run(DaemonStateCoordinator.java:246)
        at org.gradle.internal.concurrent.DefaultExecutorFactory$StoppableExecutorImpl$1.run(DefaultExecutorFactory.java:64)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
:app:forkDebug FAILED          

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':app:forkDebug'.
> Tests failed! See /Users/Blundell/MyApp/app/build/fork/debug/html/index.html

* Try:        
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED  

Total time: 20.536 secs

I would have expected it to work with all default params. It saying array is empty makes me think it might need a argument? or looking at the source https://github.com/shazam/fork/blob/master/src/main/java/com/shazam/fork/summary/OutcomeAggregator.java#L58 it's looking for test outcomes, so has it failed to pull the results from the device?

NumberFormatException: For input string: "Unspecified API Level"

This is an intermittent error ( it worked when I just re-ran it ).
It might have been an ADB hiccup.

Use -Dfork.report.subtitle=Subitle to specify a subtitle for the generated report
Error while Fork runner was executing
java.lang.NumberFormatException: For input string: "Unspecified API Level"
    at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
    at java.lang.Integer.parseInt(Integer.java:492)
    at java.lang.Integer.parseInt(Integer.java:527)
    at com.shazam.fork.pooling.PoolByApi.getParameter(PoolByApi.java:29)
    at com.shazam.fork.pooling.SmallestComputedPoolsSelector.poolForDevice(SmallestComputedPoolsSelector.java:37)
    at com.shazam.fork.pooling.ComputedDevicePoolLoader.createComputedPools(ComputedDevicePoolLoader.java:53)
    at com.shazam.fork.pooling.ComputedDevicePoolLoader.loadPools(ComputedDevicePoolLoader.java:45)
    at com.shazam.fork.pooling.CompositeDevicePoolLoader.loadPools(CompositeDevicePoolLoader.java:38)
    at com.shazam.fork.ForkRunner.run(ForkRunner.java:76)
    at com.shazam.fork.Fork.run(Fork.java:46)
    at com.shazam.fork.Fork$run.call(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:112)
    at com.shazam.fork.gradle.ForkRunTask.runFork(ForkRunTask.groovy:62)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:63)

Add fast fail option

It would be nice to have an option to fast fail a build.

i.e. if anything goes wrong or any test fails, just stop immediately and fail the build, don't bother running the rest of the tests.

Simplify package configuration: test package should be the same as app package

By default Android Studio creates tests in the same package as the package of the main application, e.g. com.example.

Even if the eventual package of the test application has the test suffix (com.example.test), the test classes are still in com.example.

Fork should be modified to follow the default AS behaviour.

Add flag for configuring the condition when keeping test assets

It would be useful to add a flag that tells Fork to keep or create assets (screenshots/videos). Something like:

never (also switch off screenshots, videos)
failed tests only
always (passed & failed)

Extend flakiness reporter to report on device flakiness

Gradle plugin configuration instructions do not seem to work

See my question on Stackoverflow here about getting an HTTP 401 error when trying to use the Fork Gradle plugin:
http://stackoverflow.com/q/27259442/112705

The Fork plugin does not seem to be hosted in Maven Central:
http://search.maven.org/#search|ga|1|shazam

I also tried adding jcenter() to my repositories list, but that didn't help. I'm using Android Studio v0.8.9.

No tag for 1.1.1 release

Move runtime configuration to the gradle plugin DSL, instead of env. variables

Device pool fails to run all the tests

I had this problem (INSTALL_FAILED_UID_CHANGED):
https://code.google.com/p/android/issues/detail?id=79716

Which caused the app to not be installed.

So Fork gave this error:
"Test run failed: Unable to find instrumentation target package:"

Because of the above error forked failed to run most of the tests in that pool, but still reported SUCCESS as the build status.

Fork should fail the overall build status in this case and really any time there is an issue that prevents a test case from being run, the build should fail. ( this could be an optional setting that is configurable )

Device serials with special characters cause exceptions on Windows

If you have device names (emulators do this, mostly) with special characters (e.g. 192.168.1.103:5555), fork will throw an exception. This is related to the way it handles files/folders that contain the device name in their name(s).

Improve summary pages

Make summary pages prettier.

1.0.0 gradle plugin: Some warning output after applying fork plugin

I also have the crashlytics plugin applied and it appears as if fork and crashlytics are stomping each other.

log4j:ERROR A "org.apache.log4j.ConsoleAppender" object is not assignable to a "com.crashlytics.reloc.org.apache.log4j.Appender" variable.
log4j:ERROR The class "com.crashlytics.reloc.org.apache.log4j.Appender" was loaded by
log4j:ERROR [java.net.URLClassLoader@67e1ff2c] whereas object of type
log4j:ERROR "org.apache.log4j.ConsoleAppender" was loaded by [org.gradle.internal.classloader.MutableURLClassLoader@34c45dca].
log4j:ERROR Could not instantiate appender named "stdout".

Fork fails to run on Gradle v1.0.0

Fork was working fine with Gradle v0.12. After updating to Gradle v1.0.0 (as part of updating to Android Studio v1.0), my builds now fail with the error below.

* What went wrong:
A problem occurred configuring project ':CookBrite'.
> Could not find property 'outputFile' on com.android.build.gradle.internal.api.ApplicationVariantImpl_Decorated@50434e01.

I was trying to run:

./gradlew forkDevDebugTest -Dandroid.test.classes=com.cookbrite.step1_unit.MySingleTestClass

This bug does not affect Spoon, so you may be able to check their code for a solution? I wasn't able to spot it by checking their most recent commits.

Implement screenshot capture

Our current solution of screenshots depends on a third party library. It would be better to avoid it and use the system commands for taking screenshots, probably once at the end of the test.

Linked to video capture: #1

Unable to include fork plugin in my project

Hi,

I modified my build.gradle file to include following two changes and then ran "./gradlew clean" command.
buildscript {
dependencies {
...
classpath 'com.shazam.fork:fork-gradle-plugin:1.1.1-SNAPSHOT'
}
repositories() {
mavenLocal();
maven {
...
}
mavenCentral();
flatDir {
dirs 'libs'
}
jcenter()
}
}
...
apply plugin: 'fork'
....
}
Am seeing the following error:
FAILURE: Build failed with an exception.

What went wrong:
A problem occurred configuring project ':App'.

Could not resolve all dependencies for configuration ':App:classpath'.
Could not find com.shazam.fork:fork-gradle-plugin:1.1.1-SNAPSHOT.

Can you please let us know what am i doing wrong here?

Thanks,
Pavan

Allow configuration in a Gradle file

It would be nice to be able to configure via the gradle file, something like:

fork {
    testClassRegex = "*.Important" 
    apiPools = { 
                  "old": 0,
                "new": 15
        }
}

Problem with missing ddmlib method

Getting this error after updating gradle to 2.9 and gradle plugin to 1.5.0

Exception in thread "DeviceExecutor-0" java.lang.NoSuchMethodError: com.android.ddmlib.IDevice.installPackage(Ljava/lang/String;Z[Ljava/lang/String;)Ljava/lang/String;
at com.shazam.fork.system.adb.Installer$1.run(Installer.java:53)
at com.shazam.fork.system.adb.Installer.tryThrice(Installer.java:77)
at com.shazam.fork.system.adb.Installer.reinstall(Installer.java:47)
at com.shazam.fork.system.adb.Installer.prepareInstallation(Installer.java:41)
at com.shazam.fork.runner.DeviceTestRunner.run(DeviceTestRunner.java:59)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)

gradle plugin update may have brought a newer version of ddmlib that is not compatible with one used by fork?

Test class scanner should only scan in the instrumentation package

Currently scans in the application package (i.e. scan in com.example.test, not com.example).

Video capture

What is the story for Screenshot support?

It looks like fork is using https://github.com/rtyley/android-screenshot-lib

It would be great if it picked-up screenshots taken by Spoon.screenshot() .

When using fork-gradle-plugin guava version 18 causes android lint to crash

Guava 17.0 works just fine.

fork 1.1.0 pool view appears broken

After opening the report the pool details view is empty. I can see all my tests in the top level view though.

Maven central update requested

Having snapshots is really nice, but to use fork for CI we need to be locked down on a stable version. Would you guys mind publishing the current state as a minor version?

ignore devices that are offline or unauthorized

Fork should ignore devices that are offline or unauthorized, not fail the build.

Or have an option for "only use 'online' devices"

This is what you get with 0.10.0 for offline devices.

com.android.ddmlib.AdbCommandRejectedException: device offline
    at com.android.ddmlib.AdbHelper.setDevice(AdbHelper.java:857)
    at com.android.ddmlib.AdbHelper.executeRemoteCommand(AdbHelper.java:460)
    at com.android.ddmlib.AdbHelper.executeRemoteCommand(AdbHelper.java:390)
    at com.android.ddmlib.AdbHelper.executeRemoteCommand(AdbHelper.java:359)
    at com.android.ddmlib.Device.executeShellCommand(Device.java:557)
    at com.shazam.fork.geometry.RegexDisplayGeometryRetrievalStrategy.getDisplayGeometry(RegexDisplayGeometryRetrievalStrategy.java:54)
    at com.shazam.fork.geometry.RegexDisplayGeometryRetrievalStrategy.retrieveGeometry(RegexDisplayGeometryRetrievalStrategy.java:45)
    at com.shazam.fork.geometry.DeviceGeometryRetriever.detectGeometry(DeviceGeometryRetriever.java:41)
    at com.shazam.fork.system.DeviceLoader.loadDeviceCharacteristics(DeviceLoader.java:83)
    at com.shazam.fork.system.DeviceLoader.loadDevices(DeviceLoader.java:53)
    at com.shazam.fork.ForkRunner.run(ForkRunner.java:70)
    at com.shazam.fork.Fork.run(Fork.java:46)
    at com.shazam.fork.Fork$run.call(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:112)
    at com.shazam.fork.gradle.ForkRunTask.runFork(ForkRunTask.groovy:62)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:63)
    at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.doExecute(AnnotationProcessingTaskFactory.java:218)
    at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.execute(AnnotationProcessingTaskFactory.java:211)
    at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.execute(AnnotationProcessingTaskFactory.java:200)
    at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:585)
    at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:568)
    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:80)
    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:61)
    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:46)
    at org.gradle.api.internal.tasks.execution.PostExecutionAnalysisTaskExecuter.execute(PostExecutionAnalysisTaskExecuter.java:35)
    at org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter.execute(SkipUpToDateTaskExecuter.java:64)
    at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:58)
    at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:42)
    at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:52)
    at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:53)
    at org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter.execute(ExecuteAtMostOnceTaskExecuter.java:43)
    at org.gradle.api.internal.AbstractTask.executeWithoutThrowingTaskFailure(AbstractTask.java:306)
    at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.executeTask(AbstractTaskPlanExecutor.java:79)
    at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.processTask(AbstractTaskPlanExecutor.java:63)
    at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.run(AbstractTaskPlanExecutor.java:51)
    at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor.process(DefaultTaskPlanExecutor.java:23)
    at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter.execute(DefaultTaskGraphExecuter.java:88)
    at org.gradle.execution.SelectedTaskExecutionAction.execute(SelectedTaskExecutionAction.java:29)
    at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:62)
    at org.gradle.execution.DefaultBuildExecuter.access$200(DefaultBuildExecuter.java:23)
    at org.gradle.execution.DefaultBuildExecuter$2.proceed(DefaultBuildExecuter.java:68)
    at org.gradle.execution.DryRunBuildExecutionAction.execute(DryRunBuildExecutionAction.java:32)
    at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:62)
    at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:55)
    at org.gradle.initialization.DefaultGradleLauncher.doBuildStages(DefaultGradleLauncher.java:149)
    at org.gradle.initialization.DefaultGradleLauncher.doBuild(DefaultGradleLauncher.java:106)
    at org.gradle.initialization.DefaultGradleLauncher.run(DefaultGradleLauncher.java:86)
    at org.gradle.launcher.exec.InProcessBuildActionExecuter$DefaultBuildController.run(InProcessBuildActionExecuter.java:80)
    at org.gradle.launcher.cli.ExecuteBuildAction.run(ExecuteBuildAction.java:33)
    at org.gradle.launcher.cli.ExecuteBuildAction.run(ExecuteBuildAction.java:24)
    at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:36)
    at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:26)
    at org.gradle.launcher.cli.RunBuildAction.run(RunBuildAction.java:51)
    at org.gradle.internal.Actions$RunnableActionAdapter.execute(Actions.java:169)
    at org.gradle.launcher.cli.CommandLineActionFactory$ParseAndBuildAction.execute(CommandLineActionFactory.java:237)
    at org.gradle.launcher.cli.CommandLineActionFactory$ParseAndBuildAction.execute(CommandLineActionFactory.java:210)
    at org.gradle.launcher.cli.JavaRuntimeValidationAction.execute(JavaRuntimeValidationAction.java:35)
    at org.gradle.launcher.cli.JavaRuntimeValidationAction.execute(JavaRuntimeValidationAction.java:24)
    at org.gradle.launcher.cli.CommandLineActionFactory$WithLogging.execute(CommandLineActionFactory.java:206)
    at org.gradle.launcher.cli.CommandLineActionFactory$WithLogging.execute(CommandLineActionFactory.java:169)
    at org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:33)
    at org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:22)
    at org.gradle.launcher.Main.doAction(Main.java:33)
    at org.gradle.launcher.bootstrap.EntryPoint.run(EntryPoint.java:45)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.gradle.launcher.bootstrap.ProcessBootstrap.runNoExit(ProcessBootstrap.java:54)
    at org.gradle.launcher.bootstrap.ProcessBootstrap.run(ProcessBootstrap.java:35)
    at org.gradle.launcher.GradleMain.main(GradleMain.java:23)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.gradle.wrapper.BootstrapMainStarter.start(BootstrapMainStarter.java:30)
    at org.gradle.wrapper.WrapperExecutor.execute(WrapperExecutor.java:127)
    at org.gradle.wrapper.GradleWrapperMain.main(GradleWrapperMain.java:56)

Fix progress counter at the console output

Document JUnit4 tests marked as ignored

We currently document tests marked as suppressed, but we need to do this for JUnit 4 tests.

Allow configuration of TEST_CLASS_PATTERN = Pattern.compile("^((?!Abstract).)*Test;$");

This heuristic doesn't work for us...

Preferably it would crawl the type hierarchy and look for the Junit3 TestCase.

As a minimum, it would be nice if we could configure this regex ( we have test classes that don't end in "Test".

Version test output so that other projects can create historic view of test results

Probably the best way to do this is to split a module for reports from the existing com.shazam.fork.summary package. It doesn't need to know anything about the runtime's state, but just parse output directories and produce individual or historic reports.

Test cases are ignored when method name contains characters like Chinese

Hi, since upgrade fork to the new release 1.3.0, My test cases are ignored to execute, i've notice that only the method name in Chinese is not executed, any ideas about this?
I'm using 2.10 and AS plugin 2.1.0-beta1, any suggestions will be appreciated:)

I feel it is because of these file changes, Do we changed a way to get the test method ?

TestCaseEvent.java
TestRun.java
TestRunFactory.java
TestRunParameters.java

Enable code coverage.

As far as I can tell Fork does not support code coverage. I think there should be an optional configuration for enabling code coverage and specifying the directory to which the coverage files should be output to. I can submit a pull request to do so, just wanted to be sure I wasn't missing something or if there was a reason for not supporting the generation of the coverage files.

Add fine-grained control for permissions on Marshmallow devices

This will most likely need a way to control what permissions get granted per test.

Tidy up file structure of Fork's output

Create -alpha releases

It would be helpful to create -alpha1, -alpha2 releases and git tags.

1.1.0-SNAPSHOT might be different in the morning than in the evening which can make troubleshooting harder, it would be nice to be able to control when we jump to the next version.

Also, by having git tags tied to specific alpha releases, it helps us debug if we want to look at the source code of fork.

Fork reports no devices are found after multiple runs

Even if adb devices reports one or more devices are found, when executing Fork consecutively, it reports that no devices are found.

This only happens when someone is using the Gradle daemon and can temporarily work around it by stopping it (./gradlew --stop).

Allow changing the order of tests that are run

It appears as if fork runs tests alphabetically based on package+class name.

My thinking is similar to how TeamCity will run failing tests from the last build as the first tests of the current build to speedup feedback.

Perhaps tests that match a provided regular expression (eg failingTest1,failingTest2) will be prioritized before the other tests. Coupled with the "fail fast" feature requested elsewhere this could help decrease the load on environments with shared device resources.

NoSuchMethodError: org.apache.commons.io.FileUtils.getFile(Ljava/io/File;[Ljava/lang/String;)Ljava/io/File;

When trying to run forkDebug from com.shazam.fork:fork-gradle-plugin:1.0.0-RC1-SNAPSHOT, I get the following NoSuchMethodError: org.apache.commons.io.FileUtils.getFile not found.

:express:forkDebug FAILED
:express:forkDebug (Thread[main,5,main]) completed. Took 0.048 secs.

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':express:forkDebug'.
> org.apache.commons.io.FileUtils.getFile(Ljava/io/File;[Ljava/lang/String;)Ljava/io/File;

* Try:
Run with --debug option to get more log output.

* Exception is:
org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':express:forkDebug'.
    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:69)
    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:46)
    at org.gradle.api.internal.tasks.execution.PostExecutionAnalysisTaskExecuter.execute(PostExecutionAnalysisTaskExecuter.java:35)
    at org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter.execute(SkipUpToDateTaskExecuter.java:64)
    at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:58)
    at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:42)
    at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:52)
    at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:53)
    at org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter.execute(ExecuteAtMostOnceTaskExecuter.java:43)
    at org.gradle.api.internal.AbstractTask.executeWithoutThrowingTaskFailure(AbstractTask.java:305)
    at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.executeTask(AbstractTaskPlanExecutor.java:79)
    at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.processTask(AbstractTaskPlanExecutor.java:63)
    at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.run(AbstractTaskPlanExecutor.java:51)
    at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor.process(DefaultTaskPlanExecutor.java:23)
    at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter.execute(DefaultTaskGraphExecuter.java:88)
    at org.gradle.execution.SelectedTaskExecutionAction.execute(SelectedTaskExecutionAction.java:29)
    at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:62)
    at org.gradle.execution.DefaultBuildExecuter.access$200(DefaultBuildExecuter.java:23)
    at org.gradle.execution.DefaultBuildExecuter$2.proceed(DefaultBuildExecuter.java:68)
    at org.gradle.execution.DryRunBuildExecutionAction.execute(DryRunBuildExecutionAction.java:32)
    at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:62)
    at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:55)
    at org.gradle.initialization.DefaultGradleLauncher.doBuildStages(DefaultGradleLauncher.java:149)
    at org.gradle.initialization.DefaultGradleLauncher.doBuild(DefaultGradleLauncher.java:106)
    at org.gradle.initialization.DefaultGradleLauncher.run(DefaultGradleLauncher.java:86)
    at org.gradle.launcher.exec.InProcessBuildActionExecuter$DefaultBuildController.run(InProcessBuildActionExecuter.java:80)
    at org.gradle.launcher.cli.ExecuteBuildAction.run(ExecuteBuildAction.java:33)
    at org.gradle.launcher.cli.ExecuteBuildAction.run(ExecuteBuildAction.java:24)
    at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:36)
    at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:26)
    at org.gradle.launcher.cli.RunBuildAction.run(RunBuildAction.java:51)
    at org.gradle.internal.Actions$RunnableActionAdapter.execute(Actions.java:171)
    at org.gradle.launcher.cli.CommandLineActionFactory$ParseAndBuildAction.execute(CommandLineActionFactory.java:237)
    at org.gradle.launcher.cli.CommandLineActionFactory$ParseAndBuildAction.execute(CommandLineActionFactory.java:210)
    at org.gradle.launcher.cli.JavaRuntimeValidationAction.execute(JavaRuntimeValidationAction.java:35)
    at org.gradle.launcher.cli.JavaRuntimeValidationAction.execute(JavaRuntimeValidationAction.java:24)
    at org.gradle.launcher.cli.CommandLineActionFactory$WithLogging.execute(CommandLineActionFactory.java:206)
    at org.gradle.launcher.cli.CommandLineActionFactory$WithLogging.execute(CommandLineActionFactory.java:169)
    at org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:33)
    at org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:22)
    at org.gradle.launcher.Main.doAction(Main.java:33)
    at org.gradle.launcher.bootstrap.EntryPoint.run(EntryPoint.java:45)
    at org.gradle.launcher.bootstrap.ProcessBootstrap.runNoExit(ProcessBootstrap.java:54)
    at org.gradle.launcher.bootstrap.ProcessBootstrap.run(ProcessBootstrap.java:35)
    at org.gradle.launcher.GradleMain.main(GradleMain.java:23)
    at org.gradle.wrapper.BootstrapMainStarter.start(BootstrapMainStarter.java:33)
    at org.gradle.wrapper.WrapperExecutor.execute(WrapperExecutor.java:130)
    at org.gradle.wrapper.GradleWrapperMain.main(GradleWrapperMain.java:48)
Caused by: java.lang.NoSuchMethodError: org.apache.commons.io.FileUtils.getFile(Ljava/io/File;[Ljava/lang/String;)Ljava/io/File;
    at com.shazam.fork.system.adb.SpoonUtils.initAdb(SpoonUtils.java:35)
    at com.shazam.fork.injector.system.AndroidDebugBridgeInjector.<clinit>(AndroidDebugBridgeInjector.java:21)
    at com.shazam.fork.injector.pooling.DeviceLoaderInjector.deviceLoader(DeviceLoaderInjector.java:24)
    at com.shazam.fork.injector.pooling.PoolLoaderInjector.poolLoader(PoolLoaderInjector.java:21)
    at com.shazam.fork.injector.ForkRunnerInjector.forkRunner(ForkRunnerInjector.java:39)
    at com.shazam.fork.Fork.<init>(Fork.java:41)
    at com.shazam.fork.ForkBuilder.build(ForkBuilder.java:153)
    at com.shazam.fork.ForkBuilder$build$6.call(Unknown Source)
    at com.shazam.fork.gradle.ForkRunTask.runFork(ForkRunTask.groovy:70)
    at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:63)
    at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.doExecute(AnnotationProcessingTaskFactory.java:218)
    at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.execute(AnnotationProcessingTaskFactory.java:211)
    at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.execute(AnnotationProcessingTaskFactory.java:200)
    at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:579)
    at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:562)
    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:80)
    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:61)
    ... 47 more

Hard stop for when the test runner does not timeout itself

Currently we have some external watchdog tasks which seem to be killing normal test runs. That needs to be replaced with something more sane.

java.lang.IllegalArgumentException: Array is empty

I'm using 'com.shazam.fork:fork-gradle-plugin:1.0.0-RC1-SNAPSHOT', and running

./gradlew forkDebug -Dfork.test.size=medium -Dfork.pool.a=192.168.56.102:5555,192.168.56.101:5555 -Dfork.eachdevice=false

results in:

Error while Fork runner was executing
java.lang.IllegalArgumentException: Array is empty
    at org.apache.commons.lang3.BooleanUtils.and(BooleanUtils.java:944)
    at com.shazam.fork.summary.OutcomeAggregator.and(OutcomeAggregator.java:58)
    at com.shazam.fork.summary.OutcomeAggregator.access$100(OutcomeAggregator.java:27)
    at com.shazam.fork.summary.OutcomeAggregator$1.apply(OutcomeAggregator.java:42)
    at com.shazam.fork.summary.OutcomeAggregator$1.apply(OutcomeAggregator.java:36)
    at com.google.common.collect.Iterators$8.transform(Iterators.java:794)
    at com.google.common.collect.TransformedIterator.next(TransformedIterator.java:48)
    at java.util.AbstractCollection.toArray(AbstractCollection.java:195)
    at com.shazam.fork.summary.OutcomeAggregator.and(OutcomeAggregator.java:58)
    at com.shazam.fork.summary.OutcomeAggregator.aggregate(OutcomeAggregator.java:32)
    at com.shazam.fork.summary.HtmlConverters.toHtmlSummary(HtmlConverters.java:36)
    at com.shazam.fork.summary.HtmlSummaryPrinter.print(HtmlSummaryPrinter.java:74)
    at com.shazam.fork.summary.CompositeSummaryPrinter.print(CompositeSummaryPrinter.java:14)
    at com.shazam.fork.summary.ReportGeneratorHook.generateReportOnlyOnce(ReportGeneratorHook.java:72)
    at com.shazam.fork.ForkRunner.run(ForkRunner.java:87)
    at com.shazam.fork.Fork.run(Fork.java:50)
    at com.shazam.fork.Fork$run.call(Unknown Source)
    at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:112)
    at com.shazam.fork.gradle.ForkRunTask.runFork(ForkRunTask.groovy:70)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:63)
    at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.doExecute(AnnotationProcessingTaskFactory.java:218)
    at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.execute(AnnotationProcessingTaskFactory.java:211)
    at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.execute(AnnotationProcessingTaskFactory.java:200)
    at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:579)
    at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:562)
    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:80)
    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:61)
    at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:46)
    at org.gradle.api.internal.tasks.execution.PostExecutionAnalysisTaskExecuter.execute(PostExecutionAnalysisTaskExecuter.java:35)
    at org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter.execute(SkipUpToDateTaskExecuter.java:64)
    at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:58)
    at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:42)
    at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:52)
    at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:53)
    at org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter.execute(ExecuteAtMostOnceTaskExecuter.java:43)
    at org.gradle.api.internal.AbstractTask.executeWithoutThrowingTaskFailure(AbstractTask.java:305)
    at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.executeTask(AbstractTaskPlanExecutor.java:79)
    at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.processTask(AbstractTaskPlanExecutor.java:63)
    at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.run(AbstractTaskPlanExecutor.java:51)
    at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor.process(DefaultTaskPlanExecutor.java:23)
    at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter.execute(DefaultTaskGraphExecuter.java:88)
    at org.gradle.execution.SelectedTaskExecutionAction.execute(SelectedTaskExecutionAction.java:29)
    at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:62)
    at org.gradle.execution.DefaultBuildExecuter.access$200(DefaultBuildExecuter.java:23)
    at org.gradle.execution.DefaultBuildExecuter$2.proceed(DefaultBuildExecuter.java:68)
    at org.gradle.execution.DryRunBuildExecutionAction.execute(DryRunBuildExecutionAction.java:32)
    at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:62)
    at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:55)
    at org.gradle.initialization.DefaultGradleLauncher.doBuildStages(DefaultGradleLauncher.java:149)
    at org.gradle.initialization.DefaultGradleLauncher.doBuild(DefaultGradleLauncher.java:106)
    at org.gradle.initialization.DefaultGradleLauncher.run(DefaultGradleLauncher.java:86)
    at org.gradle.launcher.exec.InProcessBuildActionExecuter$DefaultBuildController.run(InProcessBuildActionExecuter.java:80)
    at org.gradle.launcher.cli.ExecuteBuildAction.run(ExecuteBuildAction.java:33)
    at org.gradle.launcher.cli.ExecuteBuildAction.run(ExecuteBuildAction.java:24)
    at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:36)
    at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:26)
    at org.gradle.launcher.cli.RunBuildAction.run(RunBuildAction.java:51)
    at org.gradle.internal.Actions$RunnableActionAdapter.execute(Actions.java:171)
    at org.gradle.launcher.cli.CommandLineActionFactory$ParseAndBuildAction.execute(CommandLineActionFactory.java:237)
    at org.gradle.launcher.cli.CommandLineActionFactory$ParseAndBuildAction.execute(CommandLineActionFactory.java:210)
    at org.gradle.launcher.cli.JavaRuntimeValidationAction.execute(JavaRuntimeValidationAction.java:35)
    at org.gradle.launcher.cli.JavaRuntimeValidationAction.execute(JavaRuntimeValidationAction.java:24)
    at org.gradle.launcher.cli.CommandLineActionFactory$WithLogging.execute(CommandLineActionFactory.java:206)
    at org.gradle.launcher.cli.CommandLineActionFactory$WithLogging.execute(CommandLineActionFactory.java:169)
    at org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:33)
    at org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:22)
    at org.gradle.launcher.Main.doAction(Main.java:33)
    at org.gradle.launcher.bootstrap.EntryPoint.run(EntryPoint.java:45)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.gradle.launcher.bootstrap.ProcessBootstrap.runNoExit(ProcessBootstrap.java:54)
    at org.gradle.launcher.bootstrap.ProcessBootstrap.run(ProcessBootstrap.java:35)
    at org.gradle.launcher.GradleMain.main(GradleMain.java:23)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.gradle.wrapper.BootstrapMainStarter.start(BootstrapMainStarter.java:33)
    at org.gradle.wrapper.WrapperExecutor.execute(WrapperExecutor.java:130)
    at org.gradle.wrapper.GradleWrapperMain.main(GradleWrapperMain.java:48)

Also, Sonatype seems to be out of sync with development here. Would you guys mind updating it?

Some device names cause problems with relative paths

For instance my genymotion device name comes up as:

$ adb devices
List of devices attached
192.168.56.101:5555 device

So inside the pool the generated relative path causes a totally expected issue.

<a href="192.168.56.101:5555/com.foo.test.Test__test.html" data-original-title="" title="">
    LXSearchParamsTest : Required Params Filled
</a>

On the main page it works fine because the generated href is:

<a href="pools/192.168.56.101:5555/com.foo.test.Test__test.html" id="testMethod" data-original-title="Test" data-content="Test (Google_Nexus_5_-_5.0.0_-_API_21_-_1080x1920) [192.168.56.101:5555]">
    &nbsp;
</a>

Sort test order in the HTML report

Sort test order in the HTML report, so that each device pool shows the tests in the same order.

This will help identify patterns of broken tests ( vertical red lines across device pools ).

Fork marks build as failed, even when all tests have passed.

Fork often fails the build even when all the tests pass.

This is using 0.10.0. ( we can't use 0.11.0, because the screenshot stuff is causing problems and it can't be disabled. )

This happens frequently with 20+ devices attached.

Pools remaining: 0
  0 E    0 F   123 P: older=api0-17
  0 E    0 F   123 P: Newer=api18-up
No suppressed tests.
Device  pending after 120 seconds of inactivity
Apparently, no devices pending. Aborting in 15sec