Code Monkey home page Code Monkey logo

jmh-gradle-plugin's Introduction

JMH Gradle Plugin

Build Status (travis) Coverage Status (coveralls) Download Apache License 2

This plugin integrates the JMH micro-benchmarking framework with Gradle.

Usage

Build script snippet for use in all Gradle versions:

build.gradle
buildscript {
  repositories {
    jcenter()
    maven {
      url "https://plugins.gradle.org/m2/"
    }
  }
  dependencies {
    classpath "me.champeau.gradle:jmh-gradle-plugin:0.4.8"
  }
}

apply plugin: "me.champeau.gradle.jmh"

Build script snippet for new, incubating, plugin mechanism introduced in Gradle 2.1:

build.gradle
plugins {
  id "me.champeau.gradle.jmh" version "0.4.8"
}

What plugin version to use?

Gradle Minimal plugin version

5.1

0.4.8

4.9

0.4.7 (to benefit from lazy tasks API)

4.8

0.4.5

4.7

0.4.5

4.6

0.4.5

4.5

0.4.5

4.4

0.4.5

4.3

0.4.5

4.2

0.4.4

4.1

0.4.4

Configuration

The plugin makes it easy to integrate into an existing project thanks to a specific configuration. In particular, benchmark source files are expected to be found in the src/jmh directory:

src/jmh
     |- java       : java sources for benchmarks
     |- resources  : resources for benchmarks

The plugin creates a jmh configuration that you should use if your benchmark files depend on a 3rd party library. For example, if you want to use commons-io, you can add the dependency like this:

build.gradle
dependencies {
    jmh 'commons-io:commons-io:2.4'
}

The plugin uses JMH 1.21. You can upgrade the version just by changing the version in the dependencies block:

build.gradle
dependencies {
    jmh 'org.openjdk.jmh:jmh-core:0.9'
    jmh 'org.openjdk.jmh:jmh-generator-annprocess:0.9'
}

Tasks

The project will add several tasks:

  • jmhClasses : compiles raw benchmark code

  • jmhRunBytecodeGenerator : runs bytecode generator over raw benchmark code and generates actual benchmarks

  • jmhCompileGeneratedClasses : compiles generated benchmarks

  • jmhJar : builds the JMH jar containing the JHM runtime and your compiled benchmark classes

  • jmh : executes the benchmarks

The jmh task is the main task and depends on the others so it is in general sufficient to execute this task:

gradle jmh

Configuration options

By default, all benchmarks will be executed, and the results will be generated into $buildDir/reports/jmh. But you can change various options thanks to the jmh configuration block. All configurations variables apart from include are unset, implying that they fall back to the default JMH values:

build.gradle
jmh {
   include = ['some regular expression'] // include pattern (regular expression) for benchmarks to be executed
   exclude = ['some regular expression'] // exclude pattern (regular expression) for benchmarks to be executed
   iterations = 10 // Number of measurement iterations to do.
   benchmarkMode = ['thrpt','ss'] // Benchmark mode. Available modes are: [Throughput/thrpt, AverageTime/avgt, SampleTime/sample, SingleShotTime/ss, All/all]
   batchSize = 1 // Batch size: number of benchmark method calls per operation. (some benchmark modes can ignore this setting)
   fork = 2 // How many times to forks a single benchmark. Use 0 to disable forking altogether
   failOnError = false // Should JMH fail immediately if any benchmark had experienced the unrecoverable error?
   forceGC = false // Should JMH force GC between iterations?
   jvm = 'myjvm' // Custom JVM to use when forking.
   jvmArgs = ['Custom JVM args to use when forking.']
   jvmArgsAppend = ['Custom JVM args to use when forking (append these)']
   jvmArgsPrepend =[ 'Custom JVM args to use when forking (prepend these)']
   humanOutputFile = project.file("${project.buildDir}/reports/jmh/human.txt") // human-readable output file
   resultsFile = project.file("${project.buildDir}/reports/jmh/results.txt") // results file
   operationsPerInvocation = 10 // Operations per invocation.
   benchmarkParameters =  [:] // Benchmark parameters.
   profilers = [] // Use profilers to collect additional data. Supported profilers: [cl, comp, gc, stack, perf, perfnorm, perfasm, xperf, xperfasm, hs_cl, hs_comp, hs_gc, hs_rt, hs_thr]
   timeOnIteration = '1s' // Time to spend at each measurement iteration.
   resultFormat = 'CSV' // Result format type (one of CSV, JSON, NONE, SCSV, TEXT)
   synchronizeIterations = false // Synchronize iterations?
   threads = 4 // Number of worker threads to run with.
   threadGroups = [2,3,4] //Override thread group distribution for asymmetric benchmarks.
   timeout = '1s' // Timeout for benchmark iteration.
   timeUnit = 'ms' // Output time unit. Available time units are: [m, s, ms, us, ns].
   verbosity = 'NORMAL' // Verbosity mode. Available modes are: [SILENT, NORMAL, EXTRA]
   warmup = '1s' // Time to spend at each warmup iteration.
   warmupBatchSize = 10 // Warmup batch size: number of benchmark method calls per operation.
   warmupForks = 0 // How many warmup forks to make for a single benchmark. 0 to disable warmup forks.
   warmupIterations = 1 // Number of warmup iterations to do.
   warmupMode = 'INDI' // Warmup mode for warming up selected benchmarks. Warmup modes are: [INDI, BULK, BULK_INDI].
   warmupBenchmarks = ['.*Warmup'] // Warmup benchmarks to include in the run in addition to already selected. JMH will not measure these benchmarks, but only use them for the warmup.

   zip64 = true // Use ZIP64 format for bigger archives
   jmhVersion = '1.21' // Specifies JMH version
   includeTests = true // Allows to include test sources into generate JMH jar, i.e. use it when benchmarks depend on the test classes.
   duplicateClassesStrategy = 'fail' // Strategy to apply when encountring duplicate classes during creation of the fat jar (i.e. while executing jmhJar task)
}

JMH Options Mapping

The following table describes the mappings between JMH’s command line options and the plugin’s extension properties.

JMH Option Extension Property

-bm <mode>

benchmarkMode

-bs <int>

batchSize

-e <regexp+>

exclude

-f <int>

fork

-foe <bool>

failOnError

-gc <bool>

forceGC

-i <int>

iterations

-jvm <string>

jvm

-jvmArgs <string>

jvmArgs

-jvmArgsAppend <string>

jvmArgsAppend

-jvmArgsPrepend <string>

jvmArgsPrepend

-o <filename>

humanOutputFile

-opi <int>

operationsPerInvocation

-p <param={v,}*>

benchmarkParameters?

-prof <profiler>

profilers

-r <time>

timeOnIteration

-rf <type>

resultFormat

-rff <filename>

resultsFile

-si <bool>

synchronizeIterations

-t <int>

threads

-tg <int+>

threadGroups

-to <time>

timeout

-tu <TU>

timeUnit

-v <mode>

verbosity

-w <time>

warmup

-wbs <int>

warmupBatchSize

-wf <int>

warmupForks

-wi <int>

warmupIterations

-wm <mode>

warmupMode

-wmb <regexp+>

warmupBenchmarks

Dependency on project files

The jmh plugin makes it easy to test existing sources without having to create a separate project for this. This is the reason why you must put your benchmark source files into src/jmh/java instead of src/main/java. This means that by default, the jmh (benchmarks) task depends on your main (production) source set.

It is possible a dependency on the test source set by setting property includeTests to true inside jmh block.

Using JMH Gradle Plugin with Shadow Plugin

Optionally it is possible to use Shadow Plugin to do actual JMH jar creation. The configuration of Shadow Plugin for JMH jar is done via jmhJar block. For example:

build.gradle
jmhJar {
  append('META-INF/spring.handlers')
  append('META-INF/spring.schemas')
  exclude 'LICENSE'
}

Duplicate dependencies and classes

This plugin will merge all dependencies that are defined as part of jmh, runtime and optionally testRuntime configurations into a single set from which fat jar will be created when executing jmhJar task. This is done to ensure that no duplicate dependencies will be added the generated jar.

In addition plugin applies DuplicatesStrategy defined via duplicateClassesStrategy extension property to every class while creating fat jar. By default this property is set to DuplicatesStrategy.FAIL which means that upon detection of duplicate classes the task will fail.

It is possible to change this behavior by configuring duplicateClassesStrategy property via jmh block, e.g.:

build.gradle
jmh {
  duplicateClassesStrategy = 'warn'
}

However if you do encounter problem with defaut value it means that the classpath or sources in your project do contain duplicate classes which means that it is not possible to predict which one will be used when fat jar will generated.

To deal with duplicate files other than classes use Shadow Plugin capabilities, see Using JMH Gradle Plugin with Shadow Plugin.

Known issues

If a benchmark is written in Groovy, you must use the same version of Groovy as the one bundled with Gradle. This is a limitation of the Gradle Worker API that is going to be fixed in the future.

jmh-gradle-plugin's People

Contributors

a10y avatar aalmiray avatar bensteinert avatar big-guy avatar blackdrag avatar chrisbrookes avatar epickrram avatar grv87 avatar gvsmirnov avatar imanushin avatar krezisky avatar melix avatar reinhapa avatar steppinrazor avatar surfing avatar vyazelenko avatar wolfs avatar

Watchers

 avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.