Code Monkey home page Code Monkey logo

java-jwt-benchmark's Introduction

java-jwt-benchmark

Project for benchmarking popular Json Web Token (JWT) frameworks for Java using JMH.

Supported frameworks:

License

Apache 2.0

Obtain

The project is based on Gradle.

Usage

Modify the build version to your current snapshot, then run

./gradlew --stop && ./gradlew clean jmhClasses jmh --refresh-dependencies --info

Note: Requires Java 8 to build, there is some classpath issue with Java 11 which I've yet to understand.

Disable CPU Boost

Temporarily disable boost (untill next reboot) for AMD:

echo "0" | sudo tee /sys/devices/system/cpu/cpufreq/boost

The JMH plugin seems to have trouble refreshing the project, so restart the Gradle deamon before running.

Benchmarks

  • Parse token (if possible)
  • Verify token
  • Verify token, get claim.

General performance observations

There is not much headroom for improving the performance of RSA-based signatures; most (> 90%) of the CPU time goes to signature verification.

This observation is based on two simple 'baseline' RSA-based JWT verifiers included in this project. They perform about 5% faster than the fastest JWT libraries, but then only does signature verification and no JSON parsing / field validation. So it seems likely this is the upper limit.

Latest results

Running for OpenJDK 11 build 11.0.18+10 and OpenJDK 17 build 17.0.6+10, on Fedora Linux 6.1.8-200, AMD Ryzen 5950.

Result visualizations:

Summary: For JWT verification fusionauth-jwt and java-jwt are tied for first place. fusionauth-jwt can parse faster (without verification).

Previous results

Note that the relative number matters, benchmarks are not necessarily run on the same machine, operating system or Java version.

Version 1.4.0:

Running for OpenJDK 11 build 11.0.11.0.9, Fedora Linux 5.13.4-200, AMD Ryzen 7 PRO 3700U laptop.

A visualization_v1.4.0 is available. In short, fusionauth-jwt is the fastest parser, but Java-jwt is very close.

Version 1.3.0:

Running for OpenJDK 1.8.0 build 275-b01, Fedora Linux 5.10.7-200, AMD 5950x.

A visualization_v1.3.0 is available, summery:

Framework Version Verify (op/s) Claim (op/s) Parse (op/s)
java-jwt 3.12.0/0.15.0 30.1k 29.9k 387k
fusionauth-jwt 4.0.1 32.6k 32.4k 1196k
jjwt 0.9.1 31.4k 30.7k
Nimbus-JOSE-JWT 9.4.1 19.7k 18.9k 945k
Okta JWT Verifier for Java 0.5.0 1.3k 1.3k

In short, fusionauth-jwt is the fastest parser. Parsing without validation is now faster than before for java-jwt, otherwise no improvements.

Version 1.2.0:

Running for OpenJDK 1.8.0 build 232, Linux 5.3.7.

A visualization_v1.2.0 is available, summery:

Framework Version Verify (op/s) Claim (op/s) Parse (op/s)
java-jwt 3.8.3/0.9.0 14.6k 14.6k 230.8k
jjwt 0.9.1 14.9k 14.5k
Okta JWT Verifier for Java 0.4.0 0.6k 0.6k
fusionauth-jwt 3.1.6 15.9k 15.7k 581.5k

In short, fusionauth-jwt is the new fastest parser. Parsing without validation is now faster than before for java-jwt, otherwise no improvements.

Version 1.1.0:

Running for OpenJDK 1.8.0 build 191.

A visualization_v1.1.0 is available, summery:

Framework Version Verify (op/s) Claim (op/s)
java-jwt 3.7.0/0.7.0 14.5k 14.6k
jjwt 0.9.1 14.7k 14.9k
Okta JWT Verifier for Java 0.4.0 0.61k 0.61k

In short, java-jwt did catch up with jjwt, they were essentially just as fast. The Okta JWT verifiser regressed into a total disaster.

Only java-jwt seems to parse without validation, at about 52.3k operations per second.

Version 1.0.0:

Running for Oracle JDK 1.8 build unknown-

A visualization_v1.0.0 is available, summery:

Framework Version Verify (op/s) Claim (op/s)
java-jwt 3.3.0/0.4.0 14.2k 13.9k
jjwt 0.9.0 19k 18.3k
Okta JWT Verifier for Java 0.3.0 17.1k 17.1k

Only java-jwt seems to parse without validation, at about 65.2k operations per second.

History

  • 1.x: Added baseline JWT verifiers
  • 1.5.0: Updated dependencies
  • 1.3.0: Updated dependencies, added Nimbus JOSE + JWT
  • 1.2.0: Added FusionAuth
  • 1.1.0: Bumped versions after accepted performance-enhancing PR#255 for java-jwt.
  • 1.0.0: Initial version

java-jwt-benchmark's People

Contributors

dkomanov avatar renovate-bot avatar renovate[bot] avatar skjolber avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

java-jwt-benchmark's Issues

Algorithm comparison

Would it be possible to compare the different algorithms on verification?
I.e. how do the libraries compare on RS256, HS256, ES256 (and EdDSA)?

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Detected dependencies

gradle
gradle.properties
settings.gradle
build.gradle
  • me.champeau.jmh 0.7.2
  • com.github.ben-manes.versions 0.51.0
  • org.owasp.dependencycheck 9.0.10
  • com.github.spotbugs 5.2.1
  • io.morethan.jmhreport 0.9.6
  • org.junit.jupiter:junit-jupiter-api 5.10.0
  • org.junit.jupiter:junit-jupiter-engine 5.10.0
  • org.bouncycastle:bcprov-jdk18on 1.77
  • io.fusionauth:fusionauth-jwt 5.3.2
  • com.auth0:java-jwt 4.4.0
  • com.auth0:jwks-rsa 0.22.1
  • io.jsonwebtoken:jjwt-api 0.12.5
  • io.jsonwebtoken:jjwt-impl 0.12.5
  • io.jsonwebtoken:jjwt-jackson 0.12.5
  • javax.xml.bind:jaxb-api 2.3.1
  • org.bitbucket.b_c:jose4j 0.9.6
  • com.nimbusds:nimbus-jose-jwt 9.37.3
  • com.okta.jwt:okta-jwt-verifier 0.5.8
  • com.okta.jwt:okta-jwt-verifier-impl 0.5.8
  • org.openjdk.jmh:jmh-core 1.37
  • org.openjdk.jmh:jmh-generator-annprocess 1.37
  • org.openjdk.jmh:jmh-generator-bytecode 1.37
frameworks/baseline-bench/build.gradle
  • com.brsanthu:migbase64 2.2
frameworks/fusionauth-jwt-bench/build.gradle
frameworks/java-jwt-bench/build.gradle
frameworks/jjwt-bench/build.gradle
frameworks/jose4j-bench/build.gradle
frameworks/nimbus-bench/build.gradle
frameworks/okta-jwt-verifier-bench/build.gradle
jmh-benchmark/build.gradle
jmh-utils/build.gradle
gradle-wrapper
gradle/wrapper/gradle-wrapper.properties
  • gradle 8.4

  • Check this box to trigger a request for Renovate to run again on this repository

Fix build

For some reason I am only able to compile with Java 8. 11 and 17 give a class not found exception. This does not however block running the JMH benchmark on 11 or 17 .

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.