steinybot / scala-hedgehog Goto Github PK

Hedgehog will eat all your bugs.

Hedgehog is a modern property-based testing system, in the spirit of QuickCheck (and ScalaCheck). Hedgehog uses integrated shrinking, so shrinks obey the invariants of generated values by construction.

• Current Status
• Features
• Getting Started
  • SBT Binary Dependency
  • SBT Source Dependency
  • SBT Testing
  • IntelliJ
• Example
• Guides
  • Tutorial
  • State Tutorial
  • State Tutorial - Vars
  • Migration from ScalaCheck
• Motivation
  • Design Considerations
• Resources
• Alternatives

This project is still in some form of early release. The API may break during this stage until (if?) there is a wider adoption.

Please drop us a line if you start using scala-hedgehog in anger, we'd love to hear from you.

• Integrated shrinking, shrinks obey invariants by construction.
• Abstract state machine testing.
• Range combinators for full control over the scope of generated numbers and collections.
• SBT test runner
• Currently no external dependencies in the core module

In your build.sbt you will unfortunately need to add a custom resolver. Hedgehog is released for every commit and so the "version" will be a git commit hash. You can find the bintray repository here.

val hedgehogVersion = "${COMMIT}"

libraryDependencies ++= Seq(
  "qa.hedgehog" %% "hedgehog-core" % hedgehogVersion,
  "qa.hedgehog" %% "hedgehog-runner" % hedgehogVersion,
  "qa.hedgehog" %% "hedgehog-sbt" % hedgehogVersion
)

resolvers += "bintray-scala-hedgehog" at "https://dl.bintray.com/hedgehogqa/scala-hedgehog"

This project can be added as an SBT subproject.

// This can also be a branch name, like 'master'`, if you want to live on the edge
val hedgehogVersion = "${COMMIT}"
val hedgehogUri = uri("https://github.com/hedgehogqa/scala-hedgehog.git#" + hedgehogVersion)

lazy val root =
  (project in file("."))
    .dependsOn(ProjectRef(hedgehogUri, "core"))
    .dependsOn(ProjectRef(hedgehogUri, "runner"))
    .dependsOn(ProjectRef(hedgehogUri, "sbt-test"))

NOTE: Depending on your scala version(s) SBT might not resolve.

Scala Hedgehog comes with a very primitive runner interface, and supports the SBT testing extension.

testFrameworks += TestFramework("hedgehog.sbt.Framework")

The IntelliJ scala plugin only has hard-coded support for the most popular test frameworks. While Hedgehog is obviously not included in that list, an may never be, by extending the runner Properties tests can be run as an application (as Properties includes a handy main function). NOTE: This requires the test to be an object and not a class.

See the examples module for working versions.

import hedgehog._
import hedgehog.runner._

object PropertyTest extends Properties {

  def tests: List[Test] =
    List(
      property("reverse", testReverse)
    )

  def testReverse: Property =
    for {
      xs <- Gen.alpha.list(Range.linear(0, 100)).forAll
    } yield xs.reverse.reverse ==== xs
}

• Tutorial
• State Tutorial
• State Tutorial - Vars
• Migration from ScalaCheck

The background and motivation for Hedgehog in general is still best described by the original author in this excellent presenation:

• Gens N’ Roses: Appetite for Reduction

A very quick summary is that the original QuickCheck and it's derivatives (like ScalaCheck) separate the generation of data from the shrinking, which results in something that cannot be composed easily. It turns out it's fairly simple to combine them in a single data-type.

If you've used ScalaCheck before, it's exactly the same as writing your normal Gen functions, but now those generated value will shrink without any extra information. Magic!

As a general rule, the current Scala API is intended to be direct port of haskell-hedgehog, much like scalacheck was for QuickCheck. The idea being that people familiar with one of the libraries will be comfortable with the other. It also makes it easier not having to re-invent any wheels (or APIs). There will obviously be exceptions where Scala forces us to make a different trade-off. See haskell-differences for examples and more explanation.

Fortunately there is much in common across property-testing material, and as such the following are still relevant despite whatever language they are presented with.

• Choosing properties for property-based testing (sharpforfunandprofit.com)
• An introduction to property-based testing (sharpforfunandprofit.com)
• Integrated versus Manual Shrinking (well-typed.com)
• Property-Based Testing in a Screencast Editor (wickstrom.tech)

• Much Ado About Testing - Nicolas Rinaudo at ScalaWorld 2019
• Introduction to stateful property based testing - Tomasz Kowal at Lambda Days 2019
• Building on Developers' Intuitions to Create Effective Property-Based Tests - John Hughes at Lambda Days 2019
• Property-Based Testing The Ugly Parts: Case Studies from Komposition - Oskar Wickström at flatMap (Oslo) 2019
• Types vs Tests - Julien Truffaut at Scala eXchange 2018
• Appetite for dysfunction - Andrew McCluskey at Compose Melbourne 2018
• Property-based State Machine Testing - Andrew McCluskey at YOW! Lambda Jam 2018.
• Gens N’ Roses: Appetite for Reduction - Jacob Stanley at YOW! Lambda Jam 2017.
• Find More Bugs with Less Effort - Charles O'Farrell at YOW! Night Singapore 2017.
• Property-based Testing in Practice - Alex Chan at QCon 2017.
• Functions and Determinism in Property-based Testing - Erik Osheim at Philly ETE 2017
• Practical Property-Based Testing - Charles O’Farrell at YOW! Lambda Jam 2015.
• Property-Based Testing for Better Code - Jessica Kerr at Midwest.io 2014.
• I Dream of Gen’ning: ScalaCheck Beyond the Basics - Kelsey Gilmore-Innis at Scala By The Bay 2014.
• Testing the Hard Stuff and Staying Sane - John Hughes on property-based testing using Quviq QuickCheck.
• Testing Stateful Systems with ScalaCheck - Rickard Nilsson at ScalaDays 2014.

• Property-Based Testing in a Screencast Editor by Oskar Wickström (2019)
• Property-Based Testing with PropEr, Erlang, and Elixir by Fred Hebert (2018)
• ScalaCheck: The Definitive Guide by Rickard Nilsson (2014)

In Scala there are other property-testing alternatives:

• https://github.com/rickynils/scalacheck

  The original port of QuickCheck in Scala.

• https://zio.dev/docs/usecases/usecases_testing

  A testing library build on ZIO that includes integrated shrinking.

  • https://github.com/adamgfraser/unveiling-zio-test/blob/master/unveiling-zio-test.pdf
  • https://github.com/zio/zio/tree/master/test/shared/src/main/scala/zio/test

• https://github.com/melrief/sonic

  This is another port of Hedgehog, based on cats and monix.

• https://github.com/scalaprops/scalaprops

  Makes some improvements on the ScalaCheck implementation.

• https://github.com/japgolly/nyaya

  Another fast data generator and property testing library in Scala.