pointfreeco / swift-snapshot-testing Goto Github PK

View Code? Open in Web Editor NEW

3.6K 39.0 542.0 30.18 MB

📸 Delightful Swift snapshot testing.

Home Page: https://www.pointfree.co/episodes/ep41-a-tour-of-snapshot-testing

License: MIT License

Swift 94.97% Makefile 0.21% HTML 4.83% Roff 0.01%

swift testing snapshot-testing screenshot-testing

swift-snapshot-testing's Introduction

🎬 www.pointfree.co

This repo contains the full source code for the Point-Free website, a video series exploring advanced programming topics in Swift. The codebase is split into 3 pieces:

PointFree: This is the core application, and is responsible for routing requests, loading data and rendering HTML and CSS.
Styleguide: This library contains functions and data types for creating a consistent style across the entire website.
Server: This is the actual executable server. It uses NIO to handle the low-level server responsibilities, and hands everything else over to the PointFree package.

Getting Started

Interested in a video tour of the code base?

The repo contains an extensive test suite and some playgrounds to explore. To get things running:

Open up a terminal window and grab the code:

git clone https://github.com/pointfreeco/pointfreeco.git
cd pointfreeco

Make sure cmark is installed. You can install it with Homebrew:
```
brew install cmark # or your preferred installation method
```
Make sure Postgres is installed and running. It's our database of choice. You can install it with Homebrew:
```
brew install postgres # or your preferred installation method
brew services start postgresql # or your preferred launch method
make db
```
(If you use Postgres.app, EnterpriseDB, or another installation method, please follow some additional instructions in the CPostgreSQL README.)

With the project open in Xcode, you can:

Run the server locally
- Select the Server target
- Run: Command+R
- Visit http://localhost:8080
Explore our playgrounds
- Select the PointFree target
- Build: Command+B
- Open a playground!

Some fun things to explore

There're a lot of fun things to explore in this repo. For example:

We develop web pages in playgrounds for a continuous feedback loop. This is made possible by the fact that the entire server stack is composed of pure functions with side-effects pushed to the boundaries of the application. It allows us to load up any request in isolation, including POST requests, all without ever worrying about doing a side-effect.
We use snapshot testing to capture full data structures in order to verify their correctness. Not only do we do this in the traditional way of taking screenshots of web pages at various break points (e.g. on iPhone and desktop), but we can also snapshot any entire request-to-response lifecycle (e.g. the POST to a signup page does the correct redirect).

▿ Step
  ResponseEnded

▿ Request
  POST http://localhost:8080/launch-signup

  [email protected]

▿ Response
  Status 302 FOUND
  Location: /?success=true

Xcode Color Theme

Like the color theme we use in our episodes? Run make colortheme to install locally!

Related projects

Point-Free uses a bunch of interesting open-source software:

🗺 swift-html: A Swift DSL for type-safe, extensible, and transformable HTML documents.
🕸 swift-web: A collection of types and functions for dealing with common web server concerns, such as HTML render, CSS preprocessing, middleware and more.
🎶 swift-prelude: Offers a standard library for experimental functional programming in Swift.
🏷 swift-tagged: Helps us create strong contracts with our data boundaries, like JSON from GitHub and Stripe, and our PostgreSQL data.
📸 swift-snapshot-testing: Powers our testing infrastructure by taking snapshots of various data structures to guarantee the correctness of their output. We use this on everything from middleware to ensure requests are correctly transformed into responses, and even entire web pages to make sure the site looks correct at a variety of sizes (e.g. on iPhone and desktop).

Explore more of our open-source on the Point-Free organization.

Learn More

Brandon gave a talk about most of the core ideas that went into this project at Swift Summit 2017.

Find this interesting?

Then check out Point-Free!

License

The content of this project itself is licensed under the CC BY-NC-SA 4.0 license, and the underlying source code used to format and display that content is licensed under the MIT license.

swift-snapshot-testing's People

Contributors

Stargazers

Watchers

Forkers

palleas gsampaio vkt0r mdiep radeknovis creister jakeatoms simonbility finestructure trendingtechnology vaderdan jobandtalent killectro orta alvincrisuy christopherrogers gabrielpeart matrixy akhilamadathil szymon-sf szotyi devhttps unnamedd slashmo jsbean broadwaylamb wezsietato jensk611 aaabbbccc2018 mackoj edge-man f-meloni pwrzesinski tikitu vojtastavik rjchatfield nilstack sherlouk steve-h piotrbernad domaingrouposs crayment monkeyjb budaiev marianameireles adamcumiskey burakglobal mac-gallagher pavel-y-ivanov sger laviallb phamphiphuc carthaged afarnham ericsites wireapp mayoff openbitapp appcues adellibovi natemann duylengoc mr-v suraphanl mmrepo sairamkotha-swiggy industrialbinaries felixscheinost dirtyhenry andraskadar fespinoza dannyhertz ideasoncanvas sebastianosinski lightricks puneetchd elikohen ctreffs ldstreet tomaslinhart zerocoolhub lacyrhoades wtsnz ileitch maxx-cyber vineetshah seek-oss zhaoze1991 ashokvadivelu theleftbit dstranz tbajis sidepelican nicoonswift satotakeshix yuta24 fzehana spekke regexident hassfers

swift-snapshot-testing's Issues

Dealing With Display Scales

Question:

How do you deal with size classes on different devices?

Elaborate explanation:

We'd like to generate snapshots of views in different size classes.
To do this, we pass a UITraitCollection with different display scales to the .image function.

Consider the following test case:

func testPrimaryButtonStates() {
    // Given
    let frame = CGRect(x: 0, y: 0, width: 200, height: 48)
    let button = PrimaryButton(frame: frame)
    button.awakeFromNib()
    button.setTitle("PrimaryButton", for: .normal)

    // When
    button.testState = .enabled

    // Then
    assertSnapshot(matching: button, as: .image(traits: .init(displayScale: 2)), named: "enabled@2x")
    assertSnapshot(matching: button, as: .image(traits: .init(displayScale: 3)), named: "enabled@3x")
}

This test passes on Simulators with the same display scale (say iPhone 7 -> iPhone 8), but fails when the display scale changes (iPhone 7 -> iPhone X).
Since we run the tests on our CI on multiple devices, we can't guarantee the display scale to always be the same.

Here is a diff of the screenshots of one of the failing test cases:

Crash when used in test target with asynchronous tests

As mentioned here. We have been trying to introduce the library to our project but are getting crashes in test targets which have asynchronous tests.

For example:

Using this library in a test target with tests that use Nimble's toEventually() async testing they crash in Nimble's code to manage the runloop
With those tests disabled, but with other async tests, the test suite completes, but when the app is shutting down after testing (in UIApplicationMain) we get a crash trying which appears to be in a call to objc_msgSend.

I enabled Zombie object diagnostics and in both of the above situations just before the crash I see this in the console:

*** -[CALayer setNeedsLayout]: message sent to deallocated instance 0x600000582da0

I'm sorry I can't provide more useful info right now. I will see if I can create a small project which reproduces the issue.

[Question] Rearchitect to not require subclassing SnapshotTestCase?

We use Quick for testing and are therefore required to subclass QuickSpec, which precludes us from also subclassing SnapshotTestCase. Is it possible to architect this as a set of extensions to XCTestCase so that it can be used alongside other testing libraries that are less flexible?

I would be happy to put some work into this and put up a PR if you think it would be possible/valuable.

Weird reference size when running on travis

My test run fine locally, but when I try to run it on Travis ci I got failed test with an error like this

Expected snapshot@(320.0, 44.0) to match reference@(213.33333333333334, 29.333333333333332)

I don't know why reference size showing weird resolution like 213.33333

Anyone experienced this before?

re-dupe the de-dupe of `dump`

Classes get de-duped in the output of dump:

class User {
  let id: Int
  let name: String
  init(id: Int, name: String) {
    self.id = id
    self.name = name
  }
}

let user = User(id: 2, name: "Blobby")

let users = [
  user,
  user,
  user,
]

dump(users)

//▿ 3 elements
//  ▿ __lldb_expr_45.User #0
//    - id: 2
//    - name: "Blobby"
//  ▿ __lldb_expr_45.User #0
//  ▿ __lldb_expr_45.User #0

would be nice to de-dupe that

Add fatal error for views that have zero width or height

The current error isn't so helpful.

Improve view strategy

Right now we have separate UIView/NSView and WKWebView strategies, but really we should have a single UIView/NSView strategy that traverses the hierarchy and creates snapshots of views that aren't captured with layer.render(in:), including:

GLKView
SCNView
SKView
WKWebView

Fill in TODO error messaging for precondition

Activities are not available in SwiftPM projects (even on macOS) when using `swift test`

Hey folks, ran out of my timebox for this one, but I'm running on macOS, but using SwiftPM. This means one of the XCTContext.runActivity is running, but the closed-source XCTest isn't available. This means #if !os(Linux) isn't going to work when running via swift test .

Their check probably isn't available for us to use, if you want to test it out:

git clone https://github.com/danger/danger-swift.git
cd danger-swift
git checkout snapshots # or dc41e4e06eb2c994e0ece6fdc355386a212fc0fd
swift build
swift run komondor install
swift package generate-xcodeproj
open Danger.xcodeproj

swift test

Carthage compatibility / Xcode project

Hi! 👋

Would you accept a PR to add an .xcodeproj, making this library Carthage compatible?

Improve initial recording failure message

See #130

The file "Test.swift" doesn’t exist

Can not make SnapshotTesting work.
When I use assertSnapshot(matching: view, as: .image) I get this message "The file "failed "Test.swift" doesn’t exist"

What I can do ?

Device simulator dependence on recursive description snapshots

I have found that recursive description snapshots produce different outcomes when tests run on different iOS simulators. Is that possible to make them repetitive no matter which device simulator I use?
I didn't find any solution on that yet. Maybe adding some configuration like on the image (i.e. .image(on: .iPhoneX)) will be useful here as well?

Add size configuration option to view snapshots

I'd be nice to configure the view size and it'd be nice to have some easily configured options, like device sizes and screen resolutions.

Responsive design mode has some nice options:

It'd also be nice to adapt this to have an "below the fold" option, which renders with an adaptive height and configurable fold rendering option.

Explore failing based off snapshot state

We could explore how performance tests and snapshot tests are alike: performance tests are just snapshot tests of how fast a test should take. In this way, .image(precision: 0.9) is similar in how it allows for a certain amount of variation before failing.

One idea, for example, is to define a strategy like .viewHierarchyDepth, which could snap merely an image of how complicated a view's hierarchy is. On repeat runs, if the view hierarchy gets deeper, it'll fail.

make subdirectories for each test class

will help for lots of tests

Use a `Set` instead of `[String: ()]`

https://github.com/pointfreeco/swift-snapshot-testing/blob/master/Sources/SnapshotTesting/SnapshotTesting.swift#L6

Add support for running on CI when “build for testing” agent is different to testing agent

Firstly, really enjoying Snapshot tests. Already found a bug in our JSON parsing logic.

Problem

We compile our tests on one agent, then run the tests on various other agents in parallel. At compile time, #file is an absolute path to the file. At run time (on a different agent with a different project path), that path is not valid. Thus, all the tests try re-recording the snapshot and fail.

Suggestions

Allow to look in Bundle resources for the snapshot file
Allow a run-time file search to find the path of the snapshot

We’ve used resources for bundling JSON files in our test cases, so that is my preference here.
Either option would be “opt-in”, defaulting to your folder structure based approach.

_{Sent with GitHawk}

try a type-erased struct instead of Diffable protocol

protocols that only have Self and no associated types can be represented by a simple struct:

struct Semigroup<A> {
  let op: (A, A) -> A
}

let additiveInt = Semigroup<Int> { $0 + $1 }
let orBool = Semigroup<Bool> { $0 || $1 }
let andBool = Semigroup<Bool> { $0 && $1 }

we can use this idea for Diffable and would mean a single type could have multiple diffable representations.

I think then Snapshot could have an init(diffable: Diffable<A>) initializer, or maybe even be merged with Diffable protocol

Thank you for the awesome library

I just wanted to say thank you for designing this thing. It's fascinating, really.

I'm currently working on a PDF generation library for Linux, which is a perfect use case for SnapshotTesting, and it's amazing how I can compare the generated PDF files and get a nice diff in both string format (which is convenient when I want to be sure that the PDFs have exactly the same internal structure) and image format (to see if a change in code resulted in a visible regression).

Keep up the good work.

Support UIAppearance and visual effect views

iOSSnapshotTestCase supports UIAppearance and visual effect views using a window and drawViewHierarchyInRect:

https://github.com/uber/ios-snapshot-test-case/blob/de1c714952ca6b7f1a40670135297fd761ba43cf/FBSnapshotTestCase/Categories/UIImage%2BSnapshot.m#L39-L70

We typically write tests against framework, not app, targets, where I'm not sure this is even possible. We should investigate support, though, maybe at least by introspecting whether or not there's a window we can use.

Truncate long failures and print full thing

Last two characters of custom testName are being removed

Removing them makes sense if you are using #function, but when supplying a custom testName I don't think it makes quite as much sense.

`#if Xcode` compiler directive

I was integrating this into a project via Cocoapods and noticed that all the code between the #if Xcode directives wasn't being compiled. Where is this defined? I'm using Xcode, of course 😄

I worked around this by adding -D Xcode to OTHER_SWIFT_FLAGS for the library via a Podfile post_install hook.

Switch diffing algorithm to use patience

Patience is more human-readable.

put failure/reference/diff in artifacts directory

How to conform to DefaultSnapshottable

Hey there,

First off, this library looks awesome and I really enjoyed reading the article you posted about it.

I was starting to integrate this into a project of mine and was intrigued by applying snapshots to instance of arbitrary types—this seems to be one advantage of this lib over FBSnapshotTestCase. I quickly found out that my types would need to conform to DefaultSnapshottable in order to assert snapshot matches for them. I'm fine implementing conformance via extensions in my test target, but I've yet to figure out the intent of the associated types and how I might implement conformance.

Let's say I wanted to test NSAttributedString, since in my case my library is rendering an NSAttributedString with a lot of custom attributes. Any advice on how I would declare conformance to DefaultSnapshottable for NSAttributedString? Specifically, not sure what the associatedtype Format should be

Track outdated snapshots

We should track outdated snapshots and potentially fail if any exist. We can add a quick confirmation helper to prune outdated snapshots.

play with alternate record test failure message

trying including diff output from what was previously record + the full recording.

Convert Data in snapshot to readable format

▿ httpBody: Optional
▿ some: 175 bytes
- count: 175
▿ pointer: 0x0000604000175610
- pointerValue: 105827995702800

should be actually: - httpBody: "comments=string&phone=string&type=string"

Timeouts on WKWebView image snapshot

Hi, one more issue from me 😇 It looks like WKWebView is never performing image snapshot when there is no content loaded. I have the case where one of my views contains WKWebView that have no content initially. If I've debugged that correctly image snapshot on web view waits until the page finishes loading. But when there is no page loading it never occurs resulting in a timeout.
I am wondering if making it possible to take a snapshot when there is no content in web view is a good solution here.
Workaround for this is to load any file from disk into, even empty one.

macOS WKWebView doesn't render

Known issue: ShingoFukuyama/WKWebViewTips#7

Group helper: assertSnapshots(matching:as:)

Would be nice to be able to assert on multiple snapshot strategies at once, as we do with a helper here here:

https://github.com/pointfreeco/pointfreeco/blob/8feb17812b2f076ab4996450ee85583ec77f6f54/Tests/PointFreeTests/AboutTests.swift#L20-L26

With it defined, we can also make an extension for some common things.

extension Dictionary where Key == String, Value == Snapshotting<UIViewController, UIImage> {
  static let allDevices: Dictionary = [
    "iphone-se-portrait": .image(on: .iPhoneSe(.portrait)),
    // …
  ]
}

// …

assertSnapshots(matching: vc, as: .allDevices)

Add a reference+failed+diff attachment on a failure

automatic number for multiple tests in a single test case

helpful for ordering files

Simple ViewController UI snapshot testing workflow

I'm trying to do a simple UI snapshot testing using a ViewController with some UILabel text centered.

In the test file, I instantiate the ViewController and do an assertSnapshot. First time fails as expected and records the snapshot time in the filesystem as expected.
The second time I run it (it should pass) fails logging a mismatch between the sizes of the images.

Error: failed - Expected image@(375.0, 812.0) to match image@(250.0, 541.333333333333)

I have 2 issues/concerns here:

Images in the report are the same but somehow it compares different sizes so it reports a failing test.
The error log should report what filename is comparing to check manually in the filesystem. In this test, I only have 1 image but it can be trickier to find the proper image if I have several snapshots recorded.

Unable to find a specification for `SnapshotTesting (~> 1.0)`

Not able to install via cocoa pods.
https://cl.ly/a7de76ad96fd

Analyzing dependencies
[!] Unable to find a specification for `SnapshotTesting (~> 1.0)`

Podfile - configuration
https://cl.ly/3a1f6cfe9d21

 target "xyz" do
	  inherit! :search_paths
	  pod 'SnapshotTesting', '~> 1.0'
  end

Thanks

Make an optional base SnapshotTestCase

it will control the globals by turning them on/off per test function, but it is optional to use

Dictionary snapshots

Dictionary snapshotting is not deterministic. Let's consider the following snapshot.

let dictionary = ["first": 1, "second": 2, "third": 3]
assertSnapshot(matching: dictionary)

Each time the snapshot is made, the snapshot result is different.

▿ 3 key/value pairs
  ▿ (2 elements)
    - key: "third"
    - value: 3
  ▿ (2 elements)
    - key: "first"
    - value: 1
  ▿ (2 elements)
    - key: "second"
    - value: 2

▿ 3 key/value pairs
  ▿ (2 elements)
    - key: "first"
    - value: 1
  ▿ (2 elements)
    - key: "third"
    - value: 3
  ▿ (2 elements)
    - key: "second"
    - value: 2

Is there any way to handle this scenario correctly? Should we have a strategy for Dictionary<Comparable, T> so the snapshot is captured by sorting the keys in the first place?

Thanks

New README

failed - Recorded snapshot: …

I have added the library using Carthage and referenced it in the unit test target. My first test looks like this

func testCreateChatRequestWithJustText() {
    let group = GroupEntity(groupId: "g1id", title: "G1")

    let normalRequest = ThreadCreateRequest(group: group, title: nil, notificationAlert: "U: text", encryptionSecret: "abcde", messageBody: "text", isAlert: false)
    if #available(iOS 11.0, *) {
        assertSnapshot(matching: normalRequest, as: .json)
    }
}

Running it fails with

failed - Recorded snapshot: …

"/Users/igorkulman/Projects/XXX/iOS/Core/CoreTests/Encodable/__Snapshots__/RequestsEncodingTests/testCreateChatRequestWithJustText.1.json"

If I add record = true it does not help, I get the same result.

Improve WKWebView timeout error messaging

See #126. It'd be nice to provide some better error messaging for end user debugging here.

Include computed vars on snapshots

What's the main reason to not include computed vars on snapshots? Is it a technical limitation or it's just a design decision?

Add ARTIFACTS env var again

User TMP_DIR by default
print ksdiff

Crash on image snapshot of UIViewController without autoresizing mask constraints

I have found crash in one of my apps when running image snapshot of UIViewController. In this specific case ViewController was ment to use as child and its view have translatesAutoresizingMasksIntoConstraints property set to false. When trying to record snapshot on such ViewController snapshot was not able to produce png while somehow passing size guards (it looks like its size is 0,0).
Here is a stack trace from recording:

thread #1, queue = 'com.apple.main-thread', stop reason = Fatal error: Unexpectedly found nil while unwrapping an Optional value
    frame #0: 0x000000010f5d5960 libswiftCore.dylib`_swift_runtime_on_report
    frame #1: 0x000000010f627331 libswiftCore.dylib`_swift_stdlib_reportFatalError + 113
    frame #2: 0x000000010f32d57a libswiftCore.dylib`function signature specialization <Arg[1] = [Closure Propagated : reabstraction thunk helper from @callee_guaranteed (@unowned Swift.UnsafeBufferPointer<Swift.UInt8>) -> () to @escaping @callee_guaranteed (@unowned Swift.UnsafeBufferPointer<Swift.UInt8>) -> (@out ()), Argument Types : [@callee_guaranteed (@unowned Swift.UnsafeBufferPointer<Swift.UInt8>) -> ()]> of generic specialization <()> of Swift.StaticString.withUTF8Buffer<A>((Swift.UnsafeBufferPointer<Swift.UInt8>) -> A) -> A + 58
    frame #3: 0x000000010f5b573e libswiftCore.dylib`partial apply forwarder for closure #2 (Swift.UnsafeBufferPointer<Swift.UInt8>) -> () in Swift._fatalErrorMessage(_: Swift.StaticString, _: Swift.StaticString, file: Swift.StaticString, line: Swift.UInt, flags: Swift.UInt32) -> Swift.Never + 110
    frame #4: 0x000000010f32d57a libswiftCore.dylib`function signature specialization <Arg[1] = [Closure Propagated : reabstraction thunk helper from @callee_guaranteed (@unowned Swift.UnsafeBufferPointer<Swift.UInt8>) -> () to @escaping @callee_guaranteed (@unowned Swift.UnsafeBufferPointer<Swift.UInt8>) -> (@out ()), Argument Types : [@callee_guaranteed (@unowned Swift.UnsafeBufferPointer<Swift.UInt8>) -> ()]> of generic specialization <()> of Swift.StaticString.withUTF8Buffer<A>((Swift.UnsafeBufferPointer<Swift.UInt8>) -> A) -> A + 58
    frame #5: 0x000000010f4f76e9 libswiftCore.dylib`function signature specialization <Arg[2] = Dead, Arg[3] = Dead> of Swift._fatalErrorMessage(_: Swift.StaticString, _: Swift.StaticString, file: Swift.StaticString, line: Swift.UInt, flags: Swift.UInt32) -> Swift.Never + 105
    frame #6: 0x000000010f32d053 libswiftCore.dylib`Swift._fatalErrorMessage(_: Swift.StaticString, _: Swift.StaticString, file: Swift.StaticString, line: Swift.UInt, flags: Swift.UInt32) -> Swift.Never + 19
  * frame #7: 0x0000000129a1298a SnapshotTesting`closure #1 in static Snapshotting<>.image($0=0x0000600000fb4070) at UIImage.swift:25
    frame #8: 0x0000000129a129cf SnapshotTesting`thunk for @escaping @callee_guaranteed (@guaranteed UIImage) -> (@owned Data) at <compiler-generated>:0
    frame #9: 0x0000000129a12a31 SnapshotTesting`partial apply for thunk for @escaping @callee_guaranteed (@guaranteed UIImage) -> (@owned Data) at <compiler-generated>:0
    frame #10: 0x00000001299b81cd SnapshotTesting`verifySnapshot<A, B>(value=0x0000000129999230 AppTests`reabstraction thunk helper from @callee_guaranteed () -> (@owned __C.UIViewController, @error @owned Swift.Error) to @escaping @callee_guaranteed () -> (@out __C.UIViewController, @error @owned Swift.Error)partial apply forwarder with unmangled suffix ".21" at <compiler-generated>, snapshotting=SnapshotTesting.Snapshotting<UIKit.UIViewController, UIKit.UIImage> @ 0x00007ffee24a66e8, name=nil, recording=false, timeout=5, file="AppTests/DashboardViewTests.swift", testName="testViewe()", line=30) at AssertSnapshot.swift:122
    frame #11: 0x00000001299b4f08 SnapshotTesting`assertSnapshot<A, B>(value=0x0000000129999230 AppTests`reabstraction thunk helper from @callee_guaranteed () -> (@owned __C.UIViewController, @error @owned Swift.Error) to @escaping @callee_guaranteed () -> (@out __C.UIViewController, @error @owned Swift.Error)partial apply forwarder with unmangled suffix ".21" at <compiler-generated>, snapshotting=SnapshotTesting.Snapshotting<UIKit.UIViewController, UIKit.UIImage> @ 0x00007ffee24a66e8, name=nil, recording=false, timeout=5, file="AppTests/DashboardViewTests.swift", testName="testViewe()", line=30) at AssertSnapshot.swift:34
    frame #12: 0x0000000129998a9c AppTests`DashboardViewTests.testViewe(self=0x00006000024f5aa0) at DashboardViewTests.swift:30
    frame #13: 0x0000000129999304 AppTests`@objc DashboardViewTests.testViewe() at <compiler-generated>:0
    frame #14: 0x000000010ee8c03c CoreFoundation`__invoking___ + 140
    frame #15: 0x000000010ee894d5 CoreFoundation`-[NSInvocation invoke] + 325
    frame #16: 0x0000000127e7b018 XCTest`__24-[XCTestCase invokeTest]_block_invoke.196 + 78
    frame #17: 0x0000000127ed2c74 XCTest`-[XCTestCase(Failures) performFailableBlock:testCaseRun:shouldInterruptTest:] + 57
    frame #18: 0x0000000127ed2b91 XCTest`-[XCTestCase(Failures) _performTurningExceptionsIntoFailuresInterruptAfterHandling:block:] + 96
    frame #19: 0x0000000127e7acd9 XCTest`__24-[XCTestCase invokeTest]_block_invoke + 848
    frame #20: 0x0000000127ed8b7e XCTest`-[XCUITestContext performInScope:] + 248
    frame #21: 0x0000000127e7a8ce XCTest`-[XCTestCase testContextPerformInScope:] + 98
    frame #22: 0x0000000127e7a97c XCTest`-[XCTestCase invokeTest] + 137
    frame #23: 0x0000000127e7c4b7 XCTest`__26-[XCTestCase performTest:]_block_invoke_2 + 43
    frame #24: 0x0000000127ed2c74 XCTest`-[XCTestCase(Failures) performFailableBlock:testCaseRun:shouldInterruptTest:] + 57
    frame #25: 0x0000000127ed2b91 XCTest`-[XCTestCase(Failures) _performTurningExceptionsIntoFailuresInterruptAfterHandling:block:] + 96
    frame #26: 0x0000000127e7c3ce XCTest`__26-[XCTestCase performTest:]_block_invoke.330 + 88
    frame #27: 0x0000000127ee344b XCTest`+[XCTContext runInContextForTestCase:block:] + 225
    frame #28: 0x0000000127e7bafd XCTest`-[XCTestCase performTest:] + 675
    frame #29: 0x0000000127ebf1a2 XCTest`-[XCTest runTest] + 57
    frame #30: 0x0000000127e76ccb XCTest`__27-[XCTestSuite performTest:]_block_invoke + 365
    frame #31: 0x0000000127e764a3 XCTest`-[XCTestSuite _performProtectedSectionForTest:testSection:] + 55
    frame #32: 0x0000000127e76766 XCTest`-[XCTestSuite performTest:] + 296
    frame #33: 0x0000000127ebf1a2 XCTest`-[XCTest runTest] + 57
    frame #34: 0x0000000127e76ccb XCTest`__27-[XCTestSuite performTest:]_block_invoke + 365
    frame #35: 0x0000000127e764a3 XCTest`-[XCTestSuite _performProtectedSectionForTest:testSection:] + 55
    frame #36: 0x0000000127e76766 XCTest`-[XCTestSuite performTest:] + 296
    frame #37: 0x0000000127ebf1a2 XCTest`-[XCTest runTest] + 57
    frame #38: 0x0000000127e76ccb XCTest`__27-[XCTestSuite performTest:]_block_invoke + 365
    frame #39: 0x0000000127e764a3 XCTest`-[XCTestSuite _performProtectedSectionForTest:testSection:] + 55
    frame #40: 0x0000000127e76766 XCTest`-[XCTestSuite performTest:] + 296
    frame #41: 0x0000000127ebf1a2 XCTest`-[XCTest runTest] + 57
    frame #42: 0x0000000127eeee86 XCTest`__44-[XCTTestRunSession runTestsAndReturnError:]_block_invoke + 171
    frame #43: 0x0000000127eeefa7 XCTest`__44-[XCTTestRunSession runTestsAndReturnError:]_block_invoke.80 + 68
    frame #44: 0x0000000127e8ebc1 XCTest`-[XCTestObservationCenter _observeTestExecutionForBlock:] + 585
    frame #45: 0x0000000127eeebfa XCTest`-[XCTTestRunSession runTestsAndReturnError:] + 623
    frame #46: 0x0000000127e5b6b6 XCTest`-[XCTestDriver runTestsAndReturnError:] + 422
    frame #47: 0x0000000127edf9cd XCTest`_XCTestMain + 1478
    frame #48: 0x000000010d851bb8 libXCTestBundleInject.dylib`__RunTests_block_invoke_2 + 13
    frame #49: 0x000000010edea62c CoreFoundation`__CFRUNLOOP_IS_CALLING_OUT_TO_A_BLOCK__ + 12
    frame #50: 0x000000010ede9de0 CoreFoundation`__CFRunLoopDoBlocks + 336
    frame #51: 0x000000010ede4654 CoreFoundation`__CFRunLoopRun + 1284
    frame #52: 0x000000010ede3e11 CoreFoundation`CFRunLoopRunSpecific + 625
    frame #53: 0x00000001146a71dd GraphicsServices`GSEventRunModal + 62
    frame #54: 0x000000011a3cb81d UIKitCore`UIApplicationMain + 140
    frame #55: 0x000000010d757b27 AppApp`main at AppDelegate.swift:14
    frame #56: 0x0000000111285575 libdyld.dylib`start + 1

and from comparation with existing snapshot:

thread #1, queue = 'com.apple.main-thread', stop reason = Fatal error: Unexpectedly found nil while unwrapping an Optional value
    frame #0: 0x000000010fffd960 libswiftCore.dylib`_swift_runtime_on_report
    frame #1: 0x000000011004f331 libswiftCore.dylib`_swift_stdlib_reportFatalError + 113
    frame #2: 0x000000010fd5557a libswiftCore.dylib`function signature specialization <Arg[1] = [Closure Propagated : reabstraction thunk helper from @callee_guaranteed (@unowned Swift.UnsafeBufferPointer<Swift.UInt8>) -> () to @escaping @callee_guaranteed (@unowned Swift.UnsafeBufferPointer<Swift.UInt8>) -> (@out ()), Argument Types : [@callee_guaranteed (@unowned Swift.UnsafeBufferPointer<Swift.UInt8>) -> ()]> of generic specialization <()> of Swift.StaticString.withUTF8Buffer<A>((Swift.UnsafeBufferPointer<Swift.UInt8>) -> A) -> A + 58
    frame #3: 0x000000010ffdd73e libswiftCore.dylib`partial apply forwarder for closure #2 (Swift.UnsafeBufferPointer<Swift.UInt8>) -> () in Swift._fatalErrorMessage(_: Swift.StaticString, _: Swift.StaticString, file: Swift.StaticString, line: Swift.UInt, flags: Swift.UInt32) -> Swift.Never + 110
    frame #4: 0x000000010fd5557a libswiftCore.dylib`function signature specialization <Arg[1] = [Closure Propagated : reabstraction thunk helper from @callee_guaranteed (@unowned Swift.UnsafeBufferPointer<Swift.UInt8>) -> () to @escaping @callee_guaranteed (@unowned Swift.UnsafeBufferPointer<Swift.UInt8>) -> (@out ()), Argument Types : [@callee_guaranteed (@unowned Swift.UnsafeBufferPointer<Swift.UInt8>) -> ()]> of generic specialization <()> of Swift.StaticString.withUTF8Buffer<A>((Swift.UnsafeBufferPointer<Swift.UInt8>) -> A) -> A + 58
    frame #5: 0x000000010ff1f6e9 libswiftCore.dylib`function signature specialization <Arg[2] = Dead, Arg[3] = Dead> of Swift._fatalErrorMessage(_: Swift.StaticString, _: Swift.StaticString, file: Swift.StaticString, line: Swift.UInt, flags: Swift.UInt32) -> Swift.Never + 105
    frame #6: 0x000000010fd55053 libswiftCore.dylib`Swift._fatalErrorMessage(_: Swift.StaticString, _: Swift.StaticString, file: Swift.StaticString, line: Swift.UInt, flags: Swift.UInt32) -> Swift.Never + 19
  * frame #7: 0x000000012a43d77e SnapshotTesting`diff(old=0x0000600003dc4620, new=0x0000600003dc45b0) at UIImage.swift:99
    frame #8: 0x000000012a43ac93 SnapshotTesting`closure #3 in static Snapshotting<>.image(old=0x0000600003dc4620, new=0x0000600003dc45b0, precision=1) at UIImage.swift:29
    frame #9: 0x000000012a43b15f SnapshotTesting`partial apply for closure #3 in static Snapshotting<>.image(precision:) at <compiler-generated>:0
    frame #10: 0x000000012a43ddb2 SnapshotTesting`thunk for @escaping @callee_guaranteed (@guaranteed UIImage, @guaranteed UIImage) -> (@owned (String, [Attachment])?) at <compiler-generated>:0
    frame #11: 0x000000012a43de11 SnapshotTesting`partial apply for thunk for @escaping @callee_guaranteed (@guaranteed UIImage, @guaranteed UIImage) -> (@owned (String, [Attachment])?) at <compiler-generated>:0
    frame #12: 0x000000012a3de994 SnapshotTesting`verifySnapshot<A, B>(value=0x000000012a3c0d60 AppTests`partial apply forwarder for reabstraction thunk helper from @callee_guaranteed () -> (@owned __C.UIViewController, @error @owned Swift.Error) to @escaping @callee_guaranteed () -> (@out __C.UIViewController, @error @owned Swift.Error) at <compiler-generated>, snapshotting=SnapshotTesting.Snapshotting<UIKit.UIViewController, UIKit.UIImage> @ 0x00007ffee1a7e648, name=nil, recording=false, timeout=5, file="AppTests/DashboardViewTests.swift", testName="testView()", line=25) at AssertSnapshot.swift:129
    frame #13: 0x000000012a3dcf08 SnapshotTesting`assertSnapshot<A, B>(value=0x000000012a3c0d60 AppTests`partial apply forwarder for reabstraction thunk helper from @callee_guaranteed () -> (@owned __C.UIViewController, @error @owned Swift.Error) to @escaping @callee_guaranteed () -> (@out __C.UIViewController, @error @owned Swift.Error) at <compiler-generated>, snapshotting=SnapshotTesting.Snapshotting<UIKit.UIViewController, UIKit.UIImage> @ 0x00007ffee1a7e648, name=nil, recording=false, timeout=5, file="AppTests/DashboardViewTests.swift", testName="testView()", line=25) at AssertSnapshot.swift:34
    frame #14: 0x000000012a3c0455 AppTests`DashboardViewTests.testView(self=0x0000600001680400) at DashboardViewTests.swift:25
    frame #15: 0x000000012a3c1314 AppTests`@objc DashboardViewTests.testView() at <compiler-generated>:0
    frame #16: 0x000000010f8b403c CoreFoundation`__invoking___ + 140
    frame #17: 0x000000010f8b14d5 CoreFoundation`-[NSInvocation invoke] + 325
    frame #18: 0x00000001288a2018 XCTest`__24-[XCTestCase invokeTest]_block_invoke.196 + 78
    frame #19: 0x00000001288f9c74 XCTest`-[XCTestCase(Failures) performFailableBlock:testCaseRun:shouldInterruptTest:] + 57
    frame #20: 0x00000001288f9b91 XCTest`-[XCTestCase(Failures) _performTurningExceptionsIntoFailuresInterruptAfterHandling:block:] + 96
    frame #21: 0x00000001288a1cd9 XCTest`__24-[XCTestCase invokeTest]_block_invoke + 848
    frame #22: 0x00000001288ffb7e XCTest`-[XCUITestContext performInScope:] + 248
    frame #23: 0x00000001288a18ce XCTest`-[XCTestCase testContextPerformInScope:] + 98
    frame #24: 0x00000001288a197c XCTest`-[XCTestCase invokeTest] + 137
    frame #25: 0x00000001288a34b7 XCTest`__26-[XCTestCase performTest:]_block_invoke_2 + 43
    frame #26: 0x00000001288f9c74 XCTest`-[XCTestCase(Failures) performFailableBlock:testCaseRun:shouldInterruptTest:] + 57
    frame #27: 0x00000001288f9b91 XCTest`-[XCTestCase(Failures) _performTurningExceptionsIntoFailuresInterruptAfterHandling:block:] + 96
    frame #28: 0x00000001288a33ce XCTest`__26-[XCTestCase performTest:]_block_invoke.330 + 88
    frame #29: 0x000000012890a44b XCTest`+[XCTContext runInContextForTestCase:block:] + 225
    frame #30: 0x00000001288a2afd XCTest`-[XCTestCase performTest:] + 675
    frame #31: 0x00000001288e61a2 XCTest`-[XCTest runTest] + 57
    frame #32: 0x000000012889dccb XCTest`__27-[XCTestSuite performTest:]_block_invoke + 365
    frame #33: 0x000000012889d4a3 XCTest`-[XCTestSuite _performProtectedSectionForTest:testSection:] + 55
    frame #34: 0x000000012889d766 XCTest`-[XCTestSuite performTest:] + 296
    frame #35: 0x00000001288e61a2 XCTest`-[XCTest runTest] + 57
    frame #36: 0x000000012889dccb XCTest`__27-[XCTestSuite performTest:]_block_invoke + 365
    frame #37: 0x000000012889d4a3 XCTest`-[XCTestSuite _performProtectedSectionForTest:testSection:] + 55
    frame #38: 0x000000012889d766 XCTest`-[XCTestSuite performTest:] + 296
    frame #39: 0x00000001288e61a2 XCTest`-[XCTest runTest] + 57
    frame #40: 0x000000012889dccb XCTest`__27-[XCTestSuite performTest:]_block_invoke + 365
    frame #41: 0x000000012889d4a3 XCTest`-[XCTestSuite _performProtectedSectionForTest:testSection:] + 55
    frame #42: 0x000000012889d766 XCTest`-[XCTestSuite performTest:] + 296
    frame #43: 0x00000001288e61a2 XCTest`-[XCTest runTest] + 57
    frame #44: 0x0000000128915e86 XCTest`__44-[XCTTestRunSession runTestsAndReturnError:]_block_invoke + 171
    frame #45: 0x0000000128915fa7 XCTest`__44-[XCTTestRunSession runTestsAndReturnError:]_block_invoke.80 + 68
    frame #46: 0x00000001288b5bc1 XCTest`-[XCTestObservationCenter _observeTestExecutionForBlock:] + 585
    frame #47: 0x0000000128915bfa XCTest`-[XCTTestRunSession runTestsAndReturnError:] + 623
    frame #48: 0x00000001288826b6 XCTest`-[XCTestDriver runTestsAndReturnError:] + 422
    frame #49: 0x00000001289069cd XCTest`_XCTestMain + 1478
    frame #50: 0x000000010e279bb8 libXCTestBundleInject.dylib`__RunTests_block_invoke_2 + 13
    frame #51: 0x000000010f81262c CoreFoundation`__CFRUNLOOP_IS_CALLING_OUT_TO_A_BLOCK__ + 12
    frame #52: 0x000000010f811de0 CoreFoundation`__CFRunLoopDoBlocks + 336
    frame #53: 0x000000010f80c654 CoreFoundation`__CFRunLoopRun + 1284
    frame #54: 0x000000010f80be11 CoreFoundation`CFRunLoopRunSpecific + 625
    frame #55: 0x00000001138221dd GraphicsServices`GSEventRunModal + 62
    frame #56: 0x0000000118cba81d UIKitCore`UIApplicationMain + 140
    frame #57: 0x000000010e17fb27 AppApp`main at AppDelegate.swift:14
    frame #58: 0x0000000111cad575 libdyld.dylib`start + 1

Things to snapshot

Question: Can the image comparision take into consideration some tolerance?

This is especially the case with FBSnapshot library which we use. We will like to switch to this one for recursive description text diff but might retain some image comparision. In that case we might need some tolerance for check. Does the library support it at the moment. Sorry, didnt have time to dig into the code or try out yet.

Rename again?

I'm thinking maybe swift-snapshot-testing (import SnapshotTesting) makes more sense.

Generic parameter 'Format' could not be inferred

Hi,
I'm trying the following test in my project:

//
//  ManualEntriesCustomerFunnel.swift
//

import XCTest
import SnapshotTesting
@testable import pos

class AlertControllerMessages: XCTestCase {

    func testAlertController() {
        let vc = GeneralAlert.errorWith(body: "Hi there", buttonText: "Cancel", cancelAction: nil)
        assertSnapshot(matching: vc, as: .image)
    }

}

The compiler its giving me an error Generic parameter 'Format' could not be inferred for the .image parameter.
Do I need to implement something else?