pointfreeco / swift-snapshot-testing Goto Github PK
View Code? Open in Web Editor NEW📸 Delightful Swift snapshot testing.
Home Page: https://www.pointfree.co/episodes/ep41-a-tour-of-snapshot-testing
License: MIT License
📸 Delightful Swift snapshot testing.
Home Page: https://www.pointfree.co/episodes/ep41-a-tour-of-snapshot-testing
License: MIT License
Can not make SnapshotTesting work.
When I use assertSnapshot(matching: view, as: .image)
I get this message "The file "failed "Test.swift" doesn’t exist"
What I can do ?
Hi, one more issue from me 😇 It looks like WKWebView is never performing image snapshot when there is no content loaded. I have the case where one of my views contains WKWebView that have no content initially. If I've debugged that correctly image snapshot on web view waits until the page finishes loading. But when there is no page loading it never occurs resulting in a timeout.
I am wondering if making it possible to take a snapshot when there is no content in web view is a good solution here.
Workaround for this is to load any file from disk into, even empty one.
This is especially the case with FBSnapshot library which we use. We will like to switch to this one for recursive description text diff but might retain some image comparision. In that case we might need some tolerance for check. Does the library support it at the moment. Sorry, didnt have time to dig into the code or try out yet.
I'm trying to do a simple UI snapshot testing using a ViewController with some UILabel text centered.
In the test file, I instantiate the ViewController and do an assertSnapshot
. First time fails as expected and records the snapshot time in the filesystem as expected.
The second time I run it (it should pass) fails logging a mismatch between the sizes of the images.
Error: failed - Expected image@(375.0, 812.0) to match image@(250.0, 541.333333333333)
I have 2 issues/concerns here:
The current error isn't so helpful.
Would be nice to be able to assert on multiple snapshot strategies at once, as we do with a helper here here:
With it defined, we can also make an extension for some common things.
extension Dictionary where Key == String, Value == Snapshotting<UIViewController, UIImage> {
static let allDevices: Dictionary = [
"iphone-se-portrait": .image(on: .iPhoneSe(.portrait)),
// …
]
}
// …
assertSnapshots(matching: vc, as: .allDevices)
What's the main reason to not include computed vars on snapshots? Is it a technical limitation or it's just a design decision?
Dictionary snapshotting is not deterministic. Let's consider the following snapshot.
let dictionary = ["first": 1, "second": 2, "third": 3]
assertSnapshot(matching: dictionary)
Each time the snapshot is made, the snapshot result is different.
▿ 3 key/value pairs
▿ (2 elements)
- key: "third"
- value: 3
▿ (2 elements)
- key: "first"
- value: 1
▿ (2 elements)
- key: "second"
- value: 2
▿ 3 key/value pairs
▿ (2 elements)
- key: "first"
- value: 1
▿ (2 elements)
- key: "third"
- value: 3
▿ (2 elements)
- key: "second"
- value: 2
Is there any way to handle this scenario correctly? Should we have a strategy for Dictionary<Comparable, T>
so the snapshot is captured by sorting the keys in the first place?
Thanks
I have found crash in one of my apps when running image snapshot of UIViewController. In this specific case ViewController was ment to use as child and its view have translatesAutoresizingMasksIntoConstraints
property set to false. When trying to record snapshot on such ViewController snapshot was not able to produce png while somehow passing size guards (it looks like its size is 0,0).
Here is a stack trace from recording:
thread #1, queue = 'com.apple.main-thread', stop reason = Fatal error: Unexpectedly found nil while unwrapping an Optional value
frame #0: 0x000000010f5d5960 libswiftCore.dylib`_swift_runtime_on_report
frame #1: 0x000000010f627331 libswiftCore.dylib`_swift_stdlib_reportFatalError + 113
frame #2: 0x000000010f32d57a libswiftCore.dylib`function signature specialization <Arg[1] = [Closure Propagated : reabstraction thunk helper from @callee_guaranteed (@unowned Swift.UnsafeBufferPointer<Swift.UInt8>) -> () to @escaping @callee_guaranteed (@unowned Swift.UnsafeBufferPointer<Swift.UInt8>) -> (@out ()), Argument Types : [@callee_guaranteed (@unowned Swift.UnsafeBufferPointer<Swift.UInt8>) -> ()]> of generic specialization <()> of Swift.StaticString.withUTF8Buffer<A>((Swift.UnsafeBufferPointer<Swift.UInt8>) -> A) -> A + 58
frame #3: 0x000000010f5b573e libswiftCore.dylib`partial apply forwarder for closure #2 (Swift.UnsafeBufferPointer<Swift.UInt8>) -> () in Swift._fatalErrorMessage(_: Swift.StaticString, _: Swift.StaticString, file: Swift.StaticString, line: Swift.UInt, flags: Swift.UInt32) -> Swift.Never + 110
frame #4: 0x000000010f32d57a libswiftCore.dylib`function signature specialization <Arg[1] = [Closure Propagated : reabstraction thunk helper from @callee_guaranteed (@unowned Swift.UnsafeBufferPointer<Swift.UInt8>) -> () to @escaping @callee_guaranteed (@unowned Swift.UnsafeBufferPointer<Swift.UInt8>) -> (@out ()), Argument Types : [@callee_guaranteed (@unowned Swift.UnsafeBufferPointer<Swift.UInt8>) -> ()]> of generic specialization <()> of Swift.StaticString.withUTF8Buffer<A>((Swift.UnsafeBufferPointer<Swift.UInt8>) -> A) -> A + 58
frame #5: 0x000000010f4f76e9 libswiftCore.dylib`function signature specialization <Arg[2] = Dead, Arg[3] = Dead> of Swift._fatalErrorMessage(_: Swift.StaticString, _: Swift.StaticString, file: Swift.StaticString, line: Swift.UInt, flags: Swift.UInt32) -> Swift.Never + 105
frame #6: 0x000000010f32d053 libswiftCore.dylib`Swift._fatalErrorMessage(_: Swift.StaticString, _: Swift.StaticString, file: Swift.StaticString, line: Swift.UInt, flags: Swift.UInt32) -> Swift.Never + 19
* frame #7: 0x0000000129a1298a SnapshotTesting`closure #1 in static Snapshotting<>.image($0=0x0000600000fb4070) at UIImage.swift:25
frame #8: 0x0000000129a129cf SnapshotTesting`thunk for @escaping @callee_guaranteed (@guaranteed UIImage) -> (@owned Data) at <compiler-generated>:0
frame #9: 0x0000000129a12a31 SnapshotTesting`partial apply for thunk for @escaping @callee_guaranteed (@guaranteed UIImage) -> (@owned Data) at <compiler-generated>:0
frame #10: 0x00000001299b81cd SnapshotTesting`verifySnapshot<A, B>(value=0x0000000129999230 AppTests`reabstraction thunk helper from @callee_guaranteed () -> (@owned __C.UIViewController, @error @owned Swift.Error) to @escaping @callee_guaranteed () -> (@out __C.UIViewController, @error @owned Swift.Error)partial apply forwarder with unmangled suffix ".21" at <compiler-generated>, snapshotting=SnapshotTesting.Snapshotting<UIKit.UIViewController, UIKit.UIImage> @ 0x00007ffee24a66e8, name=nil, recording=false, timeout=5, file="AppTests/DashboardViewTests.swift", testName="testViewe()", line=30) at AssertSnapshot.swift:122
frame #11: 0x00000001299b4f08 SnapshotTesting`assertSnapshot<A, B>(value=0x0000000129999230 AppTests`reabstraction thunk helper from @callee_guaranteed () -> (@owned __C.UIViewController, @error @owned Swift.Error) to @escaping @callee_guaranteed () -> (@out __C.UIViewController, @error @owned Swift.Error)partial apply forwarder with unmangled suffix ".21" at <compiler-generated>, snapshotting=SnapshotTesting.Snapshotting<UIKit.UIViewController, UIKit.UIImage> @ 0x00007ffee24a66e8, name=nil, recording=false, timeout=5, file="AppTests/DashboardViewTests.swift", testName="testViewe()", line=30) at AssertSnapshot.swift:34
frame #12: 0x0000000129998a9c AppTests`DashboardViewTests.testViewe(self=0x00006000024f5aa0) at DashboardViewTests.swift:30
frame #13: 0x0000000129999304 AppTests`@objc DashboardViewTests.testViewe() at <compiler-generated>:0
frame #14: 0x000000010ee8c03c CoreFoundation`__invoking___ + 140
frame #15: 0x000000010ee894d5 CoreFoundation`-[NSInvocation invoke] + 325
frame #16: 0x0000000127e7b018 XCTest`__24-[XCTestCase invokeTest]_block_invoke.196 + 78
frame #17: 0x0000000127ed2c74 XCTest`-[XCTestCase(Failures) performFailableBlock:testCaseRun:shouldInterruptTest:] + 57
frame #18: 0x0000000127ed2b91 XCTest`-[XCTestCase(Failures) _performTurningExceptionsIntoFailuresInterruptAfterHandling:block:] + 96
frame #19: 0x0000000127e7acd9 XCTest`__24-[XCTestCase invokeTest]_block_invoke + 848
frame #20: 0x0000000127ed8b7e XCTest`-[XCUITestContext performInScope:] + 248
frame #21: 0x0000000127e7a8ce XCTest`-[XCTestCase testContextPerformInScope:] + 98
frame #22: 0x0000000127e7a97c XCTest`-[XCTestCase invokeTest] + 137
frame #23: 0x0000000127e7c4b7 XCTest`__26-[XCTestCase performTest:]_block_invoke_2 + 43
frame #24: 0x0000000127ed2c74 XCTest`-[XCTestCase(Failures) performFailableBlock:testCaseRun:shouldInterruptTest:] + 57
frame #25: 0x0000000127ed2b91 XCTest`-[XCTestCase(Failures) _performTurningExceptionsIntoFailuresInterruptAfterHandling:block:] + 96
frame #26: 0x0000000127e7c3ce XCTest`__26-[XCTestCase performTest:]_block_invoke.330 + 88
frame #27: 0x0000000127ee344b XCTest`+[XCTContext runInContextForTestCase:block:] + 225
frame #28: 0x0000000127e7bafd XCTest`-[XCTestCase performTest:] + 675
frame #29: 0x0000000127ebf1a2 XCTest`-[XCTest runTest] + 57
frame #30: 0x0000000127e76ccb XCTest`__27-[XCTestSuite performTest:]_block_invoke + 365
frame #31: 0x0000000127e764a3 XCTest`-[XCTestSuite _performProtectedSectionForTest:testSection:] + 55
frame #32: 0x0000000127e76766 XCTest`-[XCTestSuite performTest:] + 296
frame #33: 0x0000000127ebf1a2 XCTest`-[XCTest runTest] + 57
frame #34: 0x0000000127e76ccb XCTest`__27-[XCTestSuite performTest:]_block_invoke + 365
frame #35: 0x0000000127e764a3 XCTest`-[XCTestSuite _performProtectedSectionForTest:testSection:] + 55
frame #36: 0x0000000127e76766 XCTest`-[XCTestSuite performTest:] + 296
frame #37: 0x0000000127ebf1a2 XCTest`-[XCTest runTest] + 57
frame #38: 0x0000000127e76ccb XCTest`__27-[XCTestSuite performTest:]_block_invoke + 365
frame #39: 0x0000000127e764a3 XCTest`-[XCTestSuite _performProtectedSectionForTest:testSection:] + 55
frame #40: 0x0000000127e76766 XCTest`-[XCTestSuite performTest:] + 296
frame #41: 0x0000000127ebf1a2 XCTest`-[XCTest runTest] + 57
frame #42: 0x0000000127eeee86 XCTest`__44-[XCTTestRunSession runTestsAndReturnError:]_block_invoke + 171
frame #43: 0x0000000127eeefa7 XCTest`__44-[XCTTestRunSession runTestsAndReturnError:]_block_invoke.80 + 68
frame #44: 0x0000000127e8ebc1 XCTest`-[XCTestObservationCenter _observeTestExecutionForBlock:] + 585
frame #45: 0x0000000127eeebfa XCTest`-[XCTTestRunSession runTestsAndReturnError:] + 623
frame #46: 0x0000000127e5b6b6 XCTest`-[XCTestDriver runTestsAndReturnError:] + 422
frame #47: 0x0000000127edf9cd XCTest`_XCTestMain + 1478
frame #48: 0x000000010d851bb8 libXCTestBundleInject.dylib`__RunTests_block_invoke_2 + 13
frame #49: 0x000000010edea62c CoreFoundation`__CFRUNLOOP_IS_CALLING_OUT_TO_A_BLOCK__ + 12
frame #50: 0x000000010ede9de0 CoreFoundation`__CFRunLoopDoBlocks + 336
frame #51: 0x000000010ede4654 CoreFoundation`__CFRunLoopRun + 1284
frame #52: 0x000000010ede3e11 CoreFoundation`CFRunLoopRunSpecific + 625
frame #53: 0x00000001146a71dd GraphicsServices`GSEventRunModal + 62
frame #54: 0x000000011a3cb81d UIKitCore`UIApplicationMain + 140
frame #55: 0x000000010d757b27 AppApp`main at AppDelegate.swift:14
frame #56: 0x0000000111285575 libdyld.dylib`start + 1
and from comparation with existing snapshot:
thread #1, queue = 'com.apple.main-thread', stop reason = Fatal error: Unexpectedly found nil while unwrapping an Optional value
frame #0: 0x000000010fffd960 libswiftCore.dylib`_swift_runtime_on_report
frame #1: 0x000000011004f331 libswiftCore.dylib`_swift_stdlib_reportFatalError + 113
frame #2: 0x000000010fd5557a libswiftCore.dylib`function signature specialization <Arg[1] = [Closure Propagated : reabstraction thunk helper from @callee_guaranteed (@unowned Swift.UnsafeBufferPointer<Swift.UInt8>) -> () to @escaping @callee_guaranteed (@unowned Swift.UnsafeBufferPointer<Swift.UInt8>) -> (@out ()), Argument Types : [@callee_guaranteed (@unowned Swift.UnsafeBufferPointer<Swift.UInt8>) -> ()]> of generic specialization <()> of Swift.StaticString.withUTF8Buffer<A>((Swift.UnsafeBufferPointer<Swift.UInt8>) -> A) -> A + 58
frame #3: 0x000000010ffdd73e libswiftCore.dylib`partial apply forwarder for closure #2 (Swift.UnsafeBufferPointer<Swift.UInt8>) -> () in Swift._fatalErrorMessage(_: Swift.StaticString, _: Swift.StaticString, file: Swift.StaticString, line: Swift.UInt, flags: Swift.UInt32) -> Swift.Never + 110
frame #4: 0x000000010fd5557a libswiftCore.dylib`function signature specialization <Arg[1] = [Closure Propagated : reabstraction thunk helper from @callee_guaranteed (@unowned Swift.UnsafeBufferPointer<Swift.UInt8>) -> () to @escaping @callee_guaranteed (@unowned Swift.UnsafeBufferPointer<Swift.UInt8>) -> (@out ()), Argument Types : [@callee_guaranteed (@unowned Swift.UnsafeBufferPointer<Swift.UInt8>) -> ()]> of generic specialization <()> of Swift.StaticString.withUTF8Buffer<A>((Swift.UnsafeBufferPointer<Swift.UInt8>) -> A) -> A + 58
frame #5: 0x000000010ff1f6e9 libswiftCore.dylib`function signature specialization <Arg[2] = Dead, Arg[3] = Dead> of Swift._fatalErrorMessage(_: Swift.StaticString, _: Swift.StaticString, file: Swift.StaticString, line: Swift.UInt, flags: Swift.UInt32) -> Swift.Never + 105
frame #6: 0x000000010fd55053 libswiftCore.dylib`Swift._fatalErrorMessage(_: Swift.StaticString, _: Swift.StaticString, file: Swift.StaticString, line: Swift.UInt, flags: Swift.UInt32) -> Swift.Never + 19
* frame #7: 0x000000012a43d77e SnapshotTesting`diff(old=0x0000600003dc4620, new=0x0000600003dc45b0) at UIImage.swift:99
frame #8: 0x000000012a43ac93 SnapshotTesting`closure #3 in static Snapshotting<>.image(old=0x0000600003dc4620, new=0x0000600003dc45b0, precision=1) at UIImage.swift:29
frame #9: 0x000000012a43b15f SnapshotTesting`partial apply for closure #3 in static Snapshotting<>.image(precision:) at <compiler-generated>:0
frame #10: 0x000000012a43ddb2 SnapshotTesting`thunk for @escaping @callee_guaranteed (@guaranteed UIImage, @guaranteed UIImage) -> (@owned (String, [Attachment])?) at <compiler-generated>:0
frame #11: 0x000000012a43de11 SnapshotTesting`partial apply for thunk for @escaping @callee_guaranteed (@guaranteed UIImage, @guaranteed UIImage) -> (@owned (String, [Attachment])?) at <compiler-generated>:0
frame #12: 0x000000012a3de994 SnapshotTesting`verifySnapshot<A, B>(value=0x000000012a3c0d60 AppTests`partial apply forwarder for reabstraction thunk helper from @callee_guaranteed () -> (@owned __C.UIViewController, @error @owned Swift.Error) to @escaping @callee_guaranteed () -> (@out __C.UIViewController, @error @owned Swift.Error) at <compiler-generated>, snapshotting=SnapshotTesting.Snapshotting<UIKit.UIViewController, UIKit.UIImage> @ 0x00007ffee1a7e648, name=nil, recording=false, timeout=5, file="AppTests/DashboardViewTests.swift", testName="testView()", line=25) at AssertSnapshot.swift:129
frame #13: 0x000000012a3dcf08 SnapshotTesting`assertSnapshot<A, B>(value=0x000000012a3c0d60 AppTests`partial apply forwarder for reabstraction thunk helper from @callee_guaranteed () -> (@owned __C.UIViewController, @error @owned Swift.Error) to @escaping @callee_guaranteed () -> (@out __C.UIViewController, @error @owned Swift.Error) at <compiler-generated>, snapshotting=SnapshotTesting.Snapshotting<UIKit.UIViewController, UIKit.UIImage> @ 0x00007ffee1a7e648, name=nil, recording=false, timeout=5, file="AppTests/DashboardViewTests.swift", testName="testView()", line=25) at AssertSnapshot.swift:34
frame #14: 0x000000012a3c0455 AppTests`DashboardViewTests.testView(self=0x0000600001680400) at DashboardViewTests.swift:25
frame #15: 0x000000012a3c1314 AppTests`@objc DashboardViewTests.testView() at <compiler-generated>:0
frame #16: 0x000000010f8b403c CoreFoundation`__invoking___ + 140
frame #17: 0x000000010f8b14d5 CoreFoundation`-[NSInvocation invoke] + 325
frame #18: 0x00000001288a2018 XCTest`__24-[XCTestCase invokeTest]_block_invoke.196 + 78
frame #19: 0x00000001288f9c74 XCTest`-[XCTestCase(Failures) performFailableBlock:testCaseRun:shouldInterruptTest:] + 57
frame #20: 0x00000001288f9b91 XCTest`-[XCTestCase(Failures) _performTurningExceptionsIntoFailuresInterruptAfterHandling:block:] + 96
frame #21: 0x00000001288a1cd9 XCTest`__24-[XCTestCase invokeTest]_block_invoke + 848
frame #22: 0x00000001288ffb7e XCTest`-[XCUITestContext performInScope:] + 248
frame #23: 0x00000001288a18ce XCTest`-[XCTestCase testContextPerformInScope:] + 98
frame #24: 0x00000001288a197c XCTest`-[XCTestCase invokeTest] + 137
frame #25: 0x00000001288a34b7 XCTest`__26-[XCTestCase performTest:]_block_invoke_2 + 43
frame #26: 0x00000001288f9c74 XCTest`-[XCTestCase(Failures) performFailableBlock:testCaseRun:shouldInterruptTest:] + 57
frame #27: 0x00000001288f9b91 XCTest`-[XCTestCase(Failures) _performTurningExceptionsIntoFailuresInterruptAfterHandling:block:] + 96
frame #28: 0x00000001288a33ce XCTest`__26-[XCTestCase performTest:]_block_invoke.330 + 88
frame #29: 0x000000012890a44b XCTest`+[XCTContext runInContextForTestCase:block:] + 225
frame #30: 0x00000001288a2afd XCTest`-[XCTestCase performTest:] + 675
frame #31: 0x00000001288e61a2 XCTest`-[XCTest runTest] + 57
frame #32: 0x000000012889dccb XCTest`__27-[XCTestSuite performTest:]_block_invoke + 365
frame #33: 0x000000012889d4a3 XCTest`-[XCTestSuite _performProtectedSectionForTest:testSection:] + 55
frame #34: 0x000000012889d766 XCTest`-[XCTestSuite performTest:] + 296
frame #35: 0x00000001288e61a2 XCTest`-[XCTest runTest] + 57
frame #36: 0x000000012889dccb XCTest`__27-[XCTestSuite performTest:]_block_invoke + 365
frame #37: 0x000000012889d4a3 XCTest`-[XCTestSuite _performProtectedSectionForTest:testSection:] + 55
frame #38: 0x000000012889d766 XCTest`-[XCTestSuite performTest:] + 296
frame #39: 0x00000001288e61a2 XCTest`-[XCTest runTest] + 57
frame #40: 0x000000012889dccb XCTest`__27-[XCTestSuite performTest:]_block_invoke + 365
frame #41: 0x000000012889d4a3 XCTest`-[XCTestSuite _performProtectedSectionForTest:testSection:] + 55
frame #42: 0x000000012889d766 XCTest`-[XCTestSuite performTest:] + 296
frame #43: 0x00000001288e61a2 XCTest`-[XCTest runTest] + 57
frame #44: 0x0000000128915e86 XCTest`__44-[XCTTestRunSession runTestsAndReturnError:]_block_invoke + 171
frame #45: 0x0000000128915fa7 XCTest`__44-[XCTTestRunSession runTestsAndReturnError:]_block_invoke.80 + 68
frame #46: 0x00000001288b5bc1 XCTest`-[XCTestObservationCenter _observeTestExecutionForBlock:] + 585
frame #47: 0x0000000128915bfa XCTest`-[XCTTestRunSession runTestsAndReturnError:] + 623
frame #48: 0x00000001288826b6 XCTest`-[XCTestDriver runTestsAndReturnError:] + 422
frame #49: 0x00000001289069cd XCTest`_XCTestMain + 1478
frame #50: 0x000000010e279bb8 libXCTestBundleInject.dylib`__RunTests_block_invoke_2 + 13
frame #51: 0x000000010f81262c CoreFoundation`__CFRUNLOOP_IS_CALLING_OUT_TO_A_BLOCK__ + 12
frame #52: 0x000000010f811de0 CoreFoundation`__CFRunLoopDoBlocks + 336
frame #53: 0x000000010f80c654 CoreFoundation`__CFRunLoopRun + 1284
frame #54: 0x000000010f80be11 CoreFoundation`CFRunLoopRunSpecific + 625
frame #55: 0x00000001138221dd GraphicsServices`GSEventRunModal + 62
frame #56: 0x0000000118cba81d UIKitCore`UIApplicationMain + 140
frame #57: 0x000000010e17fb27 AppApp`main at AppDelegate.swift:14
frame #58: 0x0000000111cad575 libdyld.dylib`start + 1
iOSSnapshotTestCase supports UIAppearance and visual effect views using a window and drawViewHierarchyInRect
:
We typically write tests against framework, not app, targets, where I'm not sure this is even possible. We should investigate support, though, maybe at least by introspecting whether or not there's a window we can use.
Classes get de-duped in the output of dump
:
class User {
let id: Int
let name: String
init(id: Int, name: String) {
self.id = id
self.name = name
}
}
let user = User(id: 2, name: "Blobby")
let users = [
user,
user,
user,
]
dump(users)
//▿ 3 elements
// ▿ __lldb_expr_45.User #0
// - id: 2
// - name: "Blobby"
// ▿ __lldb_expr_45.User #0
// ▿ __lldb_expr_45.User #0
would be nice to de-dupe that
As mentioned here. We have been trying to introduce the library to our project but are getting crashes in test targets which have asynchronous tests.
For example:
toEventually()
async testing they crash in Nimble's code to manage the runloopUIApplicationMain
) we get a crash trying which appears to be in a call to objc_msgSend
.I enabled Zombie object diagnostics and in both of the above situations just before the crash I see this in the console:
*** -[CALayer setNeedsLayout]: message sent to deallocated instance 0x600000582da0
I'm sorry I can't provide more useful info right now. I will see if I can create a small project which reproduces the issue.
We should track outdated snapshots and potentially fail if any exist. We can add a quick confirmation helper to prune outdated snapshots.
Hi! 👋
Would you accept a PR to add an .xcodeproj
, making this library Carthage compatible?
We use Quick for testing and are therefore required to subclass QuickSpec
, which precludes us from also subclassing SnapshotTestCase
. Is it possible to architect this as a set of extensions to XCTestCase
so that it can be used alongside other testing libraries that are less flexible?
I would be happy to put some work into this and put up a PR if you think it would be possible/valuable.
Hey there,
First off, this library looks awesome and I really enjoyed reading the article you posted about it.
I was starting to integrate this into a project of mine and was intrigued by applying snapshots to instance of arbitrary types—this seems to be one advantage of this lib over FBSnapshotTestCase. I quickly found out that my types would need to conform to DefaultSnapshottable
in order to assert snapshot matches for them. I'm fine implementing conformance via extensions in my test target, but I've yet to figure out the intent of the associated types and how I might implement conformance.
Let's say I wanted to test NSAttributedString
, since in my case my library is rendering an NSAttributedString with a lot of custom attributes. Any advice on how I would declare conformance to DefaultSnapshottable
for NSAttributedString
? Specifically, not sure what the associatedtype Format
should be
protocols that only have Self
and no associated types can be represented by a simple struct:
struct Semigroup<A> {
let op: (A, A) -> A
}
let additiveInt = Semigroup<Int> { $0 + $1 }
let orBool = Semigroup<Bool> { $0 || $1 }
let andBool = Semigroup<Bool> { $0 && $1 }
we can use this idea for Diffable
and would mean a single type could have multiple diffable representations.
I think then Snapshot
could have an init(diffable: Diffable<A>)
initializer, or maybe even be merged with Diffable
protocol
I was integrating this into a project via Cocoapods and noticed that all the code between the #if Xcode
directives wasn't being compiled. Where is this defined? I'm using Xcode, of course 😄
I worked around this by adding -D Xcode
to OTHER_SWIFT_FLAGS
for the library via a Podfile post_install
hook.
We could explore how performance tests and snapshot tests are alike: performance tests are just snapshot tests of how fast a test should take. In this way, .image(precision: 0.9)
is similar in how it allows for a certain amount of variation before failing.
One idea, for example, is to define a strategy like .viewHierarchyDepth
, which could snap merely an image of how complicated a view's hierarchy is. On repeat runs, if the view hierarchy gets deeper, it'll fail.
Removing them makes sense if you are using #function
, but when supplying a custom testName
I don't think it makes quite as much sense.
Right now we have separate UIView/NSView and WKWebView strategies, but really we should have a single UIView/NSView strategy that traverses the hierarchy and creates snapshots of views that aren't captured with layer.render(in:)
, including:
I just wanted to say thank you for designing this thing. It's fascinating, really.
I'm currently working on a PDF generation library for Linux, which is a perfect use case for SnapshotTesting, and it's amazing how I can compare the generated PDF files and get a nice diff in both string format (which is convenient when I want to be sure that the PDFs have exactly the same internal structure) and image format (to see if a change in code resulted in a visible regression).
Keep up the good work.
▿ httpBody: Optional
▿ some: 175 bytes
- count: 175
▿ pointer: 0x0000604000175610
- pointerValue: 105827995702800
should be actually: - httpBody: "comments=string&phone=string&type=string"
Hi,
I'm trying the following test in my project:
//
// ManualEntriesCustomerFunnel.swift
//
import XCTest
import SnapshotTesting
@testable import pos
class AlertControllerMessages: XCTestCase {
func testAlertController() {
let vc = GeneralAlert.errorWith(body: "Hi there", buttonText: "Cancel", cancelAction: nil)
assertSnapshot(matching: vc, as: .image)
}
}
The compiler its giving me an error Generic parameter 'Format' could not be inferred
for the .image
parameter.
Do I need to implement something else?
Firstly, really enjoying Snapshot tests. Already found a bug in our JSON parsing logic.
We compile our tests on one agent, then run the tests on various other agents in parallel. At compile time, #file
is an absolute path to the file. At run time (on a different agent with a different project path), that path is not valid. Thus, all the tests try re-recording the snapshot and fail.
We’ve used resources for bundling JSON files in our test cases, so that is my preference here.
Either option would be “opt-in”, defaulting to your folder structure based approach.
Sent with GitHawk
How do you deal with size classes on different devices?
We'd like to generate snapshots of views in different size classes.
To do this, we pass a UITraitCollection
with different display scales to the .image
function.
Consider the following test case:
func testPrimaryButtonStates() {
// Given
let frame = CGRect(x: 0, y: 0, width: 200, height: 48)
let button = PrimaryButton(frame: frame)
button.awakeFromNib()
button.setTitle("PrimaryButton", for: .normal)
// When
button.testState = .enabled
// Then
assertSnapshot(matching: button, as: .image(traits: .init(displayScale: 2)), named: "enabled@2x")
assertSnapshot(matching: button, as: .image(traits: .init(displayScale: 3)), named: "enabled@3x")
}
This test passes on Simulators with the same display scale (say iPhone 7 -> iPhone 8), but fails when the display scale changes (iPhone 7 -> iPhone X).
Since we run the tests on our CI on multiple devices, we can't guarantee the display scale to always be the same.
Here is a diff of the screenshots of one of the failing test cases:
I'd be nice to configure the view size and it'd be nice to have some easily configured options, like device sizes and screen resolutions.
Responsive design mode has some nice options:
It'd also be nice to adapt this to have an "below the fold" option, which renders with an adaptive height and configurable fold rendering option.
Known issue: ShingoFukuyama/WKWebViewTips#7
See #130
will help for lots of tests
Patience is more human-readable.
it will control the globals by turning them on/off per test function, but it is optional to use
Not able to install via cocoa pods.
https://cl.ly/a7de76ad96fd
Analyzing dependencies
[!] Unable to find a specification for `SnapshotTesting (~> 1.0)`
Podfile - configuration
https://cl.ly/3a1f6cfe9d21
target "xyz" do
inherit! :search_paths
pod 'SnapshotTesting', '~> 1.0'
end
Thanks
I'm thinking maybe swift-snapshot-testing
(import SnapshotTesting
) makes more sense.
Hey folks, ran out of my timebox for this one, but I'm running on macOS, but using SwiftPM. This means one of the XCTContext.runActivity
is running, but the closed-source XCTest isn't available. This means #if !os(Linux)
isn't going to work when running via swift test
.
Their check probably isn't available for us to use, if you want to test it out:
git clone https://github.com/danger/danger-swift.git
cd danger-swift
git checkout snapshots # or dc41e4e06eb2c994e0ece6fdc355386a212fc0fd
swift build
swift run komondor install
swift package generate-xcodeproj
open Danger.xcodeproj
swift test
helpful for ordering files
I have found that recursive description snapshots produce different outcomes when tests run on different iOS simulators. Is that possible to make them repetitive no matter which device simulator I use?
I didn't find any solution on that yet. Maybe adding some configuration like on the image (i.e. .image(on: .iPhoneX)
) will be useful here as well?
trying including diff output from what was previously record + the full recording.
See #126. It'd be nice to provide some better error messaging for end user debugging here.
I have added the library using Carthage and referenced it in the unit test target. My first test looks like this
func testCreateChatRequestWithJustText() {
let group = GroupEntity(groupId: "g1id", title: "G1")
let normalRequest = ThreadCreateRequest(group: group, title: nil, notificationAlert: "U: text", encryptionSecret: "abcde", messageBody: "text", isAlert: false)
if #available(iOS 11.0, *) {
assertSnapshot(matching: normalRequest, as: .json)
}
}
Running it fails with
failed - Recorded snapshot: …
"/Users/igorkulman/Projects/XXX/iOS/Core/CoreTests/Encodable/__Snapshots__/RequestsEncodingTests/testCreateChatRequestWithJustText.1.json"
If I add record = true
it does not help, I get the same result.
My test run fine locally, but when I try to run it on Travis ci I got failed test with an error like this
Expected snapshot@(320.0, 44.0) to match reference@(213.33333333333334, 29.333333333333332)
I don't know why reference size showing weird resolution like 213.33333
Anyone experienced this before?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.