Golden screenshots created on Retina (MacOS) differ in size from those on Linux in CI/CD (for example Ubuntu Docker on CircleCI).

Sometimes it's hard to understand why an image doesn't match only based on the original & diff image (e.g. in my case it was due to the difference of color profiles). Having the current screenshot saved in the output folder alongside the diff image would allow to more easily debug un-matching images.

Hello,
First of all, the helpers are really great.
But, do you plan to write a documentation on how to use these helpers in end-to-end tests?

The looks-same options sometimes need to be tweaked, e.g. to bump the tolerance of the antialiasingTolerance levels, is order for a specific test of test suite to pass (e.g. difference of font rendering between OSs can sometimes be tricky)

Having compareScreenshot take an optional LooksSameOptions 4th param would allow more flexibility, without having to fork the lib.

I've noticed that blue-harvest doesn't always save the diff images even though it outputs a message saying they were, looking into it I noticed that looks-same does some funky stuff with setImmediate which explains the behaviour I was seeing, e.g. nothing saved if the e2e run finishes quickly, or only the first image being saved.

How do You plan to support multi language pages?

There are some missing feature and/or bug with the addMask function:

• if the element we want to mask has a z-index greater than 0, the mask is not visible since it's under the element we want to hide
• if the element is "dynamically" added, it seems that the element is not found and the mask is not added
• to remove the mask, one needs to execute the following script manually document.body.removeChild(document.querySelector('#BP_ELEMENT_HIGHLIGHT__')). Maybe it would be a good idea to add a removeMask function
• for the moment, if you add multiple mask, all the mask elements will have the same id which is not great and if you want to remove one of the mask it is not possible. Maybe the mask should not have an id but a class and the addMask function should allow one to specify a suffix for the class name (same for the removeMask)

I'm trying to implement blue-harvest screen comparison in a E2E test setup, running in Azure (windows machines).

When using the screen comparison on localhost, the diff.png is written within the root of the angular project that I run e2e test on, and I can copy the diff image to our test reports.. But when the test is run on the azure build agents, then the diff.png is nowhere to be found.

The question is, is there a defined place where the png could be written to?

Problem

When attempting to use the find(...) (or see(...)) action helper, I'm unable to select a specific element when multiple are found on the page. Instead, I would be presented with the following error:

> see('Aug 26, 2019');

> "Looking for Aug 26, 2019 failed. More than one element is displayed, number of displayed elements: 8
Elements:
<td _ngcontent-wup-c43="" class="mat-cell cdk-column-created mat-column-created ng-star-inserted" mat-cell="" role="gridcell">Aug 26, 2019</td>
<td _ngcontent-wup-c43="" class="mat-cell cdk-column-created mat-column-created ng-star-inserted" mat-cell="" role="gridcell">Aug 26, 2019</td>
<td _ngcontent-wup-c43="" class="mat-cell cdk-column-created mat-column-created ng-star-inserted" mat-cell="" role="gridcell">Aug 26, 2019</td>
...
"

Solution

1. Update the findGlobal(locator) function to look for a new property on the locator, e.g. locator.index and/or locator.wantMultiple
2. Update the opt_options of browserSideFind(locators, opt_options) to include the new property
3. Update retrying_find to return the new possible type of WebElement[]
4. Update BrowserSideOptions with the new property (e.g. locator.wantMultiple)
5. Update getElement(locator: FlexibleLocator, description: string): Promise<WebElement> to also accept and use options: BrowserSideOptions and return Promise<WebElement|WebElement[]>

Ultimately, this would result in the end user having the following experience:

find('Aug 26, 2019', {wantMultiple: true})[2].click();
// or
const dates = below('Created').find('Aug 26, 2019', {wantMultiple: true});
expect(dates.length).toBe(8);

1. diff.png is saved in the root folder by default, which may be inconvenient for some users.
2. The promise returned by compareScreenshot() rejects when there is an error in looksSame.createDiff() instead of rejecting in the case of !equal.
3. compareScreenshot() returns either a boolean or a promise of a string (if rejected) or a boolean (if resolved), which works but can be confusing.

I used click('buttonName') in one of my e2e tests, and it works fine on other browsers but not for the internet explorer. The error message is Failed: Failed to find click("Open Warning"): Error executing JavaScript. I have identified that it is thrown at find.js in retryingFind(), where browserSideScripts.browserSideFind is executed. I tried to debug more but couldn't identify the bug further in browserSideScripts.js.

Currently all the golden images are always updated when running UPDATE_GOLDENS=1, independently of if they have changed or not. This make code reviewing for visual changes hard when the code base contains hundreds of golden screenshots.

Instead, only changed images should be updated when running UPDATE_GOLDENS=1, which would allow to highlight the specific changes a code change is causing.

Seems that Dropdowns/Selects are unsupported.