Retry tests that failed when Chromium Renderer process crashed about cypress-cloud HOT 3 OPEN

Great idea! Spec level retries could be an easy win, and a more granular retries policy that is based on the error message pattern of the filter could be a more advanced feature.

Internal reference: https://www.notion.so/currents/Renderer-crash-retries-fab2e160f94c4c6f9b7d8a2c241ed502?pvs=4

from cypress-cloud.

Great idea on top of a great idea :-)
I mentioned this also here cypress-io/cypress#349 (comment)
But I don't think the issue in Cypress has a high priority.

It would be a good feature, not only for cypress-cloud but also for sorry-[whatever] in general (but then it would be much more difficult to implement because sorry-cypress would have to handle retries on top of Cypress retries).

But if it's done in cypress-cloud it is manageable. I guess the spec result state is stored solely on sorry-cypress instance and the test retries are handled within Cypress itself.

So before uploading the results, cypress-cloud would check the error message and check that it's only ONE attempt without any retries (so we know that cypress-cloud is not re-running a test that already had retries).

Then it would re-run the whole spec once and just use this one result (no matter if it failed again).

The only question remains if you would somehow include the first attempt or just discard it completely. I think discarding is ok, but still it would be nice to know if a certain error still occurs. So the best solution would be to add an "attempt" to the spec results object.

If the error message pattern was stored in sorry-cypress, it could later be also used for this idea sorry-cypress/sorry-cypress#761 :)

from cypress-cloud.

I like the idea - since cypress 1x we are seeing much more of those errors

from cypress-cloud.