Example threads. These questions and answers were written by the SDODS team to document real problems and how to fix them. The people named are illustrative, not real users. Questions asked on this site are marked new.
About one run in four:
Error: Screenshot comparison failed:
18420 pixels (ratio 0.02 of all image pixels) are different.The rest of the time it is clean. The only thing on that page that moves is a card that fades in when the summary request comes back.
My understanding was that captures already run with animations disabled, so what is left to be non-deterministic?
Animations are disabled on every capture, yes — finite CSS animations are fast-forwarded to their end state and infinite ones are frozen. What that cannot do is wait for the data behind the fade.
Your card is not mid-animation, it is mid-request. Sometimes the summary has landed when the comparison happens and sometimes it has not, and disabling animations makes the two outcomes crisper rather than closer.
Assert the settled state before you compare, and let the assertion be the wait:
@ui @regression @visual
Scenario: The account summary matches its baseline
Given I am on the account page
Then the element with test id "summary-total" should be visible
And the page should match the visual baseline "account-summary"No sleep, no polling loop — the visibility assertion already retries until the timeout, so by the time the baseline step runs the page is in one state rather than two.
Worth adding what your escape hatches are not.
The comparison runs with a maximum different-pixel ratio of 0.01, and that number is not exposed in the project yaml today. So a page that genuinely moves 2% will keep failing and there is no knob to turn.
I am not in a hurry to add one either. A tolerance loose enough to hide a fading card is loose enough to hide a column that dropped below the fold. Stabilise the region or mask it; those are the two answers that stay true when someone else inherits the suite.
Wrap commands and output in triple backticks and they will render as a code block.
Checking whether you’re signed in…
Have a different question? Ask it.