First @visual run fails with "A snapshot doesn't exist" and then passes. Bug?

Example threads. These questions and answers were written by the SDODS team to document real problems and how to fix them. The people named are illustrative, not real users. Questions asked on this site are marked new.

Copied the baseline example out of the guide more or less verbatim:

gherkin

@ui @regression @visual
Scenario: The sign-in page matches its baseline
  Given I am on the sign-in page
  Then the page should match the visual baseline "signin"

bash

sdods run -p rwa-bank -e local -l ui -b chromium -t @visual

It fails:

text

Error: A snapshot doesn't exist at projects/rwa-bank/features/__screenshots__/rwa-bank--ui--chromium/darwin/signin.png, writing actual.

The PNG is sitting there afterwards and the second run is green. So am I supposed to run it twice on purpose, or did I miss a setting somewhere?

visual ui

asked by Elsa Nyman

2 answers

Accepted answer
Neither. That failure is the design, and the message is doing exactly what it says: there was no baseline, so it wrote one and told you it had nothing to compare against.
The alternative is worse. If the first run passed quietly, your very first baseline would be a file nobody ever looked at, promoted to the truth by a green tick. Failing forces a person to open the PNG once.
The workshop chapter spells this out in the same words: Screenshots and visual baselines.
answered by Tomas BrekkeSDODS team
One habit that follows from that: do not let the run which writes the baseline be the run that gates a merge.
Record it deliberately, look at it, commit it:
bash
```
sdods run -p rwa-bank -e local -l ui -b chromium -t @visual --update-snapshots
```
After that, every run in the pipeline only ever compares — it never writes — and a red @visual means something changed rather than "this is the first time".
answered by Annika Solheim

Know the answer?

Wrap commands and output in triple backticks and they will render as a code block.

Checking whether you’re signed in…

Have a different question? Ask it.

First @visual run fails with "A snapshot doesn't exist" and then passes. Bug?

2 answers

Know the answer?

Related