First @visual run fails with "A snapshot doesn't exist" and then passes. Bug?

Asked 8 Nov 202115,920 views2 answers

41votes

Copied the baseline example out of the guide more or less verbatim:

gherkin

@ui @regression @visual
Scenario: The sign-in page matches its baseline
  Given I am on the sign-in page
  Then the page should match the visual baseline "signin"

bash

sdods run -p rwa-bank -e local -l ui -b chromium -t @visual

It fails:

text

Error: A snapshot doesn't exist at projects/rwa-bank/features/__screenshots__/rwa-bank--ui--chromium/darwin/signin.png, writing actual.

The PNG is sitting there afterwards and the second run is green. So am I supposed to run it twice on purpose, or did I miss a setting somewhere?

visual ui

asked by Elsa Nyman333 · 8 Nov 2021

2 answers

52votes✓
Accepted answer
Neither. That failure is the design, and the message is doing exactly what it says: there was no baseline, so it wrote one and told you it had nothing to compare against.
The alternative is worse. If the first run passed quietly, your very first baseline would be a file nobody ever looked at, promoted to the truth by a green tick. Failing forces a person to open the PNG once.
The workshop chapter spells this out in the same words: Screenshots and visual baselines.
answered by Tomas Brekkemaintainer33,915 · 8 Nov 2021
18votes
One habit that follows from that: do not let the run which writes the baseline be the run that gates a merge.
Record it deliberately, look at it, commit it:
bash
```
sdods run -p rwa-bank -e local -l ui -b chromium -t @visual --update-snapshots
```
After that, every run in the pipeline only ever compares — it never writes — and a red @visual means something changed rather than "this is the first time".
answered by Annika Solheim14,002 · 10 Nov 2021

Have the same question?

Ask it with the command you ran and the output you got, and it will be answered here.

Ask a question

First @visual run fails with "A snapshot doesn't exist" and then passes. Bug?

2 answers

Related