The nightly went red on a baseline scenario. I can see the scenario failed and I can read the pixel count in the log, which tells me nothing about whether somebody changed a button or a font.
Is there a view that puts the committed baseline and the actual capture next to each other, or am I meant to download the artefacts and open them in an image viewer?
Accepted answer
There is a view. Open the run, go to the Scenarios tab, expand the scenario, and the step that ran the baseline assertion carries a block headed "Visual baseline" with the name you gave it. Inside that block the expected and the actual image are loaded together, and it opens in diff mode by default — there is also a slider, an overlay and plain side-by-side if the diff is too noisy to read.
Locally the same thing without the server:
sdods report --last --openThe slider is the one I actually use. A font change moves every glyph edge slightly and looks like noise in a diff; drag the slider and it is obvious in a second.
Caveat for the CI half of the question: those images only reach the server if the run is ingested on the machine that produced them. The ingest route accepts run.json, NDJSON and runner JSON — it does not accept an artefacts archive yet, so uploading a tarball from a later job gets you the rows and none of the pictures.
Two ways round it:
--ingest on the CI run itself, so it ingests from the runner while the files are still there.sdods report ingest --artifacts-dir .sdods/runs as a later step in the same job.Listed under Known limitations as "Server ingest".
Have a different question? Ask it.