Reading summary.json in a shell script — which keys are stable?

Example threads. These questions and answers were written by the SDODS team to document real problems and how to fix them. The people named are illustrative, not real users. Questions asked on this site are marked new.

I want the CI job to post totals to Slack without scraping console output. .sdods/runs/<id>/summary.json looks like exactly what I need:

json

{
  "runId": "0180c7d4-9a11-7f02-b3c8-6d10a5e2f477",
  "status": "failed",
  "totals": {
    "total": 20,
    "passed": 18,
    "failed": 1,
    "skipped": 0,
    "timedOut": 0,
    "flaky": 0,
    "healed": 1,
    "durationMs": 9825,
    "workers": 4
  },
  "byProject": { "demo-shop--ui--chromium": { "total": 20, "passed": 18 } }
}

Before I build on it: is this file part of the contract or an implementation detail that is going to move under me? And what is byProject keyed on — I want to split the message per browser.

reporting ci cli

asked by Gio Castellano

2 answers

Accepted answer
It is the contract. summary.json is a RunSummary: runId, status, totals, byProject, failed, flaky and reportPaths. status is passed, failed or cancelled and is derived from the exit code, so it agrees with what CI saw rather than being computed a second way.
byProject is keyed on the runner project name, which is <project-slug>--<layer>--<browser>:
bash
```
jq -r '.byProject | keys[]' .sdods/runs/<id>/summary.json
```
text
```
rwa-bank--api--chromium
rwa-bank--hybrid--chromium
rwa-bank--ui--firefox
```
For the Slack message you probably want failed and flaky rather than counts: both are arrays of { fingerprint, title, runnerProject }, and failed carries error as well. Three scenario titles read better than "1 failed".
One caveat before you make the job depend on it: summary.json is written at the end from the dashboard reporter's metrics.json. If the process was killed mid-run — an agent timeout, a cancelled build — there is no metrics.json and therefore no summary.json. Handle the file not existing; do not treat it as "0 failures".
answered by Reeta NandalSDODS team
If you would rather not know the path at all:
bash
```
sdods report --last --json | jq '.summary.totals'
```
The same object also carries the manifest, which is where git and command live. Putting the sha and the branch in the Slack message is worth the extra two lines — otherwise the first reply is always "which commit is this".
answered by Lena Hartwig

Know the answer?

Wrap commands and output in triple backticks and they will render as a code block.

Checking whether you’re signed in…

Have a different question? Ask it.

Reading summary.json in a shell script — which keys are stable?

2 answers

Know the answer?

Related