Three shards, three HTML reports — how do I get one report out of a matrix build?

Asked 19 Aug 202112,980 views3 answers

39votes

The nightly runs on three machines:

bash

sdods run -p demo-shop -e staging -b chromium --shard 1/3
sdods run -p demo-shop -e staging -b chromium --shard 2/3
sdods run -p demo-shop -e staging -b chromium --shard 3/3

Every machine mints its own run id, so I get three artefact directories, three HTML reports and three rows in the dashboard for what is one nightly. Reading the morning report means opening three tabs and adding up.

Is there a supported way to make that one run?

ci reporting cli

asked by Vikram Rajagopal1,240 · 19 Aug 2021

3 answers

45votes✓
Accepted answer
Give every shard the same run id and have them write blob reports, then merge:
bash
```
sdods run -p demo-shop -e staging -b chromium --shard 1/3 --run-id gh-123 --reporter blob
sdods run -p demo-shop -e staging -b chromium --shard 2/3 --run-id gh-123 --reporter blob
sdods run -p demo-shop -e staging -b chromium --shard 3/3 --run-id gh-123 --reporter blob
sdods report merge --run gh-123 .sdods/runs/gh-123/shard-reports
```
The shared run id is doing more work than the report: user-pool leases are keyed off it (shard offset plus worker index) so three machines never lease the same pool user, and ingest merges the NDJSON of every shard into one run instead of three.
The whole shape is written up in Sharding and CI. Download the shard report directories from your artefact store into one place first — report merge takes directories, and it tells you which ones it could not find.
answered by Marijn Kuiper18,840 · 20 Aug 2021
19votes
Adding the half that people forget: getting the results into the database from CI without putting database credentials on the build agents.
bash
```
sdods report ingest --run-id gh-123 --server https://sdods.example.com --token $SDODS_TOKEN .sdods/runs/gh-123/messages*.ndjson
```
The upload goes through the server with a scoped token, so the agent holds nothing you would mind rotating.
answered by Priya Venkateshmaintainer29,604 · 23 Aug 2021
11votes
A year late, one operational note from running this on our farm: upload the shard artefacts unconditionally, not just on success. A failed shard is the one whose report you want, and if the upload step is skipped on failure you have merged two thirds of a story.
answered by Sunita Kale982 · 4 Oct 2022

Have the same question?

Ask it with the command you ran and the output you got, and it will be answered here.

Ask a question

Three shards, three HTML reports — how do I get one report out of a matrix build?

3 answers

Related