Docs call the insights CLI a placeholder but it clearly does something

Asked 15 Jul 20255,210 views2 answers

20votes

The known limitations table says:

sdods insights — the CLI command is a placeholder. Metrics are computed by the database package and shown in the web UI dashboard and GET /api/stats/insights.

It does not behave like a placeholder:

bash

sdods insights compute -p demo-shop --window 20

wrote .sdods/insights/demo-shop.json, printed suite health with a trend, a scenarios-by-flakiness table with p95 durations and quarantine candidates, and a locators-by-fragility table with suggested selectors. insights show --top 5 reads it back.

So which is it? I am about to put this in a nightly job and I would rather not build on something that is going to be replaced next month.

reporting cli

asked by Koen Vermeulen309 · 15 Jul 2025

2 answers

17votes✓
Accepted answer
Correct on both counts: the command is real and the limitations row is out of date. It is one computation either way — the CLI, the web dashboard and GET /api/stats/insights all call into the database package, so the numbers agree by construction rather than by luck.
For a nightly job, compute after the last ingest of the day and pin the window and the thresholds:
bash
```
sdods insights compute -p demo-shop --window 30 --min-runs 10 --flaky-threshold 0.2
```
Defaults are window 30, flaky threshold 0.2, fragility threshold 0.1, min-runs 10. Leaving them implicit is fine right up until somebody changes a default and your quarantine list moves overnight with no commit to blame.
Insights and flaky management has the formulas.
answered by Reeta Nandalmaintainer41,280 · 16 Jul 2025
22votes
Build on it. That row is stale. It was written when the computation only lived in the database package and the CLI really was a stub, and the CLI grew real logic without anybody going back to the table.
The errors are the giveaway, and they are generally the fastest way to tell whether a command is real:
text
```
No insights computed yet for demo-shop.
  hint: Run `sdods insights compute -p demo-shop` after ingesting some runs.
```
and, on a machine with no database:
text
```
Insights need the @sdods/db package and a configured database.
```
That second one is a DB_REQUIRED error with a hint telling you to migrate and ingest. Placeholders do not have a code and two failure modes.
answered by Ify Adeyemi6,584 · 15 Jul 2025

Have the same question?

Ask it with the command you ran and the output you got, and it will be answered here.

Ask a question

Docs call the insights CLI a placeholder but it clearly does something

2 answers

Related