Docs call the insights CLI a placeholder but it clearly does something

Example threads. These questions and answers were written by the SDODS team to document real problems and how to fix them. The people named are illustrative, not real users. Questions asked on this site are marked new.

The known limitations table says:

sdods insights — the CLI command is a placeholder. Metrics are computed by the database package and shown in the web UI dashboard and GET /api/stats/insights.

It does not behave like a placeholder:

bash

sdods insights compute -p demo-shop --window 20

wrote .sdods/insights/demo-shop.json, printed suite health with a trend, a scenarios-by-flakiness table with p95 durations and quarantine candidates, and a locators-by-fragility table with suggested selectors. insights show --top 5 reads it back.

So which is it? I am about to put this in a nightly job and I would rather not build on something that is going to be replaced next month.

reporting cli

asked by Koen Vermeulen

2 answers

Accepted answer
Correct on both counts: the command is real and the limitations row is out of date. It is one computation either way — the CLI, the web dashboard and GET /api/stats/insights all call into the database package, so the numbers agree by construction rather than by luck.
For a nightly job, compute after the last ingest of the day and pin the window and the thresholds:
bash
```
sdods insights compute -p demo-shop --window 30 --min-runs 10 --flaky-threshold 0.2
```
Defaults are window 30, flaky threshold 0.2, fragility threshold 0.1, min-runs 10. Leaving them implicit is fine right up until somebody changes a default and your quarantine list moves overnight with no commit to blame.
Insights and flaky management has the formulas.
answered by Reeta NandalSDODS team
Build on it. That row is stale. It was written when the computation only lived in the database package and the CLI really was a stub, and the CLI grew real logic without anybody going back to the table.
The errors are the giveaway, and they are generally the fastest way to tell whether a command is real:
text
```
No insights computed yet for demo-shop.
  hint: Run `sdods insights compute -p demo-shop` after ingesting some runs.
```
and, on a machine with no database:
text
```
Insights need the @sdods/db package and a configured database.
```
That second one is a DB_REQUIRED error with a hint telling you to migrate and ingest. Placeholders do not have a code and two failure modes.
answered by Ify Adeyemi

Know the answer?

Wrap commands and output in triple backticks and they will render as a code block.

Checking whether you’re signed in…

Have a different question? Ask it.

Docs call the insights CLI a placeholder but it clearly does something

2 answers

Know the answer?

Related