Example threads. These questions and answers were written by the SDODS team to document real problems and how to fix them. The people named are illustrative, not real users. Questions asked on this site are marked new.
A one-line change to a CSS file runs 900 scenarios. Other tools I have used take a diff and work out which tests could possibly be affected.
I have read the flag list twice and I cannot find anything diff-aware — the selection flags are all tags, layers, browsers, modules, feature and scenario. Is there something I am missing, or is this genuinely not a thing?
Accepted answer
Genuinely not a thing. There is no --since, no changed-files input, and nothing in the run pipeline looks at git beyond recording the sha in run.json. Selection is exactly the list you found.
What we do instead, in order of how much I would trust it:
@smoke small and honest, and let the pull request run only that. This is the workaround that requires no cleverness and does not go wrong.sdods run -p demo-shop -m cart -m checkout -t @smoke. Cheap to write, and modules already carry an ownership boundary, so the mapping is usually obvious.If you go with 2, still run everything before merging to main. A one-line CSS change to a shared component genuinely can break nine modules, and a path-based mapping will not know.
We do 2 and 3 together on a 4,000-scenario suite: module-scoped smoke on the pull request, full matrix nightly, and a full regression on the merge queue before anything lands on main.
The honest accounting is that module scoping has let two regressions through in three years, both from shared components, and both were caught by the merge-queue run within twenty minutes. That was an acceptable trade for us. It would not be for everybody.
Half a tool exists, for completeness: analyze_change_impact over MCP takes a project and a diff range and tells you which scenarios the change touches. It is genuinely useful when an agent or a person is deciding what to look at.
Nothing wires its output into sdods run. You would be turning a list of scenarios into --feature arguments yourself, in a script, and owning that script. I would not build a merge gate on it today.
Wrap commands and output in triple backticks and they will render as a code block.
Checking whether you’re signed in…
Have a different question? Ask it.