Reeta Nandal
Works on the runner and the config layer. Will ask you for the exit code.
SDODS team (illustrative)
Example threads. These questions and answers were written by the SDODS team to document real problems and how to fix them. The people named are illustrative, not real users. Questions asked on this site are marked new.
Asked (0)
Nothing yet.
Answered (124)
Example threads. These questions and answers were written by the SDODS team to document real problems and how to fix them. The people named are illustrative, not real users. Questions asked on this site are marked new.
Audit wants proof the agents cannot write to our repo — what do I actually show them?
Internal audit has heard the words "AI writes our tests" and wants a control, not a paragraph. Saying that agents only write proposals is exactly the kind of claim they will ask…
asked by Zeynep ArslanDoes -t smoke,sanity mean both tags or either tag?
Reading a colleague's script: No at-signs, a comma, no quotes. It runs more scenarios than -t @smoke alone, so I assume it is an or — but I want to be sure it is not something…
asked by Zeynep ArslanThe UI says saved and no row is written — how do I assert on the database?
We keep getting the same class of bug: the form submits, the toast says saved, the UI shows the new item from local state, and nothing was persisted. It survives every UI…
asked by Hsu Wei-linsdods doctor says "demo-shop/staging vars — 0 var(s) resolved" and passes. Is that right?
Green tick, zero variables. The staging environment definitely has a ${...} in it, so I expected at least one. Is the check finding nothing, or finding nothing to complain about?
asked by Zeynep ArslanScenarios for a feature flag that is on in staging and off in production
Our new checkout is behind a build-time flag. On staging it is on, on the production build it is off, and the flag is baked into the bundle at build time so there is nothing to…
asked by Zeynep ArslanAdding Allure without losing the SDODS dashboard and the ndjson
Our QA lead wants Allure because that is what the other teams publish. I am happy to produce it, but the last time I touched a reporter list in another tool I lost everything…
asked by Paulo MendesI quarantined a flaky scenario and it still reds the gate
Followed the flaky triage advice and quarantined the worst offender: It shows as quarantined on the dashboard. It also still fails the nightly, and the gate is still red, which…
asked by Renata KohlMy @env:local scenarios ran against staging — does @env: actually skip anything?
The tags table says @env:<name> "restricts a scenario to one environment", and sdods lint is happy with the tag. But: ran all 41 of our @env:local scenarios against staging. Some…
asked by Lars Vinternet::ERR_ABORTED on an authenticated route is our single biggest source of failures
Top error signature in a clean single-worker run, by a wide margin: 55 of the 192 failures I captured, plus a big pile of click timeouts that I am fairly sure are downstream of…
asked by Elsa NymanNo projects found when running inside a container, works fine on the host
Same repo, mounted into the image at /srv/app. On the host every command is fine. Inside: And naming one explicitly gives a slightly different message: The directory is…
asked by Paulo Mendesbrew install fails with "No matching version found for @sdods/agents ... with a date before"
Tried to install the day a release went out and got this, which I cannot make sense of: The version exists — I can see it on npm and npm i -g @sdods/cli works fine on the same…
asked by Paulo MendesIs there a --since? Every pull request runs the entire suite
A one-line change to a CSS file runs 900 scenarios. Other tools I have used take a diff and work out which tests could possibly be affected. I have read the flag list twice and I…
asked by Paulo Mendesdoctor reports chromium missing but the install said it downloaded it
Install finished with no errors. sdods doctor disagrees about the browser: Everything else in the report is fine. Do I need to reinstall everything, or is there a smaller fix?
asked by Tobi SchraderOAuth popup during a recording: everything I did in the popup is missing from the spec
Recording a login that goes through our identity provider. The provider opens in a popup, you finish there, the popup closes and the app is signed in. The recorded spec has the…
I use no authentication still sends the bearer token — my 401 assertion gets a 200
Deny-direction scenario, as plain as it gets: The endpoint does deny anonymous callers — I checked with curl. Our envs/staging.yaml has: and if I remove that block the scenario…
asked by Gio CastellanoLint passes but the run fails on a missing step — why is undefined-step checking opt-in?
Clean lint, then the run dies immediately on a step nobody wrote. Turning the flag on finds it: If it can find it, why is it not on by default? Feels like the most valuable check…
asked by Ingrid SolbergWhy one fixture set for ui, api and hybrid, and which of the fixtures are worker-scoped?
Reading steps/fixtures.ts in the project I inherited and trying to work out what I am extending. Two things I cannot find a straight answer to. Why is there one test rather than…
asked by Ingrid SolbergShould each tester run sdods mcp locally, or should we host one server for the team?
Eight testers, one repo, one staging environment. Right now everybody has stdio in their editor and their own checkout. It works, but nobody can ask about a run somebody else…
asked by Ingrid SolbergRoute keys like pageid-edit-components-section-palette — do I have to type these?
The generated routes: map is full of keys like and the navigate step takes the key: Nobody on my team is going to type that correctly twice. Can the key be something else?
asked by Anouk de VriesExactly which .env files get read, in which order, and how do I confirm it?
I have inherited a repo with .env, .env.local, .env.staging and .env.staging.local at the root and two more inside the project folder. Nobody remembers why. Before I delete any…
asked by Ingrid SolbergQuarantined scenarios still run in our release gate and still fail it
We quarantine flaky scenarios in the web UI while somebody fixes them. The dashboard shows them as quarantined, so I assumed that was the end of it. The release gate disagrees.…
asked by Ingrid Solberg--pm yarn is rejected — we standardised on yarn everywhere
Every repo here uses yarn, so I passed it through and got: Is that a deliberate omission or just something nobody has got to yet? I would rather not have a second package manager…
asked by Thom VasseurThe generated env yaml points api at our copilot host instead of our API
envs/staging.yaml came out with: That is a different service entirely. The correct value is in the same .env.example file that analyze read: So it is not that it could not find…
asked by Nikhil SaneDoes the installer write anything outside its own directory?
Managed laptop. The question I have to answer in writing is not "is it safe", it is "what does it touch" — specifically whether it edits my shell profile, whether it puts…
asked by Avery HollisIs there a flag to raise the analyze file budget? --max-files is not accepted
The truncation warning tells me the scan hit its limit. Nothing in the analyze reference looks like a budget flag, and guessing does not help: Same for --max-depth. Am I missing…
asked by Koen Vermeulen
More (99)
- proposals accept --branch fails with Could not switch to branch and no reason why2 answers✓
- A renamed tag left our smoke job passing with zero tests for three weeks3 answers✓
- Catching broken configuration in the pull request instead of ten minutes into the suite2 answers✓
- sdods analyze invented 18 page routes out of directories that happen to be called pages2 answers✓
- Every scenario passed and the release gate still failed the job2 answers✓
- How do I give the team coding agents read-only access to our SDODS server?4 answers✓
- Cart got redesigned — re-record the flow from scratch, or patch the old spec?4 answers
- Does ${VAR} in a CSV dataset resolve from .env.staging, or only from the shell?3 answers✓
- Can one scenario belong to two modules? It touches cart and checkout3 answers✓
- Two documentation pages say opposite things about @a11y. Which one should I believe?2 answers✓
- Docs say null clears a key and undefined is skipped — where does that apply, exactly?1 answer✓
- My mcp.servers block is ignored when the agent runs with the openai adapter2 answers
- Docs call the insights CLI a placeholder but it clearly does something2 answers✓
- Why does the HAR strict failure have no code, hint or docs link like every other error?2 answers
- Where do the generated spec files live? Git does not see them and neither do I2 answers
- Two Examples blocks, one @smoke and one @regression — lint says found @smoke @regression2 answers✓
- Three projects with nearly identical settings — how much can actually be shared?4 answers✓
- What should I paste when I report an install problem?2 answers✓
- Local model writes out the tool call as text instead of actually calling it3 answers✓
- Getting run results onto a team dashboard without scraping the CLI3 answers
- sdods run refuses to start with LINT_FAILED — can I run anyway while I fix the tags?3 answers✓
- Project references workspace "payments" which is not declared in sdods.workspace.yaml3 answers✓
- Runs are 900MB of PNGs — can I keep the baseline check and stop capturing every step?2 answers✓
- Several projects found; pass --project <slug> — it used to work without one1 answer✓
- Accepting a proposal I rejected yesterday fails with Proposal ... was rejected2 answers✓
- --project-matrix or three -b flags? What is the difference in practice?3 answers✓
- browsers: [chromium] in envs/local.yaml does nothing and raises no error2 answers✓
- perf.budgets validates and resolves, and a nine-second page still passes. What am I missing?2 answers✓
- --fix-tags added @ui everywhere but left every suite-tag error untouched2 answers✓
- Team-wide setup for keeping credentials out of the repo — what actually works day to day?4 answers
- Does sdods agent generate --dry-run call the model at all?2 answers✓
- Recorded with four workers and half the requests are missing from the file1 answer✓
- A typo in -t made CI green for three weeks — why does matching nothing exit 0?4 answers✓
- Invalid shard "1-3", expected "<current>/<total>" like 1/31 answer✓
- What should I actually look at in a proposal before running sdods proposals accept?3 answers
- Process "pr-chek" is not defined for project demo-shop — where do processes come from?2 answers✓
- Will SDODS download its own Chromium if I already have Playwright installed?2 answers✓
- Does -t "@regression and not @mock" replace the layer filter or add to it?3 answers✓
- Unknown module "api" for project demo-shop — but features/api/ exists2 answers✓
- Pipeline exits 3 and no test ran — the report is empty3 answers✓
- Why is a completely made-up tag only a warning while @jira:demo12 fails the build?4 answers✓
- Trace links in the web UI give a 404 from /trace/ — is it my deployment?2 answers✓
- Can two projects share one page object, or do we copy the file and live with it?2 answers✓
- Why did my API scenario write posts.api.har and not posts.har?2 answers✓
- sdods mcp --http exits immediately with HTTP mode needs a bearer token2 answers✓
- Repo-root .env and a project .env both set the same name — which one is used?1 answer✓
- Lint says the accessibility step is undefined, but the step library reference lists it2 answers✓
- Four environments, same app — how should base URLs be laid out so nobody copies the wrong one?3 answers
- My page object needs a signed-in session before goto() — where does the login belong?2 answers✓
- Is there a dry run? I want to see what -t and -m select before I wait ten minutes2 answers✓
- Can an SDODS agent commit the feature it wrote, or does a person always have to accept?3 answers✓
- Does every scenario log in separately, or is there one place the login state comes from?3 answers✓
- Environment "qa" has no file at projects/demo-shop/envs/qa.yaml2 answers✓
- Green on my machine, red on the runner — it was replaying a HAR the whole time2 answers✓
- Every API scenario fails at once with a connection error — where do I start?3 answers✓
- Which key names does the secret check actually match? tokenUrl is fine but standardPassword is not3 answers✓
- First UI run: five scenarios fail on the sign-in page, but the API layer was green4 answers✓
- Duplicate scenario title error but the two titles are not identical2 answers✓
- Chromium downloaded fine but will not launch on a fresh Ubuntu server4 answers✓
- CI job exits 2 with no test results and an empty report — what is code 2?2 answers✓
- sdods mcp is registered but Claude Code lists no tools from it2 answers✓
- config show --explain says envYaml for a value that came out of .env.staging3 answers
- Should HAR files go into git? Ours are nearly a megabyte each3 answers✓
- @env:qa rejected with "is not in envs.available (local, staging)" — we do have a qa env2 answers✓
- Invalid configuration in envs/staging.yaml with a bullet list — how do I read the paths?3 answers✓
- What does ${DEMO_SHOP_PASSWORD:-secret_sauce} mean, and how does it get past the secret check?1 answer✓
- If I pass --process nightly-regression and -t @smoke, which one wins?1 answer✓
- When should I reach for the mock step instead of @har, or the other way round?3 answers
- How do you actually decide between @smoke, @regression and @sanity?3 answers
- record --user stops with: auth.strategy "custom" needs an auth export2 answers✓
- Cannot parse YAML at envs/staging.yaml — all mapping items must start at the same column2 answers✓
- Guide says baselines live under __screenshots__/<browser>/ — mine are in demo-shop--ui--chromium/darwin/3 answers✓
- Project slug "demo-shop" does not match its directory "demoshop" after a rename2 answers✓
- Added one mask selector in the env file and the other three disappeared3 answers✓
- Reading summary.json in a shell script — which keys are stable?2 answers✓
- Everyone on the team runs a slightly different command. Can I name one?2 answers✓
- Warning: "Feature is not under a declared module directory" — but the file runs fine2 answers✓
- -t @regression,@mock ran more scenarios, not fewer2 answers✓
- Value in .env.staging is ignored, an old exported variable keeps winning2 answers✓
- New folder under projects/ is ignored completely — no error, just not there2 answers✓
- sdods record on our Linux build agent: codegen exited with code 1 and produced no spec3 answers✓
- Scenario tagged @har:products runs green but har/staging is still empty2 answers✓
- Can a scenario be tagged @ui and @api at once? Lint says "found @hybrid @ui @api"2 answers✓
- Project yaml, env yaml, .env, an exported variable and a flag all set the same thing — which one wins?4 answers✓
- sdods report prints (missing) for the html report after a CI run2 answers✓
- Layer "ui" is not enabled for project demo-shop — but the features are right there1 answer✓
- Environment "qa" is not in envs.available [local, staging] — I created envs/qa.yaml already3 answers✓
- Lint says "found @smoke @regression" — why can a scenario not be in both suites?2 answers✓
- lint says Undefined step: Given I am on the checkout page, but the method is right there2 answers✓
- CI has been green for a week and I do not think the suite ran at all3 answers✓
- Unknown project "demo-shp" — how do I list the slugs it actually knows about?2 answers✓
- Do -l, -b and -t stack, or does the last one win?2 answers✓
- Secret literal at "env.vars.password" — I want the password in the env file, why is that refused?2 answers✓
- Where does a run actually write its files? I cannot find the report3 answers✓
- sdods lint: "Scenario must carry exactly one layer tag" on every scenario I have2 answers✓
- Is there a way to see what the install script will do before it does it?2 answers✓
- sdods run -l api finished in under two seconds — did it actually run anything?2 answers✓
- Unresolved variable ${API_TOKEN} at "env.api.auth.token" — where is it supposed to look?2 answers✓
- Where is a page object supposed to live? Mine is never picked up2 answers✓