Questions
Anything about writing scenarios, running suites, test data, locators or wiring SDODS into CI. Search it before you ask — most errors have been hit before.
Example threads. These questions and answers were written by the SDODS team to document real problems and how to fix them. The people named are illustrative, not real users. Questions asked on this site are marked new.
Our streaming endpoint returns 200 and then dies mid-stream, and nothing catches it
The API steps assert on a completed response, which is fine until the response is a stream. Ours opens with a 200, sends a few events and then fails halfway —…
Installing onto a machine with no outbound internet at all
Air-gapped test lab. No proxy to configure, no mirror to point at — the machines genuinely have no route out, and files arrive on a reviewed share. What I am…
Linting only the staged feature files in a pre-commit hook — is --json stable enough?
Trying to keep the hook under a second on a repo with about four hundred feature files, so linting the whole project every commit is out. This is what I have:…
Unsupported architecture: i686 — is 32-bit Linux out entirely?
We still have a small fleet of older 32-bit boxes doing one specific job, and I was hoping to run a few API scenarios on them rather than provisioning…
Does colorScheme from the env use: block reach a recording, or only the runner?
envs/staging.yaml has: sdods run -l ui respects all three. A recording started with comes up in light mode for me, while the locale and the timezone are…
har record printed no scrub line and the Cookie header is still sitting in the file
We only have one project registered, so I have got into the habit of leaving -p off and letting it pick. Recorded fine, scenario replays. But there was no…
Locator fragility table is empty in insights but heal report has plenty of rows
gives me eleven rows, two of them with double-digit occurrences. prints suite health and the scenario table, and the locators section is empty. Runs are being…
Audit wants proof the agents cannot write to our repo — what do I actually show them?
Internal audit has heard the words "AI writes our tests" and wants a control, not a paragraph. Saying that agents only write proposals is exactly the kind of…
NO_PUBKEY on apt update even though I installed the SDODS key
Key is installed, source is added, and apt still will not verify the index: I saved the key into /usr/share/keyrings/ and the file is definitely there. What…
Can --explain show the values a layer lost with, not just the winner?
The provenance table gives one row per leaf with the winning layer. Passing --headed shows runtime.headed cli; exporting SDODS_HEADED=1 instead shows…
Six locales times dark and light means twelve env files — is there a per-scenario way?
We ship in six locales and both colour schemes. As far as I can tell the locale and the colour scheme are properties of the environment, so covering the…
apt update: the repository "does not have a Release file"
Added the SDODS apt source and the key, then: The odd part: the files are there. curl https://sdods.com/apt/dists/stable/InRelease returns a signed index…
Does --repeat-each 5 with --shard 1/4 repeat inside the shard or across shards?
Setting up a soak job: run the regression five times over, spread across four machines, and fail if anything is flaky. What I cannot work out is whether the…
Marijn KuiperCan I just install the .deb without adding the apt repository?
One laptop, one install, and I would rather not add a third-party apt source to a machine that our security policy says should have as few as possible. Is the…
How do you test a node-graph editor — blocks, edges and connection handles?
Half our product is a workflow builder: a canvas with about 190 block types, edges between them, and connection handles on each node. None of it is text or…
Nadia Belkacemsdods://screenshot resource returns No such screenshot for a run I can open in the UI
Asking the model to compare the before and after images of a failed step. It resolves the run fine — run_get_scenario comes back with the screenshot URIs —…
Windows protected your PC — how do I get past SmartScreen on the installer?
Downloaded the Windows installer and SmartScreen blocks it with the blue dialog. There is no obvious way to continue — only a Don't run button. Two questions…
A cleanup DELETE returned 404 and the scenario still passed — is that intended?
We register cleanups the way the guide shows: The records are not being deleted. Looking at the run detail, the cleanup ran and the request 404'd, and the…
sdods har list leaves glob and recorded blank even though the sidecar exists
Cosmetic, but it made me think the recording was broken. The sidecar is right there next to the recording and it has the timestamp in it: The replay works.…
One page object for a five-step wizard, or five? The URL never changes
Onboarding wizard, five panels, and the URL stays /onboarding the whole way through — the panel is swapped client-side and there is no route to address for…
Zeynep ArslanRun directory not found after moving the artifacts root, exit 2
We moved the artifacts root off the workspace disk on our agents: The run is fine and the directory is there with everything in it. The step afterwards that…
Can I get LCP out of a run as a recorded number, given budgets are not enforced?
I have read the workshop chapter, so I know @perf does not gate anything today and I am not asking for the gate. What I want is the measurement recorded per…
Can I build the server image myself and push it to our own registry?
Our cluster only pulls from an internal registry, so pulling from a public one is not an option regardless of how convenient it is. Is building the image…
Does -t smoke,sanity mean both tags or either tag?
Reading a colleague's script: No at-signs, a comma, no quotes. It runs more scenarios than -t @smoke alone, so I assume it is an or — but I want to be sure it…
The UI says saved and no row is written — how do I assert on the database?
We keep getting the same class of bug: the form submits, the toast says saved, the UI shows the new item from local state, and nothing was persisted. It…
winget install SDODS.SDODS says no package found — is the ID wrong?
The docs table lists it, so I ran it: winget search sdods finds nothing either. Have I got the identifier wrong, or is my winget source stale?
sdods doctor says "demo-shop/staging vars — 0 var(s) resolved" and passes. Is that right?
Green tick, zero variables. The staging environment definitely has a ${...} in it, so I expected at least one. Is the check finding nothing, or finding…
Signup verification email — is there any way to assert on an inbox?
Our signup funnel is: register, receive an email, click the link, land verified. I can test the first step and the last step and nothing in between, so the…
heal_events, heal_locator_stats and sdods heal report — which do I use for what?
Building a small internal dashboard on top of the MCP server. There appear to be three ways to get at healing data and I cannot tell from the tool…
Scenarios for a feature flag that is on in staging and off in production
Our new checkout is behind a build-time flag. On staging it is on, on the production build it is off, and the flag is baked into the bundle at build time so…
Zeynep Arslan
New questions are reviewed before they appear. Found a bug or want a feature instead? Send feedback.