Priya Venkatesh
Test data, user pools and the parts of CI nobody wants to own.
SDODS team (illustrative)
Example threads. These questions and answers were written by the SDODS team to document real problems and how to fix them. The people named are illustrative, not real users. Questions asked on this site are marked new.
Asked (0)
Nothing yet.
Answered (69)
Example threads. These questions and answers were written by the SDODS team to document real problems and how to fix them. The people named are illustrative, not real users. Questions asked on this site are marked new.
apt update: the repository "does not have a Release file"
Added the SDODS apt source and the key, then: The odd part: the files are there. curl https://sdods.com/apt/dists/stable/InRelease returns a signed index perfectly happily. So…
asked by Lars VinterA cleanup DELETE returned 404 and the scenario still passed — is that intended?
We register cleanups the way the guide shows: The records are not being deleted. Looking at the run detail, the cleanup ran and the request 404'd, and the scenario is green. Two…
asked by Zeynep ArslanThe UI says saved and no row is written — how do I assert on the database?
We keep getting the same class of bug: the form submits, the toast says saved, the UI shows the new item from local state, and nothing was persisted. It survives every UI…
asked by Hsu Wei-linSignup verification email — is there any way to assert on an inbox?
Our signup funnel is: register, receive an email, click the link, land verified. I can test the first step and the last step and nothing in between, so the coverage stops exactly…
asked by Zeynep ArslanEvery PR runs all 190 specs — is there a --since for sdods run?
PR checks take 40 minutes because we run the whole suite on every push, and a typical PR touches one module. I was expecting something like but that is not a flag. There is…
asked by Thom VasseurCan I run a suite with the Docker image, or is it only the server?
The image is described as the server and the web UI. Our CI does not need a UI, it needs to run the API layer and fail the build. Is the image the wrong tool for that, or does it…
asked by Bruno TeixeiraMy @env:local scenarios ran against staging — does @env: actually skip anything?
The tags table says @env:<name> "restricts a scenario to one environment", and sdods lint is happy with the tag. But: ran all 41 of our @env:local scenarios against staging. Some…
asked by Lars Vinter--strict on sdods run did not turn warnings into failures like it does on sdods lint
We wanted the nightly to fail on lint warnings, so somebody moved the flag onto the run: It ran, it passed, and the eleven warnings we were trying to catch are still there. On…
asked by Paulo Mendes295 API scenarios fail with QUOTA_EXCEEDED: Exceeded quota for verifying passwords
Ran the API and hybrid layers, single worker, one browser. 1,152 tests. 295 of the failures are this one message: The credentials are fine. If I run any one of those scenarios on…
asked by Ade OyinlolaWhat do the installer exit codes mean? They do not match the CLI ones
I want our provisioning script to react differently depending on why the install failed — retry a download, but fail the build for a bad flag. I found the CLI exit code table and…
asked by Nikhil Sane335 of 390 scenarios failed at -w 4 with the user pool exhausted, and doctor is all green
Full regression at four workers. 55 passed, 335 failed, and nearly all of the failures are this: The passes are all scenarios that need no user, or need @user:admin, which…
asked by Chandra PillaiPool leased an account whose username is the literal ${TEST_MEMBER_EMAIL} and auth rejected it
Local runs only. CI has been green throughout, which is why this took a day. projects/mybotbox/data/staging/users.csv: The failure is not from SDODS. It is from the auth…
asked by Ingrid SolbergWhere do I actually look at expected vs actual after a @visual failure on CI?
The nightly went red on a baseline scenario. I can see the scenario failed and I can read the pixel count in the log, which tells me nothing about whether somebody changed a…
asked by Ingrid SolbergI use no authentication still sends the bearer token — my 401 assertion gets a 200
Deny-direction scenario, as plain as it gets: The endpoint does deny anonymous callers — I checked with curl. Our envs/staging.yaml has: and if I remove that block the scenario…
asked by Gio CastellanoGetting the HTML report out of the Docker image we run the suite in
We run the suite inside ghcr.io/yarlisaisolutions/sdods-server on a self-hosted agent. The run works and the exit code is right, then the container goes away and takes the report…
asked by Ingrid Solberg"@sdods/cli is not available on the npm registry" behind an internal mirror
Our build network can only reach an internal registry mirror. The installer stops before it downloads anything: Exit code 7. The package resolves fine from my laptop, so I assume…
asked by Ify AdeyemiEvery generated API scenario hits the UI origin — ui and api baseUrl came out identical
Fresh project from analyze. The starter @api features all fail with HTML where JSON should be, and the reason is in the env file: Our API is at /api on the same origin. I…
asked by Yusuf DemirOne viewer account, twelve read-only scenarios, and the suite behaves as if -w 1
A partner gives us exactly one viewer account and will not create more — it is their system, their policy, and no amount of asking has moved it. Twelve scenarios carry…
asked by Rasha Halabianalyze stopped at 8000 files and most of them were gitignored worktrees
sdods analyze . on our monorepo finishes but the checklist has scan-truncated on it, and the report is nonsense: no CI provider, two modules, no test-id attribute. We have 48…
asked by Bea LindqvistCatching broken configuration in the pull request instead of ten minutes into the suite
Twice this month a pull request that only touched an environment file went green on review and then took the nightly down. Both times it was something a schema check would have…
asked by Anouk de Vriesauth.strategy "custom" needs an auth export in steps/auth.ts, and I wrote one
Our login is a two-step form — email, continue, then password on the next screen — so the four selectors in auth.form do not cover it. Set the strategy to custom and wrote the…
asked by Devon Marshusers.poolSize is 4 and now a row that exists in the CSV is not in the pool
envs/staging.yaml has: data/staging/users.csv has five rows. Row five is visual_user, role standard. The name is in the file, spelled the same, and the role exists. Raising…
asked by Rasha HalabiRunning the smoke suite on a plane — what do I need so nothing touches the network?
Long flight, and separately we are being asked to prove the suite runs on an air-gapped runner. Same problem twice, so I would like to get it right once. Everything is tagged…
asked by Rasha HalabiHow do I give the team coding agents read-only access to our SDODS server?
We host one SDODS server for the QA group. Everyone wants to point their editor at /mcp so they can ask about runs and coverage without a checkout. What I do not want is…
asked by Rasha HalabiDoes ${VAR} in a CSV dataset resolve from .env.staging, or only from the shell?
Our data/common/users.csv has a password column holding ${TEST_MEMBER_PASSWORD} so the value is not in the repo. It works on my machine, where the variable is exported. On a…
asked by Rasha Halabi
More (44)
- Same locator healed five times and I still have no page-object proposal2 answers✓
- Three projects with nearly identical settings — how much can actually be shared?4 answers✓
- sdods auth capture --index 3 says Role "admin" has 2 user(s); index 3 does not exist2 answers✓
- Our @smoke check takes fourteen minutes and people have stopped waiting for it4 answers✓
- CONFIG_INVALID: Project demo-shop has no data.userPool configured, but data.sources.users exists2 answers✓
- @data:users errors with "is not a declared dataset (logins, posts)" after a rename2 answers
- Can one session give me both the recorded spec and a HAR for offline replay?2 answers✓
- Team-wide setup for keeping credentials out of the repo — what actually works day to day?4 answers
- Cancelled a run with ctrl-c and now every pool account looks leased3 answers✓
- A typo in -t made CI green for three weeks — why does matching nothing exit 0?4 answers✓
- Server ingest takes the ndjson but the run viewer has no screenshots2 answers✓
- Do you commit baseline PNGs into the repo, or is this what git-lfs is for?4 answers✓
- Every checkout scenario spends 40 seconds registering a user through the sign-up form first3 answers✓
- Does @data:orders load the dataset, or do I still need the Given step?1 answer✓
- Dataset table needs env.db in envs/staging.yaml — which database is it asking for?1 answer✓
- Is it safe to leave self-healing enabled in CI, or is that cheating?2 answers✓
- Inherited 900 Cypress specs: what maps to what, and what happens to custom commands?4 answers✓
- One scenario fails about one run in twenty — what do I do with it?4 answers✓
- What is the accepted way to delete the records a scenario created?3 answers✓
- Where exactly does the "# title-format:" comment go? Lint still warns after I added one3 answers✓
- First UI run: five scenarios fail on the sign-in page, but the API layer was green4 answers✓
- Chromium downloaded fine but will not launch on a fresh Ubuntu server4 answers✓
- CI job exits 2 with no test results and an empty report — what is code 2?2 answers✓
- We set --retries 3 and the nightly went green. Why is everyone unhappy?3 answers✓
- Suite went from six minutes to nineteen after we enabled healing3 answers✓
- auth list says the cached state is fresh but the recording opens logged out3 answers✓
- How do people keep real passwords out of a users.csv that has to be committed?2 answers✓
- Factory "user" not found, hint says Known factories: (none), but the file exports it2 answers✓
- How long should we keep .sdods/runs around? It is 40 GB now2 answers✓
- Refusing to install as root — our build agent only has a root account2 answers✓
- All 4 user(s) with role "standard" are leased — passes locally, four failures in CI4 answers✓
- @visual is green locally and reported flaky on CI — attempt 1 fails, attempt 2 passes3 answers✓
- "@user:admin is not a declared role (standard, locked_out)" — where is that list?2 answers✓
- Everyone on the team runs a slightly different command. Can I name one?2 answers✓
- Dataset "users" has 3 row(s); row 5 does not exist (0-based) — is the header counted?2 answers✓
- What should a pull request run, and what should wait for the nightly?2 answers
- USER_POOL_EXHAUSTED: No users with role "admin" in dataset "users" (env staging)2 answers✓
- Three shards, three HTML reports — how do I get one report out of a matrix build?3 answers✓
- Project yaml, env yaml, .env, an exported variable and a flag all set the same thing — which one wins?4 answers✓
- Our CI step passes even when lint fails — checking for exit code 1 was wrong?1 answer✓
- Scenario Outline with 20 Examples rows, or a CSV dataset? What do reviewers expect here3 answers
- No data file for "data/{env}/users.csv" in env "staging" when the common file exists2 answers✓
- Secret literal at "env.vars.password" — I want the password in the env file, why is that refused?2 answers✓
- DATASET_NOT_FOUND says my dataset is not declared, but the CSV is right there2 answers✓