Marijn Kuiper
Runs a 4,000-scenario suite on a build farm and has opinions about shards.
Illustrative user
Example threads. These questions and answers were written by the SDODS team to document real problems and how to fix them. The people named are illustrative, not real users. Questions asked on this site are marked new.
Asked (4)
Example threads. These questions and answers were written by the SDODS team to document real problems and how to fix them. The people named are illustrative, not real users. Questions asked on this site are marked new.
Does --repeat-each 5 with --shard 1/4 repeat inside the shard or across shards?
Setting up a soak job: run the regression five times over, spread across four machines, and fail if anything is flaky. What I cannot work out is whether the repeats are part of…
asked by Marijn KuiperShard 3 finishes four minutes after shard 1 — is there anything smarter than i/n?
We run the regression across eight shards. The wall-clock time of the job is set by the slowest shard, and ours are consistently uneven — shard 3 carries two long visual features…
asked by Marijn Kuipersdods lint: "Scenario must carry exactly one layer tag" on every scenario I have
Ported a feature file over from our old cucumber project and lint rejects all of it. The Gherkin is valid — it parses, the steps exist. What is a "layer tag", and does it belong…
asked by Marijn KuiperWhere is a page object supposed to live? Mine is never picked up
I put my page object in projects/demo-shop/support/pages/LoginPage.ts because that is how our old framework was laid out. It compiles, the decorators are on the methods, and the…
asked by Marijn Kuiper
Answered (26)
Example threads. These questions and answers were written by the SDODS team to document real problems and how to fix them. The people named are illustrative, not real users. Questions asked on this site are marked new.
macOS says "SDODS is damaged and can't be opened" after downloading the app
Downloaded the desktop app, dragged it to Applications, double-clicked, and: Downloaded it twice in case something was truncated. Same message. Is the download actually broken or…
asked by Gio CastellanoEvery PR runs all 190 specs — is there a --since for sdods run?
PR checks take 40 minutes because we run the whole suite on every push, and a typical PR touches one module. I was expecting something like but that is not a flag. There is…
asked by Thom VasseurIs there a --since? Every pull request runs the entire suite
A one-line change to a CSS file runs 900 scenarios. Other tools I have used take a diff and work out which tests could possibly be affected. I have read the flag list twice and I…
asked by Paulo Mendes335 of 390 scenarios failed at -w 4 with the user pool exhausted, and doctor is all green
Full regression at four workers. 55 passed, 335 failed, and nearly all of the failures are this: The passes are all scenarios that need no user, or need @user:admin, which…
asked by Chandra Pillaibrew install asks me to trust the SDODS tap first — is that expected?
Ran the documented command: Homebrew stops and asks me to confirm I trust the tap, and prints a brew trust command to run. I have installed plenty of things with brew and not…
asked by Sergio AlcarazI use no authentication still sends the bearer token — my 401 assertion gets a 200
Deny-direction scenario, as plain as it gets: The endpoint does deny anonymous callers — I checked with curl. Our envs/staging.yaml has: and if I remove that block the scenario…
asked by Gio CastellanoEvery generated API scenario hits the UI origin — ui and api baseUrl came out identical
Fresh project from analyze. The starter @api features all fail with HTML where JSON should be, and the reason is in the env file: Our API is at /api on the same origin. I…
asked by Yusuf Demirsdods analyze invented 18 page routes out of directories that happen to be called pages
We are App Router only. No pages/ directory anywhere near the router. But: comes back with a routes table full of things that do not exist: Eighteen of them. The same paths also…
asked by Sergio AlcarazOur @smoke check takes fourteen minutes and people have stopped waiting for it
120 scenarios tagged @smoke, one browser, fourteen minutes on the pull request. The team merges and then reads the result, which means it is not a gate any more. I do not want to…
asked by Tanvi BhattDo you commit baseline PNGs into the repo, or is this what git-lfs is for?
We are about to turn @visual on for real — currently one scenario, about to be fifteen, on two browsers, and eventually two platforms once CI has its own baselines. Before I…
asked by Fiona McAllisterIs there a dry run? I want to see what -t and -m select before I wait ten minutes
Writing a new pipeline step and I keep guessing at the selection, running it, waiting, and finding out I was wrong. Twice now the wrong thing was "it selected nothing", which is…
asked by Helle BorgWe set --retries 3 and the nightly went green. Why is everyone unhappy?
Genuine question, not a troll. Our nightly had six or seven failures a week, all different, all "it passes when I run it". We set retries to 3 and it has been green for nine…
asked by Ade Oyinlolaauth list says the cached state is fresh but the recording opens logged out
Starting a recording with --user standard drops me on the sign-in page, but the cache insists it has a good state: Eighteen minutes old, and we never touched auth.maxAgeMinutes,…
asked by Renata KohlHow many workers on a two-core build box, and can I stop at the first failure?
Two questions from the same afternoon of tuning. Our agents have two cores. I have seen -w 8 in examples, which on our box makes the UI run slower and adds timeouts. What is the…
asked by Lars VinterGuide says baselines live under __screenshots__/<browser>/ — mine are in demo-shop--ui--chromium/darwin/
Screenshots and visual testing says: What is actually on disk after a run: Two extra segments and neither of them is just the browser. I want to write a cleanup script that…
asked by Gio CastellanoInstaller says "This is Windows" when I run it in Git Bash — what should I use instead?
I copied the macOS/Linux one-liner into Git Bash because that is where I live on this machine. It refuses immediately: I do have WSL. Which one is actually the right answer if I…
asked by Sam OrbisonWarning: "Feature is not under a declared module directory" — but the file runs fine
I put a new file at features/smoke-checks.feature, straight under features/. It runs, it passes, but every lint gives me: It is only a warning so nothing is blocked, but it is…
asked by Gio Castellanosdods record on our Linux build agent: codegen exited with code 1 and produced no spec
We moved recording onto the same Linux build agent that runs the nightly suite so nobody has to install browsers locally. It dies before a browser ever appears. sdods run -p…
asked by Avery HollisThree shards, three HTML reports — how do I get one report out of a matrix build?
The nightly runs on three machines: Every machine mints its own run id, so I get three artefact directories, three HTML reports and three rows in the dashboard for what is one…
asked by Vikram RajagopalCI has been green for a week and I do not think the suite ran at all
Someone renamed a tag and our pipeline step is: Exit code 0, green check, everyone happy. I only looked because a bug walked into production that we have a scenario for.…
asked by Karine MoreauDo -l, -b and -t stack, or does the last one win?
I have been typing this without really knowing what it means: Three filters. Do they narrow each other down, or is one of them authoritative? Specifically: I have a scenario…
asked by Avery HollisComing from Selenium PageFactory — what replaces @FindBy and initElements?
We have a Java suite, maybe 300 tests, built the way everyone built them: a BasePage holding the driver and the waits, @FindBy(xpath = "//div[@class='cart']//button") fields, and…
asked by Dinuka PereraWhere does a run actually write its files? I cannot find the report
First real run finished and the console printed the totals, but I have no idea where anything landed. I was expecting a reports/ folder next to features/. There is a .sdods…
asked by Hsu Wei-linIs there a way to see what the install script will do before it does it?
Our security team will not let anyone pipe a URL into a shell on a company laptop, and honestly I agree with them. I want to install SDODS but I need to be able to show someone…
asked by Dinuka PereraDATASET_NOT_FOUND says my dataset is not declared, but the CSV is right there
Trying to drive the login scenarios from a CSV instead of an Examples table. The file is at projects/demo-shop/data/common/users.csv and previewing it works: Six rows, columns…
asked by Dinuka PereraUnresolved variable ${API_TOKEN} at "env.api.auth.token" — where is it supposed to look?
Added bearer auth to the staging environment file: Then: It dies before a single request goes out: The value is in a file next to the project yaml called env.staging. What am I…
asked by Dinuka Perera