11 years of lessons, 5 years of ladder

A suite that only ran on one machine

Selenium WebDriver and a Jenkins box were the whole stack. Scenarios were pinned to CSS selectors, run on a grid somebody maintained by hand, and a failure arrived as a red line and a stack trace.

SDODS carries. One project per application, with its own environments and its own config — because a suite that borrows another team’s settings is a suite that breaks on Mondays.

• Selenium WebDriver
• Cucumber-JVM
• Jenkins
• Page objects

The browser gets a standard, the report does not

Selenium 3.0 shipped in October and pushed everyone towards the W3C WebDriver protocol. Browsers finally agreed on how to be driven. Nothing agreed on how to explain a failure.

SDODS carries. A layered step library: one phrasing style across UI, API and hybrid flows, so a change of driver never rewrites the scenarios on top of it.

• Selenium 3.0
• W3C WebDriver draft
• Hosted browser grids
• Gherkin

The browser stops needing a screen

Headless Chrome landed in June, Puppeteer in August, and Cypress arrived arguing the runner belongs inside the browser. PhantomJS was retired by its own maintainer.

SDODS carries. Screenshots as a first-class artefact rather than a debugging afterthought: if nobody is watching the browser, the run has to keep the pictures.

• Headless Chrome 59
• Puppeteer
• Cypress
• Containers in CI

The cheapest question stops being asked through a browser

W3C WebDriver became a Recommendation in June. Contract testing and API-first suites grew, because driving a browser to check a total was the slowest possible way to ask a cheap question.

SDODS carries. API seeding and hybrid scenarios — arrange the state over HTTP, assert it in the UI, and never click through a five-step wizard to build a fixture.

• W3C WebDriver Rec
• Contract testing
• OpenAPI 3
• API-first suites

CI becomes the only machine that counts

GitHub Actions went generally available in November and the pipeline moved into the repository. From here on “works on my laptop” stopped being a defence and started being the bug report.

SDODS carries. Six-layer configuration precedence that can name the exact file every value came from — the answer to staging and CI quietly disagreeing.

• GitHub Actions
• Pipelines as code
• Ephemeral runners
• Matrix builds

Three browsers, one API, no sleeps

Playwright’s first release in January brought auto-waiting and a single API across Chromium, Firefox and WebKit. Teams went remote the same season, and shared run history became the only place anyone met.

SDODS carries. Playwright underneath, not one sleep in the step library, and a cross-browser matrix that is a flag rather than a fork of the suite.

• Playwright
• Auto-waiting
• WebKit and Firefox parity
• Remote-first teams

A failure starts explaining itself

The Playwright trace viewer arrived in June and a failed run began shipping with its evidence attached, instead of a log to reconstruct the failure from. Visual baselines went mainstream beside it.

SDODS carries. Before-and-after screenshots on every UI step with a pixel diff, and a heal record that says which strategy found the element. Level 1 of the ladder is this idea, finished.

• Trace viewer
• Visual baselines
• Pixel diffing
• Step-level capture

Flakiness gets a name and a number

Suites got large enough that retries stopped hiding the problem, and quarantine, flake rate and per-test ownership entered the vocabulary. In November ChatGPT arrived and every roadmap in the industry was rewritten.

SDODS carries. Run history in a real database — SQLite or Postgres on one schema — so a flaky test is measured across runs rather than remembered by whoever was on call.

• Quarantine and budgets
• Flake rate metrics
• Test impact analysis
• ChatGPT

Models can write tests. Nobody trusts them

GPT-4 in March made test generation almost free, and the bottleneck moved the same week: reviewing a hundred generated scenarios you did not ask for is slower than writing the ten you did.

SDODS carries. Proposals. An agent writes, a person accepts, and nothing edits your working tree unasked — the rule every SDODS agent is held to.

• GPT-4
• Generated scenarios
• Self-healing locators
• Assistants in the editor

Assistants learn to read your project

The Model Context Protocol was published in November, and an assistant could be handed real tools against a real project instead of guessing from a file somebody pasted into a chat window.

SDODS carries. The SDODS MCP server: projects, features, steps, runs and results exposed as tools, so an assistant reads your suite rather than a generic one.

• Model Context Protocol
• Tool-using agents
• Local models
• Structured output

The expensive half turns out to be upkeep

Agentic coding became ordinary. Writing tests got cheap enough that the costly part was finally unmistakable: keeping a large suite alive while the application underneath it moves.

SDODS carries. Five agent roles — planner, generator, healer, upgrader, reviewer — and a healer that records why a locator healed, so the repair is reviewable instead of magic.

• Agentic coding
• Locator drift repair
• Review-first workflows
• Agents over MCP