A coverage manifest, a CI guard, and the gap between planned and delivered
A green coverage guard over nine feature files that do not exist
What makes it hard
The guard ran in CI, printed the drift in its own output, and exited zero:
coverage: 87/87 modules implemented · 269 feature file(s) · 1608 scenario(s) on disk · 1640 planned
✅ every feature file belongs to a declared module.It asserted one direction — every file on disk belongs to a declared module — and never the reverse. Nine planned feature files, thirty-eight scenarios, were never written. The numbers needed to fail were already computed and on the screen.
Two of the nine mattered far more than the rest.
The suite never completed a purchase. Thirteen checkout scenarios existed and every one stopped at the session URL. Nothing proved that a payment completes, that a subscription row is written, that entitlements change, or that the upgrade prompt goes away. For a product whose revenue path is a hosted checkout, that is the single most important scenario in the repository and it was planned, counted and never built.
The deny direction of a catalogue was dropped. "What must NOT be offered" was a planned file; what shipped was a generated positive-direction outline over what is offered. Those are not substitutes, and the deny half is the half that catches a gate failing open.
Three files on disk were absent from the manifest, consistent with a consolidation nobody recorded — which is the deeper problem. A planned file that was deliberately merged and a planned file nobody wrote look identical in a directory listing.
How to cover it
sdods coverage measures a plan, which is the thing a file-count guard structurally cannot do.
sdods coverage -p <slug> -e staging --routes --endpoints --roles --uncoveredIt joins three declared things — the routes in sdods.project.yaml and its modules, each module's endpoints: (and, with --openapi, the paths of the spec named by env.api.openapi) and the pool roles: — against the scenarios that reference them, split by suite tag. --uncovered names what nothing touches, and it exits 1 when a module has no covered target.

So declare the target first. A route that exists in the module and in no scenario becomes a red build, not a line of output nobody reads. That inverts the guard's direction for free, and it does it against the product's own surface rather than against a hand-written manifest that drifts.
Around that:
- record a consolidation as a consolidation. A per-file status — implemented, consolidated into another path, or dropped with a reason — is what separates a decision from an omission.
- make the guard fail on the drift it already prints. It computed 269 against 1,640; the only missing part was an exit code.
- keep the deny direction as its own file. If it is folded into a generated positive outline it will not come back.
One honest framework note on the checkout gap: there is no iframe step in the shared library, so a card field rendered inside a payment iframe is not reachable from Gherkin today. BasePage exposes frame(name), so a project step in a page object can reach it — but the shared library will not do it for you, and pretending otherwise is how a planned file stays planned.
Scenario sketch
sdods coverage -p <slug> -e staging --uncovered
sdods features list -p <slug> --scenarios@hybrid @regression
Scenario: A completed purchase writes the subscription and lifts the gate
Given I use a leased user with role "purchaser"
When I seed via POST "/billing/checkout" with body:
"""json
{ "plan": "starter", "workspaceId": "{{purchaserWorkspaceId}}" }
"""
And I save the response JSON path "sessionUrl" as "checkoutUrl"
And I complete the hosted checkout with the test card
When I poll GET "/billing?workspaceId={{purchaserWorkspaceId}}" until JSON path "plan" equals "starter" within 60 seconds
Then the response JSON path "isPaid" should equal "true"
When I navigate to the "workspace" page
Then I should not see the text "Upgrade to create more"I complete the hosted checkout with the test card is a project step wrapping BasePage.frame(). Everything else on that scenario is the shared library, including the poll — which is the right way to wait for a webhook to land, and the reason this reads as @hybrid rather than as two disconnected tests.
Other use cases
- Testing an eight-screen auth funnel that has no test idsLogin, signup, email verification, 2FA, the device flow, OAuth consent, invitation acceptance and password reset
- A 403 that a shared cache is free to replayAuthorisation refusals on an API served through a CDN
- Plan caps, spending limits, and a gate that never firedBilling, usage and spending-limit routes behind plan entitlements
- A workflow canvas SDODS cannot dragA node-graph editor: 193 block types, edges, handles, sub-block inputs and subflow containers