A suite where one scenario in eight was dark, and the report said otherwise

207 of 1,609 scenarios never executed. All 1,609 were reported as coverage.

Two mechanisms, and only one of them is visible.

@quarantine — 120 scenarios — is an ordinary tag, excluded by hand in each run recipe's tag expression. Clumsy and drift-prone, but at least an operator reading the expression can see it.

@fixme — 87 scenarios — is not visible at all. It matches the runner's special-tag pattern:

export const RUNNER_SPECIAL_TAGS =
  /^@(only|skip|fixme|fail|slow|timeout:\d+|retries:\d+|mode:(parallel|serial|default))$/;

Lint deliberately skips validating anything that matches, which is correct behaviour and not a lint hole — but it means sdods lint reports no findings over 87 uses of a tag that is not declared anywhere in the project. The runner turns each one into a skipped test no matter what tag expression is passed. Adding and not @fixme changes nothing. Passing @fixme cannot bring them back. They are unreachable without editing the feature files, and the run summary does not distinguish "deliberately parked" from "not selected".

Where they sat made it worse. Half the parked @fixme scenarios were in two modules, one of them described in the project YAML as a security surface. Forty-six of the quarantined ones were in the authorisation and multi-tenancy modules — the deny direction, which is the half that matters.

And there was no record anywhere of why any of the 207 were parked, or what would un-park them.

Make the parked set observable through one mechanism, then count the runnable set rather than the written one.

One mechanism. @quarantine is an ordinary project tag: declare it under tags.extra in sdods.project.yaml and it stops raising an unknown-tag warning, appears in tag expressions, and is selectable through sdods features list --tags. Converting the @fixme scenarios to it trades an invisible exclusion for a visible one. That is a real improvement even though it fixes nothing about the coverage — an operator can now see the dark slice.

Count what runs.

sdods run -p <slug> -e staging -t @regression --list
sdods features list -p <slug> --scenarios

--list prints the run targets and the tests that would run, then stops. Diff it against the scenario inventory and the difference is your dark slice. That number belongs in the headline, next to the total, not underneath it.

Require a reason. A parked scenario with no linked issue and no comment is indistinguishable from an abandoned one six months later. @jira:PROJ-123 and @github:123 are pattern-checked value tags — use one, so the park has an owner and an exit condition.

Two honest notes:

• quarantine is not enforced by the runner. The exclusion lives in each processes: recipe's tags: expression and is maintained by hand, which is exactly why it drifts between recipes.
• the underlying rule is the same one that makes a vacuous assertion worthless, one level up: a scenario that cannot fail is not coverage, and a scenario that never runs cannot fail.

@ui @regression @quarantine @jira:QA-1184
Scenario: A viewer cannot promote themselves to owner
  Given I use a leased user with role "viewer"
  When I navigate to the "members" page
  Then the element with test id "member-permission-{{viewerId}}" should contain "Viewer"

Parked, and parked legibly: @quarantine puts it in every tag expression that mentions it, and @jira:QA-1184 says who owns getting it back. Declare quarantine under tags.extra first, or lint warns on every use.

sdods lint -p <slug>
sdods run -p <slug> -e staging -t "@regression and not @quarantine" --list

The tag rules those scenarios still have to satisfy — exactly one layer tag, exactly one suite tag — are in tags and suites.