Skip to content

← All use cases

A shared fixture workspace on a free plan with a three-workflow cap, holding twenty-three

Ten scenarios said "Forbidden" and meant "you are out of workflows"

What makes it hard

Every scenario whose background created a workflow failed. The reported error was:

text
Error: creating the hitl fixture workflow (Forbidden)
  expect(received).toBe(expected)
  Expected: 200
  Received: 403

Which reads like a permissions regression, and is not one. Probed directly, the server was saying something completely different:

json
{
  "error": "Plan limit reached",
  "message": "You've reached the 3 workflow limit on the free plan.",
  "upgradeRequired": "starter",
  "current": 23,
  "limit": 3
}

Ten scenarios pointed an investigation at authorisation for most of a day while the cause was billing. The step had wrapped the response in the HTTP reason phrase and thrown the body away.

The second half is worse than the first. Of the 23 workflows, two were test artefacts. The other 21 were human work, created over three weeks by people using the same workspace — and the environment file itself describes that environment as shared with people. So the "every creating scenario registers cleanup" rule was not what failed. The environment design failed: three workflows is not headroom for a 1,600-scenario suite, and it is not headroom for one scenario.

One genuine leak was visible in the list — a fixture from a deny-direction scenario which, by construction, the actor under test cannot delete.

How to cover it

Three separate lessons, and only one of them is about the application.

Fail fast, at the top, with the real message. A BeforeAll hook in the project's steps that reads the plan and the current count and aborts when the workspace is within N of its cap would have replaced the entire investigation. It is about five lines.

sdods doctor will not do this for you, and it is worth being precise about why: its variables check resolves every ${VAR} reference declared under envs/<env>.yaml vars: against the .env files and the shell. It checks that your configuration is resolvable, not that the environment behind it is usable. Those are different questions and only the first one is generic.

Assert the body next to the status. Any step that creates a fixture should surface what the server said. In feature files, that means the reason travels with the status:

gherkin
Then the response status should be 403
And the response JSON path "error" should equal "Plan limit reached"

A red run that names its own cause is worth more than a red run that is merely red.

Fixture hygiene is environment design. The roles in an RBAC fixture must differ only inside the fixture workspace — but the contents of that workspace being unpredictable is what broke here. A workspace nobody works in costs nothing and removes a whole class of failure. Then: I register cleanup {method} {string} on every creating scenario, and an owner-level teardown for the deny-direction fixtures that the actor under test deliberately cannot remove.

Do not fix this by deleting the twenty-one workflows. They are somebody's work, and that decision belongs to their owner. Move the suite, not the humans.

Scenario sketch

gherkin
@api @sanity
Scenario: The fixture workspace has headroom before the suite runs
  Given I use a leased user with role "member" for API calls
  When I send a GET request to "/billing?workspaceId={{fixtureWorkspaceId}}"
  Then the response status should be 200
  And the response JSON path "plan" should equal "{{expectedFixturePlan}}"
  When I send a GET request to "/workflows?workspaceId={{fixtureWorkspaceId}}"
  Then the response status should be 200
  And the response JSON path "items" should have at least 1 items

@api @regression
Scenario: The plan cap refuses with a message that names the cap
  Given I use a leased user with role "free" for API calls
  When I send a POST request to "/workflows" with body:
    """json
    { "name": "sdods-cap-{{runId}}", "workspaceId": "{{cappedWorkspaceId}}" }
    """
  Then the response status should be 403
  And the response JSON path "error" should equal "Plan limit reached"
  And the response JSON path "limit" should equal "3"

Run the first as its own @sanity pass before the suite: sdods run -p <slug> -e staging -t @sanity -l api. If it is red, nothing downstream of it is telling you anything about the product.

Other use cases