We register cleanups the way the guide shows:
Given I register cleanup DELETE "/posts/{{postId}}"The records are not being deleted. Looking at the run detail, the cleanup ran and the request 404'd, and the scenario is green.
Two things I would like to understand. Why does a failed cleanup not fail the scenario — I can see arguments both ways. And is there something that would have told me, short of noticing the sandbox filling up three weeks later?
Accepted answer
The first one is a deliberate trade. A cleanup failure is attached to the result and never fails the scenario, because the alternative is a green test going red for something that is not the behaviour under test. Do that a few times and people learn to ignore red, which costs more than the leaked records.
The second one is where your actual bug is. Look at what the attachment says the request was. Cleanups are stored with a description built from the method and the rendered path at registration time, so if it reads
DELETE /posts/{{postId}}then {{postId}} was not set when the cleanup was registered, and the literal placeholder is what got sent — an unresolved template renders as itself, so the request went to a path with braces in it and the server said 404, correctly.
Two ways to get there. Either the save step never ran, or it saved under a different name than the cleanup references. And there is an ordering trap: cleanups run last-in-first-out at the end of the scenario, but the path is rendered when you register, not when it runs. Registering before the create step means rendering a variable that does not exist yet.
When I send a POST request to "/posts" with body:
"""
{ "title": "hello", "userId": 1 }
"""
When I save the response JSON path "id" as "postId"
Given I register cleanup DELETE "/posts/{{postId}}"Create, save, register, in that order.
The rendered-at-registration behaviour is the part that surprises people, and it is the right choice — a cleanup that re-renders at the end would pick up whatever the variable had become by then, which for a scenario that reuses a name is a different record.
We caught our leak by checking the attachments on a nightly rather than by counting rows in the sandbox. Anything with a {{ in the cleanup description is a bug, and it is one grep.
Have the same question?
Ask it with the command you ran and the output you got, and it will be answered here.
Ask a question