Team is split. Half want healing on everywhere because the nightly regression run keeps dying on locator drift and nobody is awake to fix it. Half say a run that heals is a run that lied to you.
Is there a position the maintainers recommend, or is this genuinely a taste thing?
Accepted answer
A concrete point in favour of leaving it on in the nightly specifically. A run that dies at the first broken locator tells you one thing. A run that heals past it tells you everything downstream of that locator as well, plus what broke and what it should be. You get a full picture and a suggested patch instead of a stack trace.
The two guards that keep it honest are already there without you configuring anything: every heal is annotated on the test and appended to heal.jsonl in the run directory, and negative assertions are never healed, so nothing can pass by matching nothing.
If you want a hard line, draw it per environment rather than per pipeline — heal: { enabled: false } in envs/prod-verify.yaml, and on everywhere else.
It is a taste thing at the edges and not a taste thing in the middle. The middle is: leave it on, and never let a heal go unnoticed.
What we do on our own pipelines:
sdods heal report --last --json as a post-run step, published as a build artifactThe half of your team worried about lying are right about the failure mode and wrong about the remedy. A locator that heals unnoticed is a problem with your reporting, not with healing.
Have the same question?
Ask it with the command you ran and the output you got, and it will be answered here.
Ask a question