Example threads. These questions and answers were written by the SDODS team to document real problems and how to fix them. The people named are illustrative, not real users. Questions asked on this site are marked new.
First week of using the generator role. sdods proposals show <id> gives me a diff and a manifest and I am reading it the way I read any PR, which I suspect is the wrong instinct — a model gets different things wrong than a colleague does.
What are the failure modes that are specific to a generated proposal?
The one that costs the most is scope. A generator asked about checkout will occasionally decide a step in the login module needs "fixing" on the way past. Look at the file list before the diff, and reject anything that touches a module you did not ask about — it is never worth untangling.
After that, in the order they bite:
records the step_find suggestions the agent saw, so you can tell a genuine gap from a phrasing the model preferred.
accept runs lint for you, so this one is cheap to get wrong.Checklist version: Reviewing agent proposals.
Locators, specifically, because that is my corner of this.
The healer works from an ARIA snapshot and locator statistics. When the element it needs is not in that snapshot — behind a portal, inside a canvas, rendered after the snapshot was taken — it falls back to whatever it can see, and what it can see is CSS. So a CSS locator in a proposal is not laziness, it is a signal that the snapshot was missing something.
Do not just tidy it into a role locator and accept. Find out why the accessible name was not there, because a missing accessible name is usually a real defect in the page.
A heal proposal must also never loosen an assertion to make a scenario pass. That one is an automatic reject.
Read the bottom of the manifest too. It records the model, the turns and the cost, and which scenarios the agent ran and with what result.
A proposal that took thirty turns and never ran the scenario it patched is a different object from one that took four and re-ran green, even when the diffs look the same.
Wrap commands and output in triple backticks and they will render as a code block.
Checking whether you’re signed in…
Have a different question? Ask it.