← All questions

perf.budgets validates and resolves, and a nine-second page still passes. What am I missing?

perf:
  budgets: { pageLoadMs: 4000, lcpMs: 3000, apiP95Ms: 1500 }

processes:
  - name: release-gate
    trigger: release
    tags: '@smoke or @regression'
    gates: { minPassRate: 100, maxFlaky: 0, perfBudgets: true, a11y: true }

sdods config show -p demo-shop -e staging --explain

2 answers

• 38votes✓

  Accepted answer

  You are missing nothing, and the reason it looks like plumbing is that it is plumbing with nothing on the end of it.

  The schema accepts perf.budgets. The schema accepts perfBudgets and a11y on a process gate. The resolver merges them and config show --explain will happily tell you which layer won. Then nothing at run time reads either value. There is no timing check and no accessibility hook — the gate booleans are recorded and never asked.

  So a green run is not evidence about the speed of that page. It is evidence that the scenario's assertions passed.

  This is stated on the workshop chapter; the accessibility and performance guide contradicts it and the guide is the one that is wrong.

  RNanswered by Reeta Nandalmaintainer41,280 · 22 Oct 2024
• 13votes

  If the budget matters for a release, put it somewhere that enforces it — a separate step in the pipeline that measures and exits non-zero, owned by whoever cares about the number.

  Keeping the yaml key costs nothing and documents the intent, so I would leave it. Just do not let anybody put "performance budgets enforced" on a slide because the key is in the file.

  HBanswered by Helle Borg352 · 25 Oct 2024