← All questions

Why is a completely made-up tag only a warning while @jira:demo12 fails the build?

✖ features/cart/cart.feature:22  tags/jira  @jira:demo12 must look like PROJ-123.
✖ features/cart/cart.feature:31  tags/skip  @skip:safari must name a browser (chromium, firefox, webkit, mobile-chrome, mobile-safari).
⚠ features/cart/cart.feature:8  tags/unknown  Unknown tag @wip. Declare it under tags.extra or a module's tags in sdods.project.yaml.
⚠ features/cart/cart.feature:14  tags/unknown  Unknown value tag @foo:bar.
2 feature file(s), 2 error(s), 2 warning(s)

4 answers

• Accepted answer

  That is right, and the boundary is worth spelling out because the set of "known keys" is fixed while the set of known plain tags is yours.

  Errors — the key is recognised and the value is checked against the project:

  • @env: against envs.available
  • @user: against tags.roles (skipped when that list is empty)
  • @data: against the keys of data.sources
  • @jira: and @github: against their patterns
  • @skip: against the supported browsers

  Warnings — nothing is claimed, so nothing can be broken:

  • a plain tag that is not a layer, not a suite, not in tags.extra and not in any module's tags
  • a value tag whose key is not one of the recognised ones at all
  • @har: naming a file that has not been recorded yet, which is a legitimate intermediate state

  And a small set is passed through untouched because the runner owns them, not lint: @only, @skip, @fixme, @fail, @slow, @timeout:30000, @retries:2, @mode:serial.

  Note @skip bare is in that list and @skip:safari is not. Bare @skip is the runner's own "do not run this"; @skip:<browser> is ours, and it is validated.

  RNanswered by Reeta NandalSDODS team

  Reply

• It is not "how wrong is this", it is "does SDODS have to do something with it".

  A tag it does not recognise at all — @wip, @foo:bar — is inert. Nothing reads it, nothing branches on it, the run is identical with or without it. So it warns: you probably meant something, and the warning makes the typo visible, but there is nothing to get wrong at run time.

  A tag whose key it does recognise is a promise about behaviour. @jira: becomes a link in the run viewer. @skip: removes the scenario from a browser. @user: leases a pool account before the context starts. If the value is wrong, that behaviour silently does not happen — @skip:safari excludes the scenario from nothing at all, because there is no engine called safari, and you find out when webkit goes red. That is worth stopping for.

  Short version: unknown key, warning. Known key, bad value, error.

  BLanswered by Bea Lindqvist

  Reply

• The practical follow-on is that warnings do not police themselves. @wip warns forever and everybody stops reading the output. Either declare the tag:

  yamlCopy

  tags:
    extra: [mock, visual, a11y, perf, data-driven, pool, wip]

  or run --strict in the pipeline so warnings are exit 3 and the drift cannot accumulate. Doing neither is how you end up with forty warnings and a team that has learned to scroll past them.

  LHanswered by Lena Hartwig

  Reply

• One more that bit us: a tag counts as declared if it appears in any module's tags, not only the module the file happens to live in. So @cart on a scenario under features/auth/ does not warn as unknown — it is a real tag, just in the wrong place. You get module/tags telling you the auth tag is missing instead, which reads oddly until you know why.

  RKanswered by Renata Kohl

  Reply

Have a different question? Ask it.