Scenario Outline with 20 Examples rows, or a CSV dataset? What do reviewers expect here

Asked 9 Jun 202111,260 views3 answers

27votes

We have a shipping tax table: twenty countries, each with a rate and an expected label on the confirmation page. I wrote it as a Scenario Outline and the feature file is now a wall of pipes — the twenty rows are longer than the rest of the file put together, and two of the rates are different on staging than they are on production.

I could move it into a CSV. But then the feature file no longer says what the expected behaviour is, and the whole point of Gherkin was that a person can read it.

Where do people draw the line? Is there a row count where an Outline stops being the right thing?

data data-driven

asked by Jide Okonkwo11,745 · 9 Jun 2021

3 answers

22votes
It is not a row count, it is whether the table is documentation or input.
An Outline earns its place when the rows are the requirement — a handful of variants a reader needs to see to understand the behaviour. A dataset earns its place when the rows are many, or when they differ per environment. Yours does both: twenty rows nobody reads line by line, and two values that change with the environment. That settles it.
gherkin
```
Given I load dataset "shipping" where "country" is "DE"
When I check out with "{{firstName}}" "{{lastName}}" "{{postalCode}}"
Then I should see the text "{{expectedTax}}"
```
Then keep a small Outline for the two or three cases that document the behaviour — zero-rated, standard, and whatever the interesting edge is — and let the dataset carry the other seventeen as coverage.
The environment half is the part that does not survive an Outline at all. data/common/shipping.csv is the baseline and data/staging/shipping.csv overrides it, and the feature file does not have to know.
Data-driven design
answered by Priya Venkateshmaintainer29,604 · 10 Jun 2021
8votes
One thing to know before you move: an Outline reports each row as its own scenario, so a failure names the row. A dataset-driven scenario is one scenario, and a failure names the scenario.
That is a real loss, and the mitigation is to give the dataset a stable identifier column and put it somewhere the failure will show it — in the step arguments, or in the assertion text. If the only thing your report says is "checkout applies the right tax" and it failed, you are opening a trace to find out which country.
Outlines have a # title-format: comment for the same reason, so the generated scenario titles are readable.
answered by Hsu Wei-lin15,230 · 16 Jun 2021
6votes
Also tag it @data:shipping. It does not do anything at run time, but lint checks the dataset exists, and when somebody edits the CSV you can run everything that depends on it:
bash
```
sdods run -p shop -e staging -t "@data:shipping"
```
answered by Lena Hartwig10,388 · 24 Sept 2021

Have the same question?

Ask it with the command you ran and the output you got, and it will be answered here.

Ask a question

Scenario Outline with 20 Examples rows, or a CSV dataset? What do reviewers expect here

3 answers

Related