Skip to content

← All questions

Does --repeat-each 5 with --shard 1/4 repeat inside the shard or across shards?

Asked 265 views0 answers

Setting up a soak job: run the regression five times over, spread across four machines, and fail if anything is flaky.

bash
sdods run -p demo-shop -e staging -t @regression --repeat-each 5 --shard 1/4 --run-id soak-0811 --reporter blob --fail-on-flaky

What I cannot work out is whether the repeats are part of the set that gets sharded — so each machine gets a quarter of 5N tests — or whether each shard takes a quarter of the N scenarios and then repeats each of them five times locally. The wall-clock maths is the same either way, but the flakiness answer is not: repeats spread across four machines exercise four different environments, and repeats on one machine exercise the same one four times, which is a weaker test of exactly what I am trying to measure.

Has anyone actually watched this happen rather than reasoned about it? I can instrument it, but if someone already knows I would rather not.

asked by Marijn Kuiper18,840 ·

No answers yet

Nobody has answered this one. If you have hit the same thing and worked it out, that is worth writing down.

Ask a related question

Have the same question?

Ask it with the command you ran and the output you got, and it will be answered here.

Ask a question

Related