How many workers on a two-core build box, and can I stop at the first failure?

Asked 6 Dec 20227,740 views2 answers

19votes

Two questions from the same afternoon of tuning.

Our agents have two cores. I have seen -w 8 in examples, which on our box makes the UI run slower and adds timeouts. What is the actual guidance?

And when a regression run is going to fail, I would rather know in three minutes than in twenty. Is there a fail-fast?

asked by Lars Vinter276 · 6 Dec 2022

2 answers

25votes
Rules of thumb, in the order I would apply them:
- UI: around one worker per core, and no more. Every worker is a browser; oversubscribing turns locator waits into timeouts and you get flakiness that looks like product bugs.
- API: much higher is fine, there is no browser to feed. We run the API layer at four times the core count.
- Measure it rather than believing me. summary.json has durations, and two runs at different -w settle the argument.
Fail-fast is --max-failures:
bash
```
sdods run -p demo-shop -e staging -t @regression --max-failures 1
```
One caveat that has confused people here: on a sharded run it stops that shard. The other shards carry on, because they are separate processes.
answered by Marijn Kuiper18,840 · 6 Dec 2022
12votes
From the platform side: on a build agent the core count you read is often the host's, not your quota. If the runner thinks it has 16 cores and the cgroup gives you two, the default worker count is already wrong before you touch a flag.
Pin workers in the process YAML per environment instead of in the workflow file. Then the agent's opinion of itself stops mattering, and the number is somewhere a person will find it.
answered by Sunita Kale982 · 8 Dec 2022

Have the same question?

Ask it with the command you ran and the output you got, and it will be answered here.