We run the regression across eight shards. The wall-clock time of the job is set by the slowest shard, and ours are consistently uneven — shard 3 carries two long visual features and lands about four minutes after shard 1, which has been idle that whole time.
--shard i/n splits by test count as far as I can tell, not by duration. We have per-scenario p95 durations sitting in the database from every previous run, so the information to do better exists.
Has anyone built anything on top of this? I am thinking of generating the shard assignment myself and passing explicit --feature lists per machine, but that is a lot of moving parts to maintain for four minutes. Curious whether anyone has gone down that road and regretted it.
asked by Marijn Kuiper