I noticed that when the PR is in the merge queue the all-in-one workflows run for almost 20min while all others finish in 5-6min. From the logs it can be seen that we're building all architectures, which is unnecessary for a merge, but the build-binaries step still succeeds faster because it parallelizes all architectures while all-in-one builds them sequentially.

The quick fix is to change .github/workflows/ci-docker-all-in-one.yml to not build all arch on PRs and merge queue events (currently only PRs are excluded).

Longer term fix is maybe to parallelize the different arch builds similar to how build-binaries is done.