ci: Cache package manager install step #26976
Conversation
|
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. ReviewsSee the guideline for information on the review process.
If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update. |
dc9a369 to
48e05b4
Compare
|
This isn't used by Cirrus, because I want to test/settle this a bit more. However, by the one task that runs on my infra, it gives a ~1min speedup: https://cirrus-ci.com/task/5552889800687616 Also, locally it seems to work: |
48e05b4 to
fa486de
Compare
| COPY ./ci/retry/retry /usr/bin/retry | ||
| COPY ./ci/test/00_setup_env.sh ./${FILE_ENV} ./ci/test/01_base_install.sh /ci_base_install/ci/test/ | ||
|
|
||
| RUN ["bash", "-c", "cd /ci_base_install/ && set -o errexit && source ./ci/test/00_setup_env.sh && ./ci/test/01_base_install.sh"] |
There was a problem hiding this comment.
To get even more of a speedup, wouldn't we want to move these steps into RUN commands earlier in the Dockerfile, and then just generate a few separate images on the basis of CI_IMAGE_NAME_TAG/FILE_ENV?
There was a problem hiding this comment.
The packages are defined in FILE_ENV, so RUN needs to be after FILE_ENV
There was a problem hiding this comment.
I'm sorry, shouldn't be reading this before the caffeine has hit; I misread RUN here for ENTRYPOINT.
|
(For what it's worth, I'm an ACK as-is because this PR represents a strict improvement over what we have now.) |
| --build-arg "CI_IMAGE_NAME_TAG=${CI_IMAGE_NAME_TAG}" \ | ||
| --build-arg "FILE_ENV=${FILE_ENV}" \ | ||
| --tag="${CONTAINER_NAME}" \ | ||
| "${BASE_ROOT_DIR}" |
There was a problem hiding this comment.
Do we have good confidence that in Cirrus environments, local build caches will persist? If not, we'll hit slowdowns because pulling from registries is about as fast as it gets.
There was a problem hiding this comment.
Based on the docs, probably yes. Though, this can be investigated in the follow-up that actually switches over Cirrus. For now I've decided to postpone it to have this settle first.
There was a problem hiding this comment.
ACK fa486de (jamesob/ackr/26976.1.MarcoFalke.ci_cache_package_manager)
Ran ./ci/test_run_all.sh, container built successfully. DOCKER_BUILDKIT=1 is cool.
|
Testing this locally, and it seems to be working as expected. Creating ubuntu:22.04 container to run in
[+] Building 73.3s (9/9) FINISHED
=> [internal] load build definition from test_imagefile Subsequent runs take ~0s: Creating ubuntu:22.04 container to run in
[+] Building 0.1s (9/9) FINISHED
=> [internal] load build definition from test_imagefile |
fa486de ci: Cache package manager install step (MarcoFalke) Pull request description: Use the local podman or docker image cache to skip the slow `apt` step ACKs for top commit: jamesob: ACK fa486de ([`jamesob/ackr/26976.1.MarcoFalke.ci_cache_package_manager`](https://github.com/jamesob/bitcoin/tree/ackr/26976.1.MarcoFalke.ci_cache_package_manager)) Tree-SHA512: 3495346c6c862b63296d2691cc492bf52a0a99ee7fae798887c792609904546013eba788045cd508a5f669f2c52e3479c122c18a5275c87af38237a1b5c9da17
Use the local podman or docker image cache to skip the slow
aptstep