Skip to content

feat: optimize docker builds #10925

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Aug 21, 2025
Merged

feat: optimize docker builds #10925

merged 2 commits into from
Aug 21, 2025

Conversation

lidel
Copy link
Member

@lidel lidel commented Aug 20, 2025 •
edited
Loading

Some experiments that aim to improve Docker build:

  1. ~8.6x faster incremental builds locally with BuildKit cache mounts
  2. 5-10x faster CI builds with persistent GitHub Actions and Docker Hub caching
  3. Cross-PR cache sharing via Docker Hub buildcache tag

How

  1. Uses dual caching strategy:
    • type=gha - GitHub Actions cache for same-repo builds
    • type=registry - Docker Hub cache for cross-fork/cross-workflow persistence (docker-build.yml benefiting from docker-image.yml in master)
      • The buildcache tag on Docker Hub will store the build cache layers, making subsequent builds much faster across all platforms.
      • Registry cache benefits:
        • Survives longer than GHA cache (no 7-day limit)
        • Available to forks and external contributors
        • Shared across all workflows that pull from Docker Hub
    • mode=max caches all layers, not just the final result, maximizes cache reuse potential
      • imo for Kubo's use case with complex Go builds and multiple platforms, mode=max is the better choice. The extra cache storage is worth the significant build time savings, especially for incremental changes.

Demo

Re-run docker-build.yml with warm cache:

image

Re-run docker-image.yml with warm cache:

image

TODO

  • test docker-image.yml by pushing to staging branch

lidel added 2 commits August 20, 2025 06:54
@lidel
feat(docker): optimize Dockerfile for faster builds
e62f9ff 
- add BuildKit syntax directive for advanced caching features
- implement cache mounts for Go modules and build cache
- reduce layers by combining RUN commands (5→2 in final stage)
- optimize apt-get with --no-install-recommends flag
- use COPY --chmod to avoid separate permission fixing

Performance improvements:
- incremental builds after code changes: ~8.6x faster (1m51s → 13s)
- go module/build cache persists between builds
- reduced layer count improves cache efficiency
@lidel
ci: optimize Docker builds with BuildKit caching
57a5772 
- enable BuildKit with GitHub Actions cache backend
- add Docker Hub registry cache for cross-workflow sharing
- move Docker login earlier to enable registry cache writes
- use dual cache strategy (gha + registry) for faster builds

expected improvements:
- PR builds can reuse main branch cache from Docker Hub
- rebuild after code changes ~5-10x faster with persistent cache
- cross-PR cache sharing reduces redundant builds
@lidel lidel added the skip/changelog This change does NOT require a changelog entry label Aug 20, 2025
@lidel lidel mentioned this pull request Aug 20, 2025
Open
8 tasks
@lidel lidel marked this pull request as ready for review August 20, 2025 05:39
@lidel lidel requested a review from a team as a code owner August 20, 2025 05:39
@lidel
Copy link
Member Author

lidel commented Aug 21, 2025 •
edited
Loading

I'm going to merge it and dogfood during 0.37.0-rc1 release dance.

Some extra notes:

  • the old cache swap workaround can be safely removed because
    1. Old issues were specific to type=local cache - which had unbounded growth
    2. type=gha has automatic management - GitHub enforces 10GB limit with LRU eviction
    3. type=registry is externally managed - Docker Hub handles storage/cleanup
    4. The workaround is obsolete - Modern cache backends don't need manual cleanup
  • using type=registry for both PR checks (docker-build.yml) and release images (docker-image.yml) should be fine, enables us to share cache across branches. But if we hit any issues (like rate-limiting by Docker Hub) we can always remove type=registry from docker-build.yml.

@lidel lidel merged commit c12d249 into master Aug 21, 2025
17 checks passed
@lidel lidel deleted the feat/optimize-dockerfile branch August 21, 2025 12:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@hsanjuan hsanjuan hsanjuan approved these changes

Assignees
No one assigned
Labels
skip/changelog This change does NOT require a changelog entry
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@lidel @hsanjuan