Build k3s overhaul #12200

dereknola · 2025-04-22T22:22:42Z

Proposed Changes

Add later CI actions of package and airgap to the Dockerfile.local, moving closer to achieving parity with the Dapper Dockerfile
Breakup version.sh script into git_version.sh and version.sh. This improves caching around build layers, as .git no longer needs to be added to build targets, all git versioning information is calculated beforehand.

Overhaul of `build_k3s.yaml` GHA workflow:

Rework workflow inputs: arch and cache. Initial support for emulated builds.
Introduce layer caching using GHCR. This save about 1min to the build times and uses GHCR (with unlimited storage) vs the dedicated GHA cache (which we only have 10GB of)
Use faster Dockerfile.local targets where possible, using native GHA calls to docker

Types of Changes

CI/Build

Verification

CI Still green
use new make targets make local-image or make local-binary

Performance

Setup is as follows:

1 is initial build with no cache (this is the most variable and dependent on download time)
2 is build with a single line change in pkg/cli (i.e. code change)
3 is build with no line changes (just to see maximum caching usage)

Target	Stage	Dapper	Old Local	New Local
Binary	1	3m 22s	3m 22s	3m 36s
	2	1m 58s	45s	40s
	3	1m 21s	2s	2s
Image	1	3m 18s	N/A	3m 33s
	2	1m 36s	N/A	1m
	3	1m 34s	N/A	3s

Testing

N/A

Linked Issues

#12136

User-Facing Change

Further Comments

Future work is required to achieve nice build emulation times, with an eye on building arm32bit and potential riscv64. Need to look at QEMU emulation cost or implement xx usage, possibly with a separate builds stage.

Seperate git and other version tags, improves caching on binary builds Signed-off-by: Derek Nola <derek.nola@suse.com>

Signed-off-by: Derek Nola <derek.nola@suse.com>

- Build binary using GHA native commands - Use internal setup-go action for e2e.yaml Signed-off-by: Derek Nola <derek.nola@suse.com>

- Be consistent in k3s artifact names - Fix sha256sum CI naming Signed-off-by: Derek Nola <derek.nola@suse.com>

Signed-off-by: Derek Nola <derek.nola@suse.com>

codecov · 2025-04-23T03:49:45Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 19.81%. Comparing base (9604f27) to head (1bded1c).
Report is 3 commits behind head on master.

❗ There is a different number of reports uploaded between BASE (9604f27) and HEAD (1bded1c). Click for more details.

HEAD has 18 uploads less than BASE

Flag BASE (9604f27) HEAD (1bded1c)

inttests 12 0

e2etests 6 0

Additional details and impacted files

@@             Coverage Diff             @@
##           master   #12200       +/-   ##
===========================================
- Coverage   45.13%   19.81%   -25.32%     
===========================================
  Files         188      184        -4     
  Lines       19219    19144       -75     
===========================================
- Hits         8675     3794     -4881     
- Misses       9310    14920     +5610     
+ Partials     1234      430      -804

Flag	Coverage Δ
e2etests	`?`
inttests	`?`
unittests	`19.81% <ø> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Signed-off-by: Derek Nola <derek.nola@suse.com>

brandond · 2025-04-23T17:06:03Z

Makefile

+	@echo "INFO: Building K3s airgap tarball..."
+	./scripts/package-airgap
+
+.PHONY: local-ci


so the new recommended dev workflow is make local-ci instead of make ci? Or does the old target still work, at least for now?

The old targets still work, I have not removed Dapper or touched the old build process. But yes, for better performance, I would recommend make local-ci or whatever subtarget you need for the dev process.

Dapper removal will be a future PR, likely only once we have successfully published a release with these new processes.

Signed-off-by: Derek Nola <derek.nola@suse.com>

GreyXor · 2025-04-30T12:36:29Z

Thanks, would that make sense to also add GOAMD64=v4 build for whom have x86_64-v4 cpu ?

* Add full ci support without Dapper * Seperate git and other version tags, improves caching on binary builds * Use new local targets for build-k3s.yaml workflow * Allow optional ghcr build caching * Build binary using GHA native commands * Use internal setup-go action for e2e.yaml * Add emulation builds to k3s-build.yaml (for arm32 and future riscv64) * Be consistent in k3s artifact names * Fix package/dockerfile warnings * Fix install script for PR installs Signed-off-by: Derek Nola <derek.nola@suse.com>

dereknola requested a review from a team as a code owner April 22, 2025 22:22

dereknola added 5 commits April 22, 2025 20:36

Add full ci support without Dapper

12debe5

Seperate git and other version tags, improves caching on binary builds Signed-off-by: Derek Nola <derek.nola@suse.com>

Use new local targets for build-k3s.yaml workflow

318e494

Signed-off-by: Derek Nola <derek.nola@suse.com>

Allow optional ghcr build caching

69c649e

- Build binary using GHA native commands - Use internal setup-go action for e2e.yaml Signed-off-by: Derek Nola <derek.nola@suse.com>

Add emulation build to k3s-build.yaml

102d355

- Be consistent in k3s artifact names - Fix sha256sum CI naming Signed-off-by: Derek Nola <derek.nola@suse.com>

Fix dockerfile warnings

6bb2ca5

Signed-off-by: Derek Nola <derek.nola@suse.com>

dereknola force-pushed the build_k3s_overhaul branch from 9dbedeb to 6bb2ca5 Compare April 23, 2025 03:36

vitorsavian previously approved these changes Apr 23, 2025

View reviewed changes

VestigeJ previously approved these changes Apr 23, 2025

View reviewed changes

Fix incorrect sourcing of new git_version.sh script

7debdfc

Signed-off-by: Derek Nola <derek.nola@suse.com>

dereknola dismissed stale reviews from VestigeJ and vitorsavian via 7debdfc April 23, 2025 17:00

vitorsavian previously approved these changes Apr 23, 2025

View reviewed changes

VestigeJ previously approved these changes Apr 23, 2025

View reviewed changes

brandond reviewed Apr 23, 2025

View reviewed changes

brandond previously approved these changes Apr 23, 2025

View reviewed changes

Fix binary checksum generation

73ecd6e

Signed-off-by: Derek Nola <derek.nola@suse.com>

dereknola dismissed stale reviews from brandond, VestigeJ, and vitorsavian via 73ecd6e April 24, 2025 16:17

Fix install script for PR installs

1bded1c

Signed-off-by: Derek Nola <derek.nola@suse.com>

dereknola force-pushed the build_k3s_overhaul branch from a4a9be6 to 1bded1c Compare April 24, 2025 19:14

brandond approved these changes Apr 24, 2025

View reviewed changes

brandond approved these changes Apr 25, 2025

View reviewed changes

VestigeJ approved these changes Apr 25, 2025

View reviewed changes

mgfritch approved these changes Apr 25, 2025

View reviewed changes

dereknola merged commit 3ce4a63 into k3s-io:master Apr 25, 2025
61 checks passed

dereknola deleted the build_k3s_overhaul branch April 28, 2025 15:51

cguertin14 mentioned this pull request May 10, 2025

new release: k3s update from v1.32.4+k3s1 to v1.33.0+k3s1 cguertin14/k3s-ansible-ha#51

Merged

dereknola mentioned this pull request Jun 9, 2025

[Release-1.31] GHCR image release #12461

Merged

dereknola mentioned this pull request Jun 9, 2025

[Release-1.32] GHCR image release #12463

Merged

eoghanlawless mentioned this pull request Jun 16, 2025

feat: add k3s rpm build spec open-edge-platform/edge-microvisor-toolkit#134

Merged

3 tasks

dereknola mentioned this pull request Jul 3, 2025

[Release-1.30] Build and push k3s image to GHCR #12576

Merged

dereknola mentioned this pull request Jul 10, 2025

[Epic] Drone to Github Actions Migration #12136

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Build k3s overhaul #12200

Build k3s overhaul #12200

dereknola commented Apr 22, 2025

Uh oh!

codecov bot commented Apr 23, 2025 •

edited

Loading

Uh oh!

brandond Apr 23, 2025

Uh oh!

dereknola Apr 23, 2025

Uh oh!

Uh oh!

GreyXor commented Apr 30, 2025

Uh oh!

Uh oh!

Build k3s overhaul #12200

Build k3s overhaul #12200

Conversation

dereknola commented Apr 22, 2025

Proposed Changes

Overhaul of build_k3s.yaml GHA workflow:

Types of Changes

Verification

Performance

Testing

Linked Issues

User-Facing Change

Further Comments

Uh oh!

codecov bot commented Apr 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

brandond Apr 23, 2025

Choose a reason for hiding this comment

Uh oh!

dereknola Apr 23, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

GreyXor commented Apr 30, 2025

Uh oh!

Uh oh!

Overhaul of `build_k3s.yaml` GHA workflow:

codecov bot commented Apr 23, 2025 •

edited

Loading