Skip to content

Bug: problem re-building from squashfs due to cache change #454

@smoser

Description

@smoser

stacker version

v1.0.0-rc4-8e267fc

Describe the bug

Given a stacker file shown below, the following produces error.

  1. stacker build --layer-type=squashfs --substitute=NUMBER=1
  2. stacker build --layer-type=squashfs --substitute=NUMBER=2

To reproduce

  1. Configuration

    • Make a local copy of a published squashfs image. This step isn't necessary, the bug reproduces with remote url also.

      $ ref="docker://zothub.io/machine/bootkit/rootfs:0.0.5.230327-squashfs"
      $ lref="rootfs:latest-squashfs"
      $ skopeo copy "$ref" oci:oci.d:"$lref"
      
    • create stacker.yaml with the following:

      base:
        from:
          type: docker
          url: "oci:oci.d:rootfs:latest-squashfs"
      
      rootfs:
        from:
          type: built
          tag: base
        run: |
          echo hello-${{NUMBER:1}}
  2. Run stacker clean just to make sure state is clean.

  3. Run stacker build --layer-type=squashfs --substitute=NUMBER=1 - this will pass.

  4. Run stacker build --layer-type=squashfs --substitute=NUMBER=2 - This will fail with the error below.

$ stacker --debug build --layer-type=squashfs --substitute=NUMBER=2
stacker version v1.0.0-rc4-8e267fc
usernsexec-ing [u 0 1000 1 1 100001 65535 g 0 1000 1 1 100001 65535 -- /usr/local/bin/stacker --internal-userns --debug build --layer-type=squashfs --substitute=NUMBER=2]
stacker version v1.0.0-rc4-8e267fc
initializing stacker recipe: stacker.yaml
substituting $NUMBER to 2
substituting $STACKER_ROOTFS_DIR to /home/smoser/src/machine/trust/live/roots
substituting $STACKER_STACKER_DIR to /home/smoser/src/machine/trust/live/.stacker
substituting $STACKER_OCI_DIR to /home/smoser/src/machine/trust/live/oci
substituting $STACKER_WORK_DIR to 
stacker build order:
0 build /home/smoser/src/machine/trust/live/stacker.yaml: requires: []
building: 0 /home/smoser/src/machine/trust/live/stacker.yaml
substituting $NUMBER to 2
substituting $STACKER_ROOTFS_DIR to /home/smoser/src/machine/trust/live/roots
substituting $STACKER_STACKER_DIR to /home/smoser/src/machine/trust/live/.stacker
substituting $STACKER_OCI_DIR to /home/smoser/src/machine/trust/live/oci
substituting $STACKER_WORK_DIR to 
Dependency Order [base rootfs]
preparing image base...
overlay-dirs, possibly modified after import: []
loading oci:oci.d:rootfs:0.0.5.230327-squashfs
Copying blob 25c2c00faedd skipped: already exists  
Copying blob bcd9b263edd3 skipped: already exists  
Copying blob c2c670f1c1af skipped: already exists  
Copying config 3bd19ef6e2 done  
Writing manifest to image destination
Storing signatures
found cached layer base-squashfs
preparing image rootfs...
overlay-dirs, possibly modified after import: []
cached: types.Layer{From:types.ImageSource{Type:"built", Url:"", Tag:"base", Insecure:false}, Imports:types.Imports(nil), OverlayDirs:types.OverlayDirs(nil), Run:types.StringList{"echo hello-1\n"}, Cmd:types.Command(nil), Entrypoint:types.Command(nil), FullCommand:types.Command(nil), BuildEnvPt:[]string(nil), BuildEnv:map[string]string(nil), Environment:map[string]string(nil), Volumes:[]string(nil), Labels:map[string]string(nil), GenerateLabels:types.StringList(nil), WorkingDir:"", BuildOnly:false, Binds:types.Binds(nil), RuntimeUser:"", Annotations:map[string]string(nil), OS:(*string)(0xc00034a5b0), Arch:(*string)(0xc00034a5c0)}
new: types.Layer{From:types.ImageSource{Type:"built", Url:"", Tag:"base", Insecure:false}, Imports:types.Imports(nil), OverlayDirs:types.OverlayDirs(nil), Run:types.StringList{"echo hello-2\n"}, Cmd:types.Command(nil), Entrypoint:types.Command(nil), FullCommand:types.Command(nil), BuildEnvPt:[]string(nil), BuildEnv:map[string]string(nil), Environment:map[string]string(nil), Volumes:[]string(nil), Labels:map[string]string(nil), GenerateLabels:types.StringList(nil), WorkingDir:"", BuildOnly:false, Binds:types.Binds(nil), RuntimeUser:"", Annotations:map[string]string(nil), OS:(*string)(0xc00034ba70), Arch:(*string)(0xc00034ba80)}
cache miss because layer definition was changed
lxc rootfs overlay arg overlayfs:/home/smoser/src/machine/trust/live/roots/base/overlay:/home/smoser/src/machine/trust/live/roots/sha256_c2c670f1c1af96cab5865c25ba566095513887b1d2be75375930cc7b553465c9/overlay:/home/smoser/src/machine/trust/live/roots/sha256_bcd9b263edd3ed4c6c7f5f5e91bcfcc4887c65c6570ccd63d0fee59484fde4ad/overlay:/home/smoser/src/machine/trust/live/roots/sha256_25c2c00faedd2c307ea94c1146337e8aabd53edc3085b0f26486f955ede5eb4d/overlay:/home/smoser/src/machine/trust/live/roots/rootfs/overlay
stacker version v1.0.0-rc4-8e267fc
stacker subcommand: [/usr/local/bin/stacker --oci-dir /home/smoser/src/machine/trust/live/oci --roots-dir /home/smoser/src/machine/trust/live/roots --stacker-dir /home/smoser/src/machine/trust/live/.stacker --storage-type overlay --internal-userns --debug internal-go check-aa-profile lxc-container-default-cgns]
bind mounting /home/smoser/src/machine/trust/live/.stacker/imports/rootfs into container
lxc rootfs 20230404005130.476 ERROR    cgroup2_devices - ../src/lxc/cgroups/cgroup2_devices.c:bpf_program_load_kernel:332 - Operation not permitted - Failed to load bpf program: 
lxc rootfs 20230404005130.476 ERROR    cgfsng - ../src/lxc/cgroups/cgfsng.c:__cgfsng_delegate_controllers:3341 - Resource busy - Could not enable "+memory +pids" controllers in the unified cgroup 9
lxc rootfs 20230404005130.476 ERROR    cgfsng - ../src/lxc/cgroups/cgfsng.c:__cgfsng_delegate_controllers:3341 - Resource busy - Could not enable "+memory +pids" controllers in the unified cgroup 9
error: run commands failed: execute failed: exit status 255
stackerbuild.io/stacker/pkg/stacker.(*Builder).build
	/stacker-tree/pkg/stacker/build.go:471
stackerbuild.io/stacker/pkg/stacker.(*Builder).BuildMultiple
	/stacker-tree/pkg/stacker/build.go:568
main.doBuild
	/stacker-tree/cmd/stacker/build.go:117
github.com/urfave/cli.HandleAction
	/stacker-tree/.build/gopath/pkg/mod/github.com/urfave/cli@v1.22.12/app.go:524
github.com/urfave/cli.Command.Run
	/stacker-tree/.build/gopath/pkg/mod/github.com/urfave/cli@v1.22.12/command.go:175
github.com/urfave/cli.(*App).Run
	/stacker-tree/.build/gopath/pkg/mod/github.com/urfave/cli@v1.22.12/app.go:277
main.main
	/stacker-tree/cmd/stacker/main.go:324
runtime.main
	/usr/lib/go/src/runtime/proc.go:250
runtime.goexit
	/usr/lib/go/src/runtime/asm_amd64.s:1598
error: exit status 1
stackerbuild.io/stacker/pkg/container.MaybeRunInNamespace
	/stacker-tree/pkg/container/userns.go:102
main.main.func3
	/stacker-tree/cmd/stacker/main.go:319
github.com/urfave/cli.(*App).Run
	/stacker-tree/.build/gopath/pkg/mod/github.com/urfave/cli@v1.22.12/app.go:264
main.main
	/stacker-tree/cmd/stacker/main.go:324
runtime.main
	/usr/lib/go/src/runtime/proc.go:250
runtime.goexit
	/usr/lib/go/src/runtime/asm_amd64.s:1598

Expected behavior

No response

Screenshots

No response

Additional context

This is likely related to

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions