Skip to content

Bug: race condition in squashfuse mount path - command not found #428

@smoser

Description

@smoser

stacker version

v1.0.0-rc1

Describe the bug

I see this problem occasionally when building bootkit and have seen it elsewhere as well.
here is a github-actions reproduce.

The problem manifests itself like:

2023-03-02T17:01:18.1854206Z preparing image stubby-build-env...
2023-03-02T17:01:18.1987179Z couldn't find AppArmor profile lxc-container-default-cgns
2023-03-02T17:01:18.2097611Z + pkgtool install binutils gcc gnu-efi libc6-dev make tar
2023-03-02T17:01:18.2098345Z /stacker/.stacker-run.sh: 2: pkgtool: not found
2023-03-02T17:01:18.2153865Z error: run commands failed: execute failed: exit status 127
2023-03-02T17:01:18.2233898Z error: exit status 1
2023-03-02T17:01:18.2238643Z make: *** [Makefile:8: build] Error 1

the 'not found' file lives in a layer 'minbase'. they look like this:

  • docker://ubuntu:jammy
  • minbase (does an import pkgtool and runs 'pkgtool upgrade' and 'pkgtool install ca-certificates'
  • other layer (stubby-build-env) based on minbase, runs 'pkgtool' as the first thing in its build process.

bootkit builds with 'stacker recursive-build'.
here is an attempt at a stacker.yaml that would reproduce, but I have not tried it. pkgtool could really be any script.

minbase:
  build_only: true
  from:
    type: docker
    url: ${{DOCKER_BASE}}ubuntu:jammy
  import:
    - pkgtool
  run: |
    mirror=${{UBUNTU_MIRROR}}

    rel=$(. /etc/os-release && echo "${VERSION_CODENAME}")
    [ -n "$rel" ]

    cp /etc/apt/sources.list /etc/apt/sources.list.dist
    cat > /etc/apt/sources.list <<EOF
    deb [by-hash=force] $mirror ${rel} main universe
    deb [by-hash=force] $mirror ${rel}-updates main universe
    deb [by-hash=force] $mirror ${rel}-security main universe
    EOF

    cp /stacker/pkgtool /usr/local/bin

    #pkgtool upgrade
    #pkgtool install ca-certificates

rootfs:
  from:
    type: built
    tag: minbase
  run: |
    pkgtool install ca-certificates

To reproduce

I see the issue occasionally just using 'make' in https://github.com/project-machine/bootkit.
I have only seen it once or twice on github actions before now.

Expected behavior

No response

Screenshots

No response

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions