Sometime this head request fails. Looks random, maybe some network issue:

shell: /bin/bash -e {0}
  env:
    REPO_SLUG_ORIGIN: moby/buildkit
    LINUXKIT_BINFMT: v0.8
    BUILDKIT_HOST: tcp://0.0.0.0:1234
    BUILDKIT_PORT: 1234
    PLATFORMS: linux/amd64,linux/arm/v7,linux/arm64,linux/s390x,linux/ppc64le
    PREFER_BUILDCTL: 1
    PROGRESS: plain
    IID: buildkit-tests
    IID_FILE: /tmp/docker-iidfile
    OCI_FILE: /tmp/docker-ocifile
    VOL: /tmp/.buildkit-cache-test

buildctl build --progress $PROGRESS --frontend=dockerfile.v0 \
    --local context=. --local dockerfile=. \
    --opt target=integration-tests \
    --import-cache type=local,src=/tmp/.buildkit-cache/integration-tests \
    --output type=docker,name=$IID,dest=$IID_FILE

#1 [internal] load .dockerignore
#1 transferring context: 56B done
#1 DONE 0.0s

#2 [internal] load build definition from Dockerfile
#2 transferring dockerfile: 12.90kB done
#2 DONE 0.0s

#3 resolve image config for docker.io/docker/dockerfile:1.1-experimental
#3 ERROR: failed to do request: Head https://registry-1.docker.io/v2/docker/dockerfile/manifests/1.1-experimental: EOF
------
 > resolve image config for docker.io/docker/dockerfile:1.1-experimental:
------
error: failed to solve: rpc error: code = Unknown desc = failed to solve with frontend dockerfile.v0: failed to solve with frontend gateway.v0: failed to do request: Head https://registry-1.docker.io/v2/docker/dockerfile/manifests/1.1-experimental: EOF
Error: Process completed with exit code 1.

I have created the build workflow and the docker image is now built using our Docker action. Artifacts are upload locally through actions/upload-artifact GitHub action and release on GitHub is done through GitHub Release action.

Now workflows only use our Docker actions instead of buildctl and dockerfile-frontend workflow has been added.

Leaving summary here, mostly what we discussed in slack already. Again, apologies for not reacting sooner.

I think we should start this work with refactoring the scripts in ./hack directory. They were created before buildx (therefore used buildctl where full BuildKit was needed, eg. multi-platform), and needed to work without BuildKit in Docker as well as BuildKit was just being developed and could not be expected to be installed. The latter we have gradually already removed(we used to have duplicate Dockerfiles) but still exists in hack scripts. It doesn't make sense to migrate this old buildctl logic into new CI.

We should replace it with buildx. Maybe it would be good if main flows also worked with DOCKER_BUILDKIT=1 docker build . but not high priority. Some flows that work well with buildx bake may start using it. For example, vendor, lint, code generation. This is not a requirement, but where it is as easy as refactoring the script we can just skip the script part. In https://github.com/tonistiigi/fsutil I've already written some of the similar targets for a go project.

The goal is to have the dev flow covered with containerized actions that can be run without a special host setup, without much performance overhead. That means that we should avoid docker/build-push-action@v2 in here as that duplicates logic from the steps that can be invoked locally. We can use other actions, of course, setup-buildx, login etc. CI should invoke the same containerized flows that are provided for dev. An advanced user may use their local setup as well if they know how to, eg. go test works fine in this repo and doesn't require a special invocation wrapper.

The above also means that we shouldn't have things using actions/setup-go etc. to install host dependencies. This is also so that we dogfood our own tools and see if there are disadvantages and can improve them.

We should build from git. This improves caching precision and makes sure there are no leaked files.

Regarding caching, as we discussed, it looks like it is possible to use the keys and scope of actions/cache to cache both master and PRs in a similar manner as the current Travis cache, without a special token server and registry. The way the caching works currently is that there are some sequential steps that run and export cache. And then the rest of the steps run in parallel (eg the test matrix) without rebuilding things again in separate vms. As long as buildkit doesn't support fully distributed mode we should replicate this logic and chain steps. Also, no need to test if build fails, or try to build release images if tests fail. Not sure if splitting the workflow files makes sense.

For the cache, using type=local and compressing the directory isn't optimal but seems to be good enough based on your timings. To use a registry backend for cache, we would need to extend https://github.com/tonistiigi/ci-token-server what probably doesn't make sense. Possibly we can make a custom backend for Github cache in the future or something better. The worst part about the current cache is that it can only be used in the CI and not locally, unlike the registry cache that I can use to build locally using the cache from master CI.

We don't need to do this all at once. Can just pick one section, fix the ./hack part, then add GH workflow invocation. We can run Travis and GH actions in parallel for some migration time. Or some steps in travis and other in GHA.

@tonistiigi

Some flows that work well with buildx bake may start using it.

Maybe we could take a look at https://github.com/crazy-max/ghaction-docker-buildx-bake and move it to docker org?

fix the ./hack part

I will work on it and let you know about my progress.

We can run Travis and GH actions in parallel for some migration time.

Yes that's what I thought when I started to work on it.

Feel free to close this PR and thanks for your report!

Closing this one in favor of #1843