This is a messy topic. It all relates to the giant mistake that docker did originally (and was carried forward into OCI) of checksumming compressed data.

Build locally, install, push to registry problem domain

• podman build -t localhost/mybootc . (image is only in containers-storage:)
• podman run <image> bootc install to-filesystem (we call skopeo, and that always gives us a manifest with all uncompressed layers)
• Start e.g. VM based on that
• podman push localhost/mybootc quay.io/mybootc (Note: this is where layers get compressed!)
• bootc upgrade in VM

In this scenario we'll have switched from seeing the uncompressed-only layers to compressed versions for the first time.

vs opposite "production" paths

Note the above is not a usual production path, that one is where we install an image that pre-exists in a registry, i.e. the first step is podman pull and not podman build. Here, the manifest will have compressed layers from the start, and we will identify them by that digest.

TODO:

• There are tests for "only fetch changed layers" in e.g.

  bootc/ostree-ext/tests/it/main.rs

  Line 1021 in 1ba15a9

  assert_eq!(to_fetch.len(), 2);

• But it'd probably be good to have something more "end to end" that actually verified from a registry

More detail: Conceptually it's pretty normal actually even in the "build" phase to be reusing prior layers that have a compressed representation.

So containers-storage: actually has the knowledge of the mapping between any blob ⟺ diffid from the FROM that was used in the base image. But the proxy does not expose this. An important thing to understand here is that a lot of the containers/image logic is based on a Copy API...that we're not using.

Option 1:

• Change the GetManifest API in skopeo proxy to add an annotation into the layers with the "canonical" compressed digest even when it gave us a compressed one

Related issues came up in the past here that c/storage doesn't offer a way (at least by default?) to access the real original manifest (with the compressed layers). See containers/container-libs#197 (and containers/container-libs#144 ).

But, I think we can and should fix the ostree-container storage here to also track layers via diffid - when we go to do a fetch, map from the uncompressed layers to the diffids from the
manifest we're going to fetch, then we can skip downloading those etc.

See also: cgwalters/cstor-dist#3