Skip to content

During docker push, calculate and compare sha256 digest even if it doesn't exist yet on the image #18924

@reinhrst

Description

@reinhrst

As far as I understand, the current workflow for docker push on a registry v2 is this:

  • If there is a cached sha256 digest for the image/layer, the do a HEAD call to the registry to see if it's already present; if present skip upload, continue to next layer
  • If there is no cached sha256 digest present, or it cannot be found on the registry, upload the layer through a set POST+ PATCH + PUT call (and then verify that uploading succeeded with a HEAD call). In the PUT call, the client generates the sha256 digest and sends this to the registry (to verify that it received the correct data).

Since the sha 256 is being calculated by the client anyways during upload, (if it's not cached already), it will be only minor overhead to calculate it before doing the actual upload and putting in an extra HEAD call.

The reason why I would like to have this, is that the sha256 digests are not preserved through a docker save / docker load cycle. I build my containers on CircleCI, and they suggest using docker load/save to have build cache (however even if the sha256 digest was preserved, it wouldn't help me, since the build cache directory is saved long before I push the image to the registry). As a result pushing to a v2 registry always results in uploading all layers, costing time and money (since I pay for traffic to my registry). Pushing to a v1 registry does only upload the changed layers (since it compares by imageid, not sha256 digest).

As an alternative to my request, I could imagine having a standalone command to generate the sha256 digest cache could be used.

In the slightly longer run, I think "fixing" docker load/save to cache sha256 and having a command to generate the sha256 digests (so I can do that before I run docker save) would work, since I think that even the calculating the sha256 for each image on each run is quite heavy.

Metadata

Metadata

Assignees

Labels

area/distributioncontainerd-integrationIssues and PRs related to containerd integrationkind/enhancementEnhancements are not bugs or new features but can improve usability or performance.

Projects

Status

To be closed

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions