Enable MD5 check on GCS driver #4586
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Apparently you can upload 0-size content wihtout GCS reportin any errors back to you. This is something a lot of our users experienced and reported. See here for at least one example: github.com/distribution/issues/3018 This sets tbe MD5 sum on the uploaded content which should rectify things according to the docs: https://pkg.go.dev/cloud.google.com/go/storage#ObjectAttrs Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
joaodrp
approved these changes
Feb 28, 2025
|
PTAL @thaJeztah |
thaJeztah
approved these changes
Mar 1, 2025
| @@ -425,6 +426,10 @@ func (d *driver) putContent(ctx context.Context, obj *storage.ObjectHandle, cont | |||
| if _, err := bytes.NewReader(content).WriteTo(wc); err != nil { | |||
| return err | |||
| } | |||
| h := md5.New() | |||
There was a problem hiding this comment.
Would it be worth adding a comment here (to save future visitors from having to go through git history?
There was a problem hiding this comment.
LOL autocorrect; it's really git I meant, not hot (edited) 😆
There was a problem hiding this comment.
Makese sense: 7884c71
Adding a code comment that explains setting MD5 Sum field. Signed-off-by: Milos Gajdos <milosthegajdos@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Apparently, you can upload 0-size content without GCS reporting any errors back to you. Registry operations fail in these situations
This is something a lot of our users experienced and reported. See here for at least one example:
#3018
This PR sets the MD5 sum on the uploaded content which should rectify things according to the docs:
https://pkg.go.dev/cloud.google.com/go/storage#ObjectAttrs
The alternative would be to set
CRC32Cbut that's not been verified unlike the MD5 which is used by GitLab in their registry.Closes: #3018