Skip to content

[release-5.27] Preparing 5.27 backport #2075

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 52 commits into from
Aug 7, 2023

Conversation

mtrmac
Copy link
Collaborator

@mtrmac mtrmac commented Aug 4, 2023

Looking at #2074, CI is not running on the new release-5.27 branch yet.

This:

I.e. this should be all that is necessary to tag a tested 5.27.0 release — except that this PR is filed against the 5.26 branch, to trigger a test run.

If the test passes, this can be merged into the 5.27 branch, and (the right commit) tagged as c/image 5.27.0. (Alternatively, if the move to 5.27 were unreasonable, a variant of #2074 can be used on the release-5.26 branch.

@mtrmac
Copy link
Collaborator Author

mtrmac commented Aug 5, 2023

Rebased with updates from #2071.

@flouthoc
Copy link
Contributor

flouthoc commented Aug 5, 2023

@mtrmac Could you point this exact PR to a newer release-5.27 branch and we can merge this one.

flouthoc and others added 25 commits August 5, 2023 18:13
`c/image` uses `Instance(` API to get the required details of an
instance while performing replication `Platform` of instance is needed
hence `ListUpdate` must include platform.

Needed by: containers#1987

Signed-off-by: Aditya R <arajan@redhat.com>
TryReusingBlob now contains a new option `RequiredCompression` which
filters the blob by checking against a compression of the blob which is
considerd to be resued, in case `RequiredCompression` is set and `info`
of the blob being reused does not matches, no blob is returned.

Signed-off-by: Aditya R <arajan@redhat.com>
Fixes: containers#2023 (comment).

`assert.Equal` expects `assert.Equal(t, expected, actual)` instead of `assert.Equal(t, actual, expected)`.

Ref:https://pkg.go.dev/github.com/stretchr/testify/assert#Assertions.Equal

Signed-off-by: Aditya R <arajan@redhat.com>
Setting SystemContext.ArchitectureChoice to "" does not mean "match any/the first platform";
it's the default behavior of SystemContext, and it means "choose for the current runtime
architecture". (Originally discussed in
containers#1789 (comment) )

I.e. on amd64 these two test cases are redundant with the first two instances above,
and on other architectures (notably ARM) they cause failures.

So just drop them.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
…gorithmNames

There is a need to read annotations of a particular instance to get its
compression. Expose `Annotations` as a read-only field.

Needed By: containers#1987

Signed-off-by: Aditya R <arajan@redhat.com>
- Generally I discourage unsing named return values, it's easy
  to miss that it wasn't set
- I have no idea what the "in the middle of a multi-streamed copy"
  paragraph refers to.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
... so that we don't have to carry it around in extra
parameters. Migrating individual functions will follow.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
... so that we can eliminate three parameters.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
... so that the code looks the same here and in possible callees.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Use options.Progress and ProgressInterval directly.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Use c.options.OciDecryptConfig directly.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Use c.options.OciEncryptConfig directly.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Use c.options.DownloadForeignLayers directly.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Use ic.c.options.OciEncryptLayers directly.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
So that we don't need to pass it around in a parameter.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Use copier.unparsedToplevel now that it exists.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
... so that we don't need to carry it around in parameters.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
It is no longer used.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
flouthoc and others added 15 commits August 5, 2023 18:13
Signed-off-by: Aditya R <arajan@redhat.com>
Implement `instanceCopyClone` for multiple compression.

Signed-off-by: Aditya R <arajan@redhat.com>
While performing copy, set a custom compression passed generated from
prepareInstanceCopies.

Signed-off-by: Aditya R <arajan@redhat.com>
Following option does not provides a way to detect and exclude
compression if it already exists, this feature may be implemented in
future.

See: containers#1987 (comment)

Signed-off-by: Aditya R <arajan@redhat.com>
After implementing `instanceCopyClone` now instance to be copied can
exceed the original number of instances in the source, so amend report
message to make it more meaningful.

Signed-off-by: Aditya R <arajan@redhat.com>
Signed-off-by: Aditya R <arajan@redhat.com>
* test multiple copy requests of same compression

Signed-off-by: Aditya R <arajan@redhat.com>
Signed-off-by: Aditya R <arajan@redhat.com>
EnsureCompressionVariantsExist is only valid when working with a
manifest list.

Signed-off-by: Aditya R <arajan@redhat.com>
When copying multiple images i.e `instanceCopyClone` and no image
exists in registry in such case if clones are prepared and
copied first then original copies will reuse blobs from the clone which
is unexpected since argument `requireCompressionFormatMatch` is by
default false for `instanceCopyCopy` case.

Problem described in following PR is easily reproduceable when working
with tools such as buildah, because without this PR buildah will push
`clones` first instead of originals and later on `originals` will reuse
blobs from `clones`.

Signed-off-by: Aditya R <arajan@redhat.com>
Signed-off-by: Aditya R <arajan@redhat.com>
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
@mtrmac mtrmac force-pushed the testing-5.27-backport branch from c04b0f7 to 22f70e8 Compare August 5, 2023 16:15
@mtrmac mtrmac changed the base branch from release-5.26 to release-5.27 August 5, 2023 16:15
@mtrmac mtrmac changed the title [release-5.26] Do not merge for now: Testing 5.27 backport [release-5.26] Testing 5.27 backport Aug 5, 2023
@mtrmac mtrmac changed the title [release-5.26] Testing 5.27 backport [release-5.26] Preparing 5.27 backport Aug 5, 2023
@mtrmac mtrmac changed the title [release-5.26] Preparing 5.27 backport [release-5.27] Preparing 5.27 backport Aug 5, 2023
@mtrmac
Copy link
Collaborator Author

mtrmac commented Aug 5, 2023

Redirected to the 5.27 branch, rebased to be unambiguously after the 5.27.0-dev version bump.

(Contra #2071, SKOPEO_CI_TAG targets a branch, not a tagged version.)

Copy link
Contributor

@flouthoc flouthoc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Looks like this will need force merge, last check failed because free limit of cirrus is exceeded.

@flouthoc
Copy link
Contributor

flouthoc commented Aug 6, 2023

@rhatdan Could you force merge this please.

@mtrmac mtrmac marked this pull request as ready for review August 7, 2023 04:39
@mtrmac
Copy link
Collaborator Author

mtrmac commented Aug 7, 2023

I think that’s just a warning… but Cc: @cevich — this introduces a new release-5.27 branch, that might need configuration updates.

Force-merging now, hopefully I didn’t screw this up.

@mtrmac mtrmac merged commit b6b51e8 into containers:release-5.27 Aug 7, 2023
@mtrmac mtrmac deleted the testing-5.27-backport branch August 7, 2023 04:41
@mtrmac
Copy link
Collaborator Author

mtrmac commented Aug 7, 2023

@flouthoc
Copy link
Contributor

flouthoc commented Aug 7, 2023

@mtrmac Thanks.

@@ -6,7 +6,7 @@ env:
#### Global variables used for all tasks
####
# Name of the ultimate destination branch for this CI run
DEST_BRANCH: "main"
DEST_BRANCH: "release-5.26"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mtrmac shouldn't this be release-5.27?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This may not be important, IIRC (though my memory is really fuzzy), we don't actually use this value anywhere in any CI scripts.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mtrmac shouldn't this be release-5.27?

Totally. #2084

Thanks!


we don't actually use this value anywhere in any CI scripts.

It’s used for the git-validation command run in validate_task. In this case, where release-5.27 is a continuation of release-5.26, it is just a small optimization; in the general case it’s a bit more complex (but it should only matter in rare cases, e.g. if git-validation rules were tightened.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants