Skip to content

Conversation

microyahoo
Copy link
Contributor

The garbage-collect should remove unsed layer link file

P.S. This was originally contributed by @m-masataka, now I would like to take over it.
Thanks @m-masataka efforts with PR #2288

Signed-off-by: Liang Zheng zhengliang0901@gmail.com

@@ -100,3 +100,18 @@ func (v Vacuum) RemoveRepository(repoName string) error {

return nil
}

// RemoveLayer removes a layer link path from the storage
func (v Vacuum) RemoveLayer(repoName string, dgest digest.Digest) error {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
func (v Vacuum) RemoveLayer(repoName string, dgest digest.Digest) error {
func (v Vacuum) RemoveLayer(repoName string, dgst digest.Digest) error {

nit: dgst is used everywhere else, just follow suite here.

Copy link
Collaborator

@squizzi squizzi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There should be a way to test for this in https://github.com/distribution/distribution/blob/main/registry/storage/garbagecollect_test.go, I'd like to see a test added for this where a link file is created and then checked for deletion after a successful GC run.

@microyahoo
Copy link
Contributor Author

There should be a way to test for this in https://github.com/distribution/distribution/blob/main/registry/storage/garbagecollect_test.go, I'd like to see a test added for this where a link file is created and then checked for deletion after a successful GC run.

hi @squizzi, I will add it ASAP.

The garbage-collect should remove unsed layer link file

P.S. This was originally contributed by @m-masataka, now I would like to take over it.
Thanks @m-masataka efforts with PR distribution#2288

Signed-off-by: Liang Zheng <zhengliang0901@gmail.com>
@microyahoo
Copy link
Contributor Author

microyahoo commented Jul 2, 2024

=== RUN   TestGCWithUnusedLayerLinkPath
unusedlayerlink
manifest tags path of repository unusedlayerlink does not exist

0 blobs marked, 4 blobs and 0 manifests eligible for deletion
blob eligible for deletion: sha256:91321a43b4fc8f906ebcdf10343a8179773d63df38ab5b7326536129b29239b4
time="2024-07-03T00:14:27+08:00" level=info msg="Deleting blob: /docker/registry/v2/blobs/sha256/91/91321a43b4fc8f906ebcdf10343a8179773d63df38ab5b7326536129b29239b4" go.version=go1.22.1 instance.id=0dcba6c7-7a3d-4eb8-b374-68f35e39833d
blob eligible for deletion: sha256:c0f88f158f4e9b87dc3e54d7320ed21f82cef632379144a56e1c8ff63f4ea87c
time="2024-07-03T00:14:27+08:00" level=info msg="Deleting blob: /docker/registry/v2/blobs/sha256/c0/c0f88f158f4e9b87dc3e54d7320ed21f82cef632379144a56e1c8ff63f4ea87c" go.version=go1.22.1 instance.id=0dcba6c7-7a3d-4eb8-b374-68f35e39833d
blob eligible for deletion: sha256:c7cb042dc0bdd38b66aa3eff84ae109902245b12ca9a1cd8a3588c764513f20f
time="2024-07-03T00:14:27+08:00" level=info msg="Deleting blob: /docker/registry/v2/blobs/sha256/c7/c7cb042dc0bdd38b66aa3eff84ae109902245b12ca9a1cd8a3588c764513f20f" go.version=go1.22.1 instance.id=0dcba6c7-7a3d-4eb8-b374-68f35e39833d
blob eligible for deletion: sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
time="2024-07-03T00:14:27+08:00" level=info msg="Deleting blob: /docker/registry/v2/blobs/sha256/e3/e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" go.version=go1.22.1 instance.id=0dcba6c7-7a3d-4eb8-b374-68f35e39833d
time="2024-07-03T00:14:27+08:00" level=info msg="Deleting layer link path: /docker/registry/v2/repositories/unusedlayerlink/_layers/sha256/c0f88f158f4e9b87dc3e54d7320ed21f82cef632379144a56e1c8ff63f4ea87c/link" go.version=go1.22.1 instance.id=0dcba6c7-7a3d-4eb8-b374-68f35e39833d
time="2024-07-03T00:14:27+08:00" level=info msg="Deleting layer link path: /docker/registry/v2/repositories/unusedlayerlink/_layers/sha256/c7cb042dc0bdd38b66aa3eff84ae109902245b12ca9a1cd8a3588c764513f20f/link" go.version=go1.22.1 instance.id=0dcba6c7-7a3d-4eb8-b374-68f35e39833d
time="2024-07-03T00:14:27+08:00" level=info msg="Deleting layer link path: /docker/registry/v2/repositories/unusedlayerlink/_layers/sha256/e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855/link" go.version=go1.22.1 instance.id=0dcba6c7-7a3d-4eb8-b374-68f35e39833d
--- PASS: TestGCWithUnusedLayerLinkPath (0.94s)

@milosgajdos milosgajdos merged commit 10d90f7 into distribution:main Jul 2, 2024
@thewolt
Copy link
Contributor

thewolt commented Jul 29, 2024

@microyahoo is it normal that there is no dry-run protection when deleting the deleteLayerSet?

like adding inside the loop :

		if opts.DryRun {
			continue
		}

before the RemoveLayer statement ?

I was testing 3.0.0-beta.1 with the --dry-run option and i got a lot of Deleting layer link path: logs 😱

@microyahoo
Copy link
Contributor Author

@microyahoo is it normal that there is no dry-run protection when deleting the deleteLayerSet?

like adding inside the loop :

		if opts.DryRun {
			continue
		}

before the RemoveLayer statement ?

I was testing 3.0.0-beta.1 with the --dry-run option and i got a lot of Deleting layer link path: logs 😱

Hi @thewolt, I will fix it ASAP, thanks for your reminder.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants