Skip to content

Conversation

Felixoid
Copy link
Contributor

Figuring out what S3 storage does under the hood was tricky. I hope this small note will help other people set up the infrastructure.

@@ -7,6 +7,9 @@ title: S3 storage driver
An implementation of the `storagedriver.StorageDriver` interface which uses
Amazon S3 or S3 compatible services for object storage.

> **Note** When S3 storage is used, registry redirects to a pre-signed URL for the requested object.
> It allows to deploy a thin host with registry.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you mean by "thin host" here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Host with a few or even single vCPU, GiB of memory, and low network throughput.

From the documentation, it could be more explicit about what S3 storage does under the hood. For all the time I've been using registry:2, we deployed hosts with high network bandwidth and relatively big local disks for caching proxy. Because S3 just looked like exactly "storage", where objects would be transferred from the S3 to clients via the registry itself, and the traffic would be even more significant.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah gotchya! Thanks. I am not sure mentioning this adds any value to technical documentation. If we do want to add a note like that I'd suggest rephrasing it to something like

By not having to serve the stored images directly, this allows running registry on hosts with lightweight resource requirements

WDYT? CC @thaJeztah @Jamstah @squizzi

Copy link
Contributor Author

@Felixoid Felixoid Sep 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, looks good 👍

Regarding the value, I literally didn't know that the registry provides such a useful feature, and I couldn't find it anywhere in the docs. Knowing it years ago would save me a lot of slapping stuff together to decrease our costs.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I should have stressed that I do not question this:

Note When S3 storage is used, registry redirects to a pre-signed URL for the requested object.

My point was more about

It allows to deploy a thin host with registry.

Signed-off-by: Mikhail f. Shiryaev <mr.felixoid@gmail.com>
@milosgajdos milosgajdos merged commit b746186 into distribution:main Sep 19, 2024
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants