Skip to content

Conversation

dimitar-kostadinov
Copy link
Contributor

@dimitar-kostadinov dimitar-kostadinov commented Sep 6, 2023

If registry is configured as pull-through cache the following proxy related prometheus metrics are exposed:

  • registry_proxy_requests_total - total incoming proxy request received for blob/manifest
  • registry_proxy_hits_total - total proxy request hits for blob/manifest
  • registry_proxy_misses_total - total proxy request misses for blob/manifest
  • registry_proxy_pulled_bytes_total - total bytes pulled from the upstream for blob/manifest
  • registry_proxy_pushed_bytes_total - total bytes pushed to the client for blob/manifest

@dimitar-kostadinov dimitar-kostadinov changed the title Add prometheus proxy related metrics Add proxy related prometheus metrics Sep 6, 2023
@dimitar-kostadinov dimitar-kostadinov changed the title Add proxy related prometheus metrics Add prometheus proxy related metrics Sep 6, 2023
@milosgajdos milosgajdos added the area/proxy Related to registry as a pull-through cache label Sep 6, 2023
@dimitar-kostadinov dimitar-kostadinov force-pushed the add/prometheus_proxy_metrics branch from 10f0892 to 1a20b1f Compare September 6, 2023 08:11
@ialidzhikov
Copy link
Contributor

@thaJeztah @milosgajdos , a friendly ping. We would like to proceed with the monitoring story on our side.

Signed-off-by: Dimitar Kostadinov <dimitar.kostadinov@sap.com>
Signed-off-by: Dimitar Kostadinov <dimitar.kostadinov@sap.com>
@dimitar-kostadinov dimitar-kostadinov force-pushed the add/prometheus_proxy_metrics branch from d1c8018 to 2ce5c81 Compare October 23, 2023 19:55
@milosgajdos
Copy link
Member

Sorry about the delay with this. I've had other priorities to look at. Hopefully this week.

@milosgajdos milosgajdos requested a review from Jamstah October 23, 2023 22:45
@Jamstah
Copy link
Collaborator

Jamstah commented Oct 24, 2023

There is no test code here, which is a shame. Looking at the code we're missing lots of tests for the proxy though.

I assume there has been a manual test? Can you provide some test results for completeness?

@dimitar-kostadinov
Copy link
Contributor Author

I assume there has been a manual test? Can you provide some test results for completeness?

Yes, here are steps for a simple manual test:

  • Checkout this PR gh pr checkout 4047
  • Start registry as pull through cache
# build
% make binaries
# prepare configuration
% mkdir -p ~/registry-test
% export REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY=~/registry-test
% export REGISTRY_PROXY_REMOTEURL=https://registry-1.docker.io
% export REGISTRY_HTTP_ADDR=":5500"
% export REGISTRY_HTTP_DEBUG_ADDR=":5501"
# start the registry as proxy cache to docker.io
% ./bin/registry serve cmd/registry/config-dev.yml
  • Configure docker to use registry mirror
{
  "registry-mirrors": [
    "http://test.registry.mirror:5500"
  ]
}
  • Update /etc/hosts file with following record
echo '127.0.0.1 test.registry.mirror' >> /etc/host
  • Pull a image % docker pull nginx
  • Get the prometeus metrics
 % curl http://test.registry.mirror:5501/metrics
...
# HELP registry_proxy_hits_total The number of total proxy request hits
# TYPE registry_proxy_hits_total counter
registry_proxy_hits_total{type="blob"} 8
registry_proxy_hits_total{type="manifest"} 3
# HELP registry_proxy_misses_total The number of total proxy request misses
# TYPE registry_proxy_misses_total counter
registry_proxy_misses_total{type="manifest"} 2
# HELP registry_proxy_pulled_bytes_total The size of total bytes pulled from the upstream
# TYPE registry_proxy_pulled_bytes_total counter
registry_proxy_pulled_bytes_total{type="manifest"} 3640
# HELP registry_proxy_pushed_bytes_total The size of total bytes pushed to the client
# TYPE registry_proxy_pushed_bytes_total counter
registry_proxy_pushed_bytes_total{type="blob"} 6.7238076e+07
registry_proxy_pushed_bytes_total{type="manifest"} 5502
# HELP registry_proxy_requests_total The number of total incoming proxy request received
# TYPE registry_proxy_requests_total counter
registry_proxy_requests_total{type="blob"} 8
registry_proxy_requests_total{type="manifest"} 3
...

Note: registry_proxy_pulled_bytes_total{type="blob"} metric is missing due to #4044

Copy link
Collaborator

@Jamstah Jamstah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, having some proof in the PR is better than nothing :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/proxy Related to registry as a pull-through cache
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants