Skip to content

chore: upgrade prometheus to version 3.1.0 #4805

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 15 commits into from
Mar 14, 2025

Conversation

javiermolinar
Copy link
Contributor

@javiermolinar javiermolinar commented Mar 8, 2025

What this PR does:
It upgrades prometheus to version 3.1.0

It fixes the following warning we get now on mod tidy:

❯ go mod tidy
go: finding module for package github.com/prometheus/common/promlog
go: github.com/grafana/tempo/cmd/tempo/build imports
        github.com/prometheus/prometheus/web/api/v1 tested by
        github.com/prometheus/prometheus/web/api/v1.test imports
        github.com/prometheus/common/promlog: module github.com/prometheus/common@latest found (v0.62.0), but does not contain package github.com/prometheus/common/promlog

Breaking changes

Prometheus has deprecated go-kit/log, and now we need to pass the built-in slog:
prometheus/prometheus#14355

Now, the metric name labels are also quoted:
prometheus/prometheus#15531

`{"span.foo"="baz"}`

This can be seen in the promLabels, but it doesn't affect the displayed label

 "promLabels": "{\"resource.service.name\"=\"article-service\"}",
    {
      "labels": [
        {
          "key": "resource.service.name",
          "value": {
            "stringValue": "auth-service"
          }
        }
      ],

Checklist

  • Tests updated
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

@@ -364,7 +364,7 @@ func TestQuantileOverTime(t *testing.T) {
0,
},
},
`{p="0.5", span.foo="bar"}`: TimeSeries{
`{p="0.5", "span.foo"="bar"}`: TimeSeries{
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can still keep the name label unquoted. The simplest way to do it would be to use a modified label version.Labels.String() without quoting the name

https://github.com/prometheus/prometheus/blob/e32d89af7fb8baee53770de74793e8b2fb828f52/model/labels/labels_common.go#L43-L65

// TODO the passed logger does not include any other context attribute
// Should we standarize slog and deprecate go-kit/log too?
func New(cfg *Config, o Overrides, tenant string, reg prometheus.Registerer, _ log.Logger) (Storage, error) {
logger := slog.New(slog.NewTextHandler(os.Stdout, nil)).With("tenant", tenant)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With this we lose some logger functionality, like adding the logging line.

This is how Thanos has adapted go-kit/log to slog, maybe we can do the same: https://github.com/SungJin1212/thanos/blob/c280c664de7e7ba591ca0f46908b80a5039b1bc3/pkg/logutil/logutil.go

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if adding a new dependency to deprecate an existing one is a wise choice

sloggk "github.com/tjhop/slog-gokit"

Copy link
Contributor Author

@javiermolinar javiermolinar Mar 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can do a similar method to get the log level from the go-kit/log and include the contextual information for this PR. Eventually, we should pass the logger from the upstream code. But not for this PR, I have created a new issue for that #4819

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if adding a new dependency to deprecate an existing one is a wise choice

We're not deprecating one right now, no?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added the ability to include the source (line of code and method name). I think it's ok for this PR

@mdisibio
Copy link
Contributor

Checking the metrics-generator label sanitization, the prometheus regex hasn't changed. So that means attributes like http.status_code still become http_status_code even though those are now valid label names. Not sure the rationale, but it means Tempo functionality should be unchanged.

Could you add some spanmetrics tests that lock in this functionality, so that when/if the regex changes it will be caught? Likely we will need to add some config option at that time to generate the new label names. But for now just protecting against those regex changes should be ok. (The existing tests only cover special cases like service.instance.id -> instance).

@javiermolinar javiermolinar requested a review from mapno March 11, 2025 15:59
Copy link
Contributor

@mapno mapno left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@javiermolinar javiermolinar merged commit 7653100 into grafana:main Mar 14, 2025
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants