-
Notifications
You must be signed in to change notification settings - Fork 9.8k
Add script for converting PromQL tests to new syntax format #16562
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
b386ab6
to
3171bb2
Compare
Tagging @beorn7 and @NeerajGartia21 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you very much.
I still have to look at the code in detail. (Maybe @NeerajGartia21 will beat me to that. :)
However, one note: I don't think we should put this as a 1st class top level binary into the directory structure. This is just for a one time transition. I would put the file with the Go main
package somewhere underneath the promqltest
directory and give instructions in the README.md how to go run ...
it.
About the actual conversion in this PR: This should be done in a separate commit from the conversion script. Maybe even in a separate PR. And we should do a manual pass through it to see where the "strict" or the "basic" conversion makes more sense. (Bonus points for separate commits: First the auto-conversion, presumably in strict mode, and then a separate commit to manually relax the checks where it makes sense.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As a follow-up to my previous comment, here one example each for both cases:
- We want to convert a simple
eval
into a new-style test withexpect no_info
(orexpect no_warn
in other cases) in cases where the absence of an annotation is relevant. - We want to convert an old-style
eval_info
into a simple new-styleeval
in cases where we do not really care about annotations.
All the tests have to be judged similarly, as suggested in my previous comment. (It doesn't have to be rocket science, but it should generally make sense.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some preliminary code comments.
I am right now hardcoding the directory path: "promql/promqltest/testdata", do we need to make it configurable? We have 3 modes now: tolerant, basic, strict and the indentation per eval block is consistent (using reference from existing indentation) so that the things don't look weird The main method is also now under promqltest folder with instructions in the corresponding README.md |
MigrateTestData is now part of new file- test_migrate.go with a doc comment on top Do we need 2 separate PR one using strict mode and one with the other mode? |
Signed-off-by: Kapil Lamba <kapillamba4@gmail.com>
Signed-off-by: Kapil Lamba <kapillamba4@gmail.com>
Let's first finish the review here and then think about the conversion PR. (I guess we should have one PR with two commits. First the auto-conversion and then the manual adjustments.) |
I'll try to look at the code here in detail ASAP. |
Indeed, the directory should be customizable by the user. This is mainly meant for converting tests outside of this repo. (For a one-time conversion just within this repo, we did not need to publish the script here as part of the repo.) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall, the code looks good. Apologies for the delayed review.
Additionally, it would be helpful to include tests for this script. This will make it easier for reviewers to verify the functionality without running it locally and will also benefit users by providing clear examples for how to use and test the script.
Signed-off-by: Kapil Lamba <kapillamba4@gmail.com>
Signed-off-by: Kapil Lamba <kapillamba4@gmail.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me! Just a small nit.
Thanks for the script @kapillamba4 !
Let's now wait for @beorn7's review.
Co-authored-by: Neeraj Gartia <80708727+NeerajGartia21@users.noreply.github.com> Signed-off-by: Kapil Lamba <kapillamba4@gmail.com>
Thanks everyone. And apologies for the review delay. I'm still caught between several events at work and in my usual life, but I will finally be back at reviewing code next week. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you very much, and once more apologies for the delay.
Code looks good, I just have a few documentation improvements to suggest.
Signed-off-by: Kapil Lamba <kapillamba4@gmail.com>
Thanks for reviewing @beorn7, I’ve implemented the doc updates you suggested. Have also rebased the strict migration mode PR with this PR |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great, thank you very much.
* MAINTAINERS: add Jan-Otto Kröpke as an discovery/azure maintainer Signed-off-by: Jan-Otto Kröpke <mail@jkroepke.de> * [REFACTOR] PromQL: DRY `stddev` and `stdvar` functions (#16451) * DRY stddev and stdvar --------- Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com> * feat: Support 'NoTranslation' mode in OTLP endpoint (#16441) * feat: Support 'NoTranslation' mode in OTLP endpoint Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> --------- Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> * promql: histogram_fraction for bucket histograms (#16095) * promql: histogram_fraction for bucket histograms This PR extends the histogram_fraction function to also work with classic bucket histograms. This is beneficial because it allows expressions like sum(increase(my_bucket{le="0.5"}[10m]))/sum(increase(my_total[10m])) to be written without knowing the actual values of the "le" label, easing the transition to native histograms later on. It also feels natural since histogram_quantile also can deal with classic histograms. Signed-off-by: Michael Hoffmann <mhoffmann@cloudflare.com> * promql: histogram_fraction for bucket histograms * Add documentation and reduce code duplication * Fix a bug in linear interpolation between bucket boundaries * Add more PromQL tests Signed-off-by: Michael Hoffmann <mhoffmann@cloudflare.com> * Update docs/querying/functions.md Co-authored-by: Björn Rabenstein <github@rabenste.in> Signed-off-by: Michael Hoffmann <mhoffm@posteo.de> --------- Signed-off-by: Michael Hoffmann <mhoffmann@cloudflare.com> Signed-off-by: Michael Hoffmann <mhoffm@posteo.de> Co-authored-by: Björn Rabenstein <github@rabenste.in> * Add primitive support for ingesting OTLP delta metrics as-is (#16360) * Add simple delta support Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Rename delta2cumulative part Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Whoops bad refactor Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Add example yml Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Feature flag instead and histogram hint handling Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Delete otel_delta.yml - outdated Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Renaming to native delta support Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Add more explanatory comments Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Add more explanation to histograms Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Correct comment on d2c consumer Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Add tests for counters and fix bug Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Add histogram tests Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Add docs Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Sort series to make test deterministic Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * More formatting Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Change flag name to ingestion Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Explain where rate calculation can go wrong Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Add warning about duplicate timestamps Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Update docs/feature_flags.md Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com> Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Fix tests Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Remove unnecessary if Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Add warning to d2c section Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Make unknown type error when getting temporality Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Correct type comment - not planning to add delta metric metadata type Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Remove unused param for empty type Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Rewrite temporality logic to be clearer Signed-off-by: Fiona Liao <fiona.liao@grafana.com> * Change spurious to unnecessary - better description Signed-off-by: Fiona Liao <fiona.liao@grafana.com> --------- Signed-off-by: Fiona Liao <fiona.liao@grafana.com> Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com> * Add documentation for 'NoTranslation' mode (#16470) * Add documentation for 'NoTranslation' mode Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> * Update docs/configuration/configuration.md Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com> Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> --------- Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com> * fix conversion from time range in seconds to time duration at parser Signed-off-by: Zhang Zhanpeng <zhangzhanpeng.zzp@alibaba-inc.com> * promql: histogram_stddev and histogram_stdvar should use arithmetic mean for custom buckets Signed-off-by: amanycodes <amanycodes@gmail.com> * stable and definitely unique card UI key in /alerts Signed-off-by: amanycodes <amanycodes@gmail.com> * fix-ci-errors Signed-off-by: amanycodes <amanycodes@gmail.com> * stable and unique key for /rules page and key fixes Signed-off-by: amanycodes <amanycodes@gmail.com> * removed comments Signed-off-by: amanycodes <amanycodes@gmail.com> * Fix PRW2 data diff error message typo (#16481) Signed-off-by: SungJin1212 <tjdwls1201@gmail.com> * fix typo (#16480) Signed-off-by: Yuchen Wang <yuchen.wang@databricks.com> * Strip newline in error when reading from remote client (#16487) Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com> * chore(deps): bump golang.org/x/net Bumps [golang.org/x/net](https://github.com/golang/net) from 0.37.0 to 0.38.0. - [Commits](https://github.com/golang/net/compare/v0.37.0...v0.38.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-version: 0.38.0 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump github.com/aws/aws-sdk-go from 1.55.6 to 1.55.7 Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.55.6 to 1.55.7. - [Release notes](https://github.com/aws/aws-sdk-go/releases) - [Changelog](https://github.com/aws/aws-sdk-go/blob/main/CHANGELOG_PENDING.md) - [Commits](https://github.com/aws/aws-sdk-go/compare/v1.55.6...v1.55.7) --- updated-dependencies: - dependency-name: github.com/aws/aws-sdk-go dependency-version: 1.55.7 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump github.com/docker/docker Bumps [github.com/docker/docker](https://github.com/docker/docker) from 28.0.4+incompatible to 28.1.1+incompatible. - [Release notes](https://github.com/docker/docker/releases) - [Commits](https://github.com/docker/docker/compare/v28.0.4...v28.1.1) --- updated-dependencies: - dependency-name: github.com/docker/docker dependency-version: 28.1.1+incompatible dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump github.com/Azure/azure-sdk-for-go/sdk/azidentity Bumps [github.com/Azure/azure-sdk-for-go/sdk/azidentity](https://github.com/Azure/azure-sdk-for-go) from 1.8.2 to 1.9.0. - [Release notes](https://github.com/Azure/azure-sdk-for-go/releases) - [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/release.md) - [Commits](https://github.com/Azure/azure-sdk-for-go/compare/sdk/azidentity/v1.8.2...sdk/azcore/v1.9.0) --- updated-dependencies: - dependency-name: github.com/Azure/azure-sdk-for-go/sdk/azidentity dependency-version: 1.9.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump github.com/prometheus/client_model from 0.6.1 to 0.6.2 Bumps [github.com/prometheus/client_model](https://github.com/prometheus/client_model) from 0.6.1 to 0.6.2. - [Release notes](https://github.com/prometheus/client_model/releases) - [Commits](https://github.com/prometheus/client_model/compare/v0.6.1...v0.6.2) --- updated-dependencies: - dependency-name: github.com/prometheus/client_model dependency-version: 0.6.2 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump google.golang.org/api from 0.229.0 to 0.230.0 Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.229.0 to 0.230.0. - [Release notes](https://github.com/googleapis/google-api-go-client/releases) - [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md) - [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.229.0...v0.230.0) --- updated-dependencies: - dependency-name: google.golang.org/api dependency-version: 0.230.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump github.com/gophercloud/gophercloud/v2 Bumps [github.com/gophercloud/gophercloud/v2](https://github.com/gophercloud/gophercloud) from 2.6.0 to 2.7.0. - [Release notes](https://github.com/gophercloud/gophercloud/releases) - [Changelog](https://github.com/gophercloud/gophercloud/blob/v2.7.0/CHANGELOG.md) - [Commits](https://github.com/gophercloud/gophercloud/compare/v2.6.0...v2.7.0) --- updated-dependencies: - dependency-name: github.com/gophercloud/gophercloud/v2 dependency-version: 2.7.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump google.golang.org/grpc from 1.71.1 to 1.72.0 Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.71.1 to 1.72.0. - [Release notes](https://github.com/grpc/grpc-go/releases) - [Commits](https://github.com/grpc/grpc-go/compare/v1.71.1...v1.72.0) --- updated-dependencies: - dependency-name: google.golang.org/grpc dependency-version: 1.72.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump github.com/linode/linodego from 1.48.1 to 1.49.0 Bumps [github.com/linode/linodego](https://github.com/linode/linodego) from 1.48.1 to 1.49.0. - [Release notes](https://github.com/linode/linodego/releases) - [Commits](https://github.com/linode/linodego/compare/v1.48.1...v1.49.0) --- updated-dependencies: - dependency-name: github.com/linode/linodego dependency-version: 1.49.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump github.com/miekg/dns from 1.1.64 to 1.1.65 Bumps [github.com/miekg/dns](https://github.com/miekg/dns) from 1.1.64 to 1.1.65. - [Changelog](https://github.com/miekg/dns/blob/master/Makefile.release) - [Commits](https://github.com/miekg/dns/compare/v1.1.64...v1.1.65) --- updated-dependencies: - dependency-name: github.com/miekg/dns dependency-version: 1.1.65 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump github.com/prometheus/client_golang Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.21.1 to 1.22.0. - [Release notes](https://github.com/prometheus/client_golang/releases) - [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md) - [Commits](https://github.com/prometheus/client_golang/compare/v1.21.1...v1.22.0) --- updated-dependencies: - dependency-name: github.com/prometheus/client_golang dependency-version: 1.22.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump github.com/digitalocean/godo from 1.142.0 to 1.144.0 Bumps [github.com/digitalocean/godo](https://github.com/digitalocean/godo) from 1.142.0 to 1.144.0. - [Release notes](https://github.com/digitalocean/godo/releases) - [Changelog](https://github.com/digitalocean/godo/blob/main/CHANGELOG.md) - [Commits](https://github.com/digitalocean/godo/compare/v1.142.0...v1.144.0) --- updated-dependencies: - dependency-name: github.com/digitalocean/godo dependency-version: 1.144.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump github.com/prometheus/client_golang Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.21.1 to 1.22.0. - [Release notes](https://github.com/prometheus/client_golang/releases) - [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md) - [Commits](https://github.com/prometheus/client_golang/compare/v1.21.1...v1.22.0) --- updated-dependencies: - dependency-name: github.com/prometheus/client_golang dependency-version: 1.22.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * [BUILD] Remove Go toolchain line in internal/tools (#16516) * Upgraded npm dependencies pre 3.4.0-rc.0 (#16493) * Volunteer Bryan Boreham as Shepherd for Release 3.5 Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * Add `label_selector` to hetzner service discovery Allows to filter the servers when sending the listing request to the API. This feature is only available when using the `role=hcloud`. See https://docs.hetzner.cloud/#label-selector for details on how to use the label selector. Signed-off-by: Jonas Lammler <jonas.lammler@hetzner-cloud.de> * Clarify the docs of promtool --{min,max}-time options State explicitely what kind of timestamps are expected for the --min-time and --max-time options of promtool tsdb commands. This is especially important for the dump-openmetrics command as users could otherwise mistakenly think it would be in seconds, like the OpenMetrics timestamps themselves. Signed-off-by: Nicolas Peugnet <nicolas.peugnet@lip6.fr> * Labels: simpler/faster stringlabels encoding (#16069) Instead of using varint to encode the size of each label, use a single byte for size 0-254, or a flag value of 255 followed by the size in 3 bytes little-endian. This reduces the amount of code, and also the number of branches in commonly-executed code, so it runs faster. The maximum allowed label name or value length is now 2^24 or 16MB. Memory used by labels changes as follows: * Labels from 0 to 127 bytes length: same * From 128 to 254: 1 byte less * From 255 to 16383: 2 bytes more * From 16384 to 2MB: 1 byte more * From 2MB to 16MB: same Labels: panic on string too long. Slightly more user-friendly than encoding bad data and finding out when we decode. Clarify that Labels.Bytes() encoding can change --------- Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * [BUGFIX] Top-level: Update GOGC before loading TSDB (#16491) We should use the configured value, or Prometheus' default of 75%, while initializing and loading the WAL. Since the Go default is 100%, most Prometheus users would experience higher memory usage before the value is configured. Also: move Go runtime params earlier in initialization. E.g. if a module starting up looks at GOMAXPROCS to size something, we need to have set it already. --------- Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * promtool: Optional fuzzy float64 comparison in rules unittests (#16395) Make fuzzy compare opt-in via fuzzy_compare boolean in each unittest file. Signed-off-by: Graham Reed <greed@hypervolt.co.uk> * chore(deps): bump github/codeql-action from 3.28.13 to 3.28.16 Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.28.13 to 3.28.16. - [Release notes](https://github.com/github/codeql-action/releases) - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) - [Commits](https://github.com/github/codeql-action/compare/1b549b9259bda1cb5ddde3b41741a82a2d15a841...28deaeda66b76a05916b6923827895f2b14ab387) --- updated-dependencies: - dependency-name: github/codeql-action dependency-version: 3.28.16 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump github.com/ionos-cloud/sdk-go/v6 from 6.3.3 to 6.3.4 Bumps [github.com/ionos-cloud/sdk-go/v6](https://github.com/ionos-cloud/sdk-go) from 6.3.3 to 6.3.4. - [Release notes](https://github.com/ionos-cloud/sdk-go/releases) - [Changelog](https://github.com/ionos-cloud/sdk-go/blob/master/docs/CHANGELOG.md) - [Commits](https://github.com/ionos-cloud/sdk-go/compare/v6.3.3...v6.3.4) --- updated-dependencies: - dependency-name: github.com/ionos-cloud/sdk-go/v6 dependency-version: 6.3.4 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump the go-opentelemetry-io group across 2 directories with 5 updates Bumps the go-opentelemetry-io group with 4 updates in the / directory: [go.opentelemetry.io/collector/component](https://github.com/open-telemetry/opentelemetry-collector), [go.opentelemetry.io/collector/consumer](https://github.com/open-telemetry/opentelemetry-collector), [go.opentelemetry.io/collector/processor](https://github.com/open-telemetry/opentelemetry-collector) and [go.opentelemetry.io/collector/semconv](https://github.com/open-telemetry/opentelemetry-collector). Bumps the go-opentelemetry-io group with 2 updates in the /documentation/examples/remote_storage directory: [go.opentelemetry.io/collector/pdata](https://github.com/open-telemetry/opentelemetry-collector) and [go.opentelemetry.io/collector/semconv](https://github.com/open-telemetry/opentelemetry-collector). Updates `go.opentelemetry.io/collector/component` from 1.30.0 to 1.31.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-collector/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-collector/blob/main/CHANGELOG-API.md) - [Commits](https://github.com/open-telemetry/opentelemetry-collector/compare/pdata/v1.30.0...pdata/v1.31.0) Updates `go.opentelemetry.io/collector/consumer` from 1.30.0 to 1.31.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-collector/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-collector/blob/main/CHANGELOG-API.md) - [Commits](https://github.com/open-telemetry/opentelemetry-collector/compare/pdata/v1.30.0...pdata/v1.31.0) Updates `go.opentelemetry.io/collector/pdata` from 1.30.0 to 1.31.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-collector/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-collector/blob/main/CHANGELOG-API.md) - [Commits](https://github.com/open-telemetry/opentelemetry-collector/compare/pdata/v1.30.0...pdata/v1.31.0) Updates `go.opentelemetry.io/collector/processor` from 1.30.0 to 1.31.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-collector/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-collector/blob/main/CHANGELOG-API.md) - [Commits](https://github.com/open-telemetry/opentelemetry-collector/compare/pdata/v1.30.0...pdata/v1.31.0) Updates `go.opentelemetry.io/collector/semconv` from 0.124.0 to 0.125.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-collector/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-collector/blob/main/CHANGELOG-API.md) - [Commits](https://github.com/open-telemetry/opentelemetry-collector/compare/v0.124.0...v0.125.0) Updates `go.opentelemetry.io/collector/pdata` from 1.30.0 to 1.31.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-collector/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-collector/blob/main/CHANGELOG-API.md) - [Commits](https://github.com/open-telemetry/opentelemetry-collector/compare/pdata/v1.30.0...pdata/v1.31.0) Updates `go.opentelemetry.io/collector/semconv` from 0.124.0 to 0.125.0 - [Release notes](https://github.com/open-telemetry/opentelemetry-collector/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-collector/blob/main/CHANGELOG-API.md) - [Commits](https://github.com/open-telemetry/opentelemetry-collector/compare/v0.124.0...v0.125.0) --- updated-dependencies: - dependency-name: go.opentelemetry.io/collector/component dependency-version: 1.31.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: go-opentelemetry-io - dependency-name: go.opentelemetry.io/collector/consumer dependency-version: 1.31.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: go-opentelemetry-io - dependency-name: go.opentelemetry.io/collector/pdata dependency-version: 1.31.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: go-opentelemetry-io - dependency-name: go.opentelemetry.io/collector/processor dependency-version: 1.31.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: go-opentelemetry-io - dependency-name: go.opentelemetry.io/collector/semconv dependency-version: 0.125.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: go-opentelemetry-io - dependency-name: go.opentelemetry.io/collector/pdata dependency-version: 1.31.0 dependency-type: indirect update-type: version-update:semver-minor dependency-group: go-opentelemetry-io - dependency-name: go.opentelemetry.io/collector/semconv dependency-version: 0.125.0 dependency-type: indirect update-type: version-update:semver-minor dependency-group: go-opentelemetry-io ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump github.com/digitalocean/godo from 1.144.0 to 1.145.0 Bumps [github.com/digitalocean/godo](https://github.com/digitalocean/godo) from 1.144.0 to 1.145.0. - [Release notes](https://github.com/digitalocean/godo/releases) - [Changelog](https://github.com/digitalocean/godo/blob/main/CHANGELOG.md) - [Commits](https://github.com/digitalocean/godo/compare/v1.144.0...v1.145.0) --- updated-dependencies: - dependency-name: github.com/digitalocean/godo dependency-version: 1.145.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump google.golang.org/api from 0.230.0 to 0.231.0 Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.230.0 to 0.231.0. - [Release notes](https://github.com/googleapis/google-api-go-client/releases) - [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md) - [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.230.0...v0.231.0) --- updated-dependencies: - dependency-name: google.golang.org/api dependency-version: 0.231.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * chore(cmd): Add info log for automemlimit config Signed-off-by: machine424 <ayoubmrini424@gmail.com> * Prepare release 3.4.0-rc.0 (#16517) Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com> * Fix linting issues found by golangci-lint v2.0.2 (#16368) * Fix linting issues found by golangci-lint v2.0.2 --------- Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * chore(deps): bump github.com/open-telemetry/opentelemetry-collector-contrib/processor/deltatocumulativeprocessor Bumps [github.com/open-telemetry/opentelemetry-collector-contrib/processor/deltatocumulativeprocessor](https://github.com/open-telemetry/opentelemetry-collector-contrib) from 0.124.1 to 0.125.0. - [Release notes](https://github.com/open-telemetry/opentelemetry-collector-contrib/releases) - [Changelog](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/CHANGELOG-API.md) - [Commits](https://github.com/open-telemetry/opentelemetry-collector-contrib/compare/v0.124.1...v0.125.0) --- updated-dependencies: - dependency-name: github.com/open-telemetry/opentelemetry-collector-contrib/processor/deltatocumulativeprocessor dependency-version: 0.125.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Bump golangci-lint to v2.1.5 (#16545) Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com> * chore(deps): bump actions/setup-node from 4.3.0 to 4.4.0 (#16533) Bumps [actions/setup-node](https://github.com/actions/setup-node) from 4.3.0 to 4.4.0. - [Release notes](https://github.com/actions/setup-node/releases) - [Commits](https://github.com/actions/setup-node/compare/cdca7365b2dadb8aad0a33bc7601856ffabcc48e...49933ea5288caeca8642d1e84afbd3f7d6820020) --- updated-dependencies: - dependency-name: actions/setup-node dependency-version: 4.4.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * promql: don't emit a value from `histogram_fraction` or `histogram_quantile` if classic and native histograms are present at the same timestamp Signed-off-by: Charles Korn <charles.korn@grafana.com> * Add global config option for always_scrape_classic_histograms Addresses https://github.com/prometheus/prometheus/issues/16371 This will help with migrating to native histograms with `convert_classic_histograms_to_nhcb` since users may still need to keep the classic histograms during a migration Signed-off-by: chardch <otwordsne@gmail.com> * fix promql-test error message and readme Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com> * chore(scripts/sync_repo_files): add link to source script Signed-off-by: machine424 <ayoubmrini424@gmail.com> * tsdb/errors.MultiError: support errors.As (#16544) * tsdb/errors.MultiError: implement Unwrap the multierror was hiding some errors in Mimir. I also added unit tests because I had them handy from a similar change I and yuri did in XXX and some time ago --------- Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com> Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> * test(cmd): add test for GOGC setting As suggested in https://github.com/prometheus/prometheus/pull/14176#issuecomment-2150308054 Signed-off-by: machine424 <ayoubmrini424@gmail.com> * [PERF] TSDB: Pass down label value limit into implementation (#16158) * allow limiting label values calls Signed-off-by: Andre Branchizio <andrejbranch@gmail.com> * update fix configuration.md Signed-off-by: dongjiang <dongjiang1989@126.com> * Add a test for aggregation wrapped in ParenExpr Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com> * fix(parser): parenthesis around aggregate expression parsing The position range of nested aggregate expression was wrong, for the expression "(sum(foo))" the position of "sum(foo)" should be 1-9, but the parser could not decide the end of the expression on pos 9, instead it read ahead to pos 10 and then emitted the aggregate. But we only kept the last closing position (10) and wrote that into the aggregate. The reason for this is that the parser cannot know from "(sum(foo)" alone if the aggregate is finished. It could be finished as in "(sum(foo))" but equally it could continue with group modifier as "(sum(foo) by (bar))". The fix is to track ")" tokens in a stack in addition to the lastClosing position, which is overloaded with other things like offset number tracking. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> * Fix position in TestPreprocessAndWrapWithStepInvariantExpr This is failing after 26088d01b9518b3315805186dcee534b986bf79f and it seems that currently tested position is incorrect, fix it Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com> * docs: Fix `metric_name_escaping_scheme` config parameter This is a fix for documentation update done in https://github.com/prometheus/prometheus/pull/16066, setting correct name for a configuration value. Signed-off-by: Martin Danko <46035688+zepellin@users.noreply.github.com> * chore(engine): add simple NHCB benchmark Copy the benchmark for native histograms with exponential buckets and adopt to native histograms with custom buckets. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> * perf(chunkenc): intern the custom values for native histograms The custom values are the "le" bucket boundaries of native histograms with custom buckets. They are never modified. It is ok to not copy them when iterating a chunk, just reference them. If we will ever have a function that modifies the custom values, like 'trim' for example. That function will have to make a copy on write. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> * UI: optimize pool searching in /target page Signed-off-by: Ryan Wu <rongjun0821@gmail.com> * refactor: use slices.Contains to simplify code Signed-off-by: hardlydearly <799511800@qq.com> * promql: Refactor some functions to make them more DRY (#16532) Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com> * promql: support variable scalar parameter in aggregations in range queries (#16404) This fixes the regression introduced in https://github.com/prometheus/prometheus/issues/15971 while preserving the performance improvements. Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com> * Try fixing potential deadlocks in discovery Manager.ApplyConfig() uses multiple locks: - Provider.mu - Manager.targetsMtx Manager.cleaner() uses the same locks but in the opposite order: - First it locks Manager.targetsMtx - The it locks Provider.mu I've seen a few strange cases of Prometheus hanging up on shutdown and never compliting that shutdown. From a few traces I was given it appears that while Prometheus is still running only discovery.Manager and notifier.Manager are running running. From that trace it also seems like they are stuck on a lock from two functions: - cleaner waits on a RLock() - ApplyConfig waits on a Lock() I cannot reproduce it but I suspect this is a race between locks. Imagine this scenario: - Manager.ApplyConfig() is called - Manager.ApplyConfig locks Provider.mu.Lock() - at the same time cleaner() is called on the same Provider instance and it calls Manager.targetsMtx.Lock() - Manager.ApplyConfig() now calls Manager.targetsMtx.Lock() but that lock is already held by cleaner() function so ApplyConfig() hangs there - at the same time cleaner() now wants to lock Provider.mu.Rlock() but that lock is already held by Manager.ApplyConfig() - we end up with both functions locking each other out without any way to break that lock Re-order lock calls to try to avoid this scenario. I tried writing a test case for it but couldn't hit this issue. Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com> * Optimize memoization and search debouncing on /targets page (#16589) Moving the debouncing of the search field to the parent component and then memoizing the ScrapePoolsList component prevents a lot of superfluous re-renders of the entire scrape pools list that previously got triggered immediately when you typed in the search box or even just collapsed a pool. (While the computation of what data to show was already memoized in the ScrapePoolList component, the component itself still had to re-render a lot with the same data.) Discovered this problem + verified fix using react-scan. Signed-off-by: Julius Volz <julius.volz@gmail.com> * Fix full-page re-rendering when opening status nav menu (#16590) When opening the status pages menu while already viewing one of the status pages, the whole page would be re-rendered because the menu target's default action of following the current page's URL was not prevented. Also, we don't need to use a NavLink component for the menu target when we are not viewing a status page, because then the component won't need to be highlighted anyways. Discovered + fixed with the help of react-scan. Signed-off-by: Julius Volz <julius.volz@gmail.com> * fix(config): respect GOGC environment variable if no "runtime" block exists Fixes: https://github.com/prometheus/prometheus/issues/16334 Related to: - https://github.com/prometheus/prometheus/pull/15238 - https://github.com/prometheus/prometheus/pull/16052 Currently, when the GOGC environment variable is set -- and no `runtime` block is set in the Prometheus config file -- it is ignored and the default value of 75% is always used. However, if there is an empty runtime block (e.g. `runtime: {}`), _then_ the GOGC environment variable is checked. This PR changes this behavior to consistently check and use the GOGC environment variable when it is set (unless the `gogc` field is set in the `runtime` block of the loaded config file, in which case it still gives that precedence). Co-authored-by: Adam Rambo <arambo@protonmail.com> Signed-off-by: Will Hegedus <whegedus@akamai.com> * test(cmd): enable test cases for GOGC environment variable Signed-off-by: Will Hegedus <whegedus@akamai.com> * Clean up codeboxes and headings in docs The new docs site will have syntax highlighting, so this adds language tags to code boxes that are currently missing them. I didn't add `promql` as a language yet since the highlighter doesn't support it yet, plus a lot of the PromQL codeboxes in our docs aren't strictly valid PromQL, they are more like multiple expressions listed in the same code box on multiple lines. So I'm leaving that for sometime later. In the HTTP API page, I moved the curl examples from the JSON codeboxes to their own ones above the JSON output. I considered putting an "Output:" text between the curl + JSON output, but I think the way it currently looks without it is probably fine. I also fixed a number of headings which were at the wrong level relative to their nesting in the document. I also removed `go` as a language from the Go template language examples, because the Go template language isn't Go at all. I also adjusted the indent on one codebox to be more reasonable (2 spaces instead of 8). And then finally, my editor made a bunch of whitespace changes automatically, like removing trailing spaces. Signed-off-by: Julius Volz <julius.volz@gmail.com> Signed-off-by: Julius Volz <julius.volz@gmail.com> * Move m.targetsMtx.Lock down into the loop Make sure the order of locks is always the same in all functions. In ApplyConfig() we have m.targetsMtx.Lock() after provider is locked, so replicate the same in allGroups(). Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com> * Fixup make proto (#16603) Use `common-` prefix for `make proto` so downstream projects like client_golang can implement their own `make proto`. Signed-off-by: SuperQ <superq@gmail.com> * Add golangci-lint fmt (#16602) With golangci-lint v2, it now has "formatters" that can be configured. Add `golangci-lint fmt` to the `make format` in Makefile.common. * Enable goimports formatter. Signed-off-by: SuperQ <superq@gmail.com> * SD UI: Better total target count display when using `keep_dropped_targets` option Fixes https://github.com/prometheus/prometheus/issues/16586 Signed-off-by: Julius Volz <julius.volz@gmail.com> * Add health & text filtering on the /rules page Addresses part of https://github.com/prometheus/prometheus/issues/16515 For now, I'm adding very similar filtering to the /rules page as we have on the /alerts page, with the difference being: * The state filter filters by rule health (ok/warn/unknown) instead of alert state (firing/pending/inactive) * We don't collect & show detailed stats on the different state counts as we do on the /alerts page There is a lot of copied / very similar code between those two pages (and also some others) around filtering and pagination, so maybe there is an opportunity for more code sharing in the future here. Signed-off-by: Julius Volz <julius.volz@gmail.com> * docs: fix typo in operators.md Signed-off-by: marcoderama <marcoderamagit@gmail.com> * Release 3.4.0 (#16600) * Adjust release date in CHANGELOG.md Signed-off-by: Jan-Otto Kröpke <mail@jkroepke.de> * Merge release-3.4 into main (#16609) Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com> * Fix flakiness in TestOTLPWriteHandler (#16608) * feature: type-and-unit-labels (PROM-39 implementation) (#16228) * feature: type-and-unit-labels (extended MetricIdentity) Experimental implementation of https://github.com/prometheus/proposals/pull/39 Previous (unmerged) experiments: * https://github.com/prometheus/prometheus/compare/main...dashpole:prometheus:type_and_unit_labels * https://github.com/prometheus/prometheus/pull/16025 Signed-off-by: bwplotka <bwplotka@gmail.com> feature: type-and-unit-labels (extended MetricIdentity) Experimental implementation of https://github.com/prometheus/proposals/pull/39 Previous (unmerged) experiments: * https://github.com/prometheus/prometheus/compare/main...dashpole:prometheus:type_and_unit_labels * https://github.com/prometheus/prometheus/pull/16025 Signed-off-by: bwplotka <bwplotka@gmail.com> * Fix compilation errors Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> Lint Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> Revert change made to protobuf 'Accept' header Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> Fix compilation errors for 'dedupelabels' tag Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> * Rectored into schema.Metadata Signed-off-by: bwplotka <bwplotka@gmail.com> * texparse: Added tests for PromParse Signed-off-by: bwplotka <bwplotka@gmail.com> * add OM tests. Signed-off-by: bwplotka <bwplotka@gmail.com> * add proto tests Signed-off-by: bwplotka <bwplotka@gmail.com> * Addressed comments. Signed-off-by: bwplotka <bwplotka@gmail.com> * add schema label tests. Signed-off-by: bwplotka <bwplotka@gmail.com> * addressed comments. Signed-off-by: bwplotka <bwplotka@gmail.com> * fix tests. Signed-off-by: bwplotka <bwplotka@gmail.com> * add promql tests. Signed-off-by: bwplotka <bwplotka@gmail.com> * lint Signed-off-by: bwplotka <bwplotka@gmail.com> * Addressed comments. Signed-off-by: bwplotka <bwplotka@gmail.com> --------- Signed-off-by: bwplotka <bwplotka@gmail.com> Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com> * util/httputil: Always add Vary header in SetCORS Closes #15406 Signed-off-by: jub0bs <jcretel-infosec+github@protonmail.com> * refactor: use the built-in max/min to simplify the code Signed-off-by: carrychair <linghuchong404@gmail.com> * refactor(endpointslice): use service cache.Indexer to achieve better iteration performance (#16365) * refactor(endpointslice): use cache.Indexer to index endpointslices by LabelServiceName so not have to iterate over all endpoint objects. Signed-off-by: Ryan Wu <rongjun0821@gmail.com> * check the type and error early and add 'TestEndpointSliceDiscoveryWithUnrelatedServiceUpdate' unit test to give a regression test Signed-off-by: Ryan Wu <rongjun0821@gmail.com> * make service indexer namespaced Signed-off-by: Ryan Wu <rongjun0821@gmail.com> * remove unneeded test func Signed-off-by: Ryan Wu <rongjun0821@gmail.com> * Apply suggestions from code review Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com> Signed-off-by: Ryan Wu <rongjun0821@gmail.com> --------- Signed-off-by: Ryan Wu <rongjun0821@gmail.com> Co-authored-by: Ayoub Mrini <ayoubmrini424@gmail.com> * feat(tsdb): introduce --use-uncached-io feature flag and allow using it for chunks writing (#15365) Signed-off-by: machine424 <ayoubmrini424@gmail.com> Signed-off-by: Ayoub Mrini <ayoubmrini424@gmail.com> * test(cmd/main/TestRuntimeGOGCConfig): add checks on reloads as well Signed-off-by: machine424 <ayoubmrini424@gmail.com> * chore(config): add guidelines for adding a new RuntimeConfig field based on learnings from GoGC addition Signed-off-by: machine424 <ayoubmrini424@gmail.com> * fix: add reproducer for a dangling-reference issue in parsers Signed-off-by: machine424 <ayoubmrini424@gmail.com> * fix(model/textparse): Labels(): copy the input to avoid dangling references Signed-off-by: machine424 <ayoubmrini424@gmail.com> * fix metric_name_escaping_scheme option (#16625) * remote_write azure auth : allow empty client_id to suppport system assigned managed identity (#16421) * squash (#1) * remote-write: allow empty azure client_id to support system assigned managed identity * add blank line for tests * remote-write: allow empty azure client_id to support system assigned managed identity Signed-off-by: Kaveesh Dubey <kadubey@microsoft.com> * add blank line for tests Signed-off-by: Kaveesh Dubey <kadubey@microsoft.com> --------- Signed-off-by: Kaveesh Dubey <kadubey@microsoft.com> * treat empty client_id as system-assigned identity; this is a valid case Signed-off-by: Kaveesh Dubey <kadubey@microsoft.com> * rename file Signed-off-by: bragi92 <kadubey@microsoft.com> --------- Signed-off-by: Kaveesh Dubey <kadubey@microsoft.com> Signed-off-by: bragi92 <kadubey@microsoft.com> * ci: address zizmor gh action vulnerabilities (#16530) * ci: address zizmor gh action vulnerabilities --------- Signed-off-by: Joe Harvey <51208233+jharvey10@users.noreply.github.com> Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Add support for promoting all OTel resource attributes (#16426) Add support for promoting all OTel resource attributes via `promote_all_resource_attributes`, except for those ignored using 'ignore_resource_attributes'. --------- Signed-off-by: Antonio Jimenez <antonjim@thousandEyes.com> Signed-off-by: Antonio Jimenez <123171955+antonjim-te@users.noreply.github.com> * Add parse alerting for rules files (#16601) Builds over https://github.com/prometheus/prometheus/pull/16462 Addresses comments, adds invalid rules file Signed-off-by: subhramit <subhramit.bb@live.in> Co-authored-by: marcodebba <marcodebonis74@gmail.com> * chore: Adjust test and add comment about DNS resolution issue for failing tests (#16200) * chore: Add comment about DNS resolution issue for failing tests Signed-off-by: zhengkezhou1 <madzhou1@gmail.com> * remove unexported-return Signed-off-by: zhengkezhou1 <madzhou1@gmail.com> --------- Signed-off-by: zhengkezhou1 <madzhou1@gmail.com> * [BACKPORT-3.4] fix: add reproducer for a dangling-reference issue in parsers and fix (#16633) * chore: fix some comments Signed-off-by: tongjicoder <tongjicoder@icloud.com> * refactor: Move 'for' state restoration metrics to defer block Signed-off-by: Anand Rajagopal <anrajag@amazon.com> * tsdb: fix issue where a new segment file is created for every chunk if `WithSegmentSize` not called (#16635) * tsdb: fix issue where a new segment file is created for every chunk Signed-off-by: Charles Korn <charles.korn@grafana.com> * Address PR feedback Signed-off-by: Charles Korn <charles.korn@grafana.com> --------- Signed-off-by: Charles Korn <charles.korn@grafana.com> * Standardize doc page title handling See https://groups.google.com/g/prometheus-developers/c/cwL3cM66Em8 Signed-off-by: Julius Volz <julius.volz@gmail.com> * fix(test): fall back to default direct I/O requirements in tests when statx isn't supported by using a higher lever util Signed-off-by: machine424 <ayoubmrini424@gmail.com> * Update otlptranslator with new API (#16626) * Update otlptranslator with new API Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> --------- Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> * Release 3.4.1 (#16652) * test(promql): histogram_count inconsistent Ref: https://github.com/prometheus/prometheus/issues/16681 Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> * fix(promql): histogram_count inconsistent The problem is in the counter reset detection. The code that loads the samples is matrixIterSlice which uses the typed Buffer iterator, which will preload the integer histogram samples, however the last sample is always(!) loaded as a float histogram sample in matrixIterSlice and the optimized iterator fails to detect counter resets in that case. Also the iterator does not reset lastH, lastFH properly. Ref: https://github.com/prometheus/prometheus/issues/16681 Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> * Make translatorMetricFromOtelMetric public (#16683) Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com> * fix(promql): do not loose information about buckets when doing the detect Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> * bugfix: Default OTLP config not respected when `otlp:` block is unset (#16693) Co-authored-by: Jan-Otto Kröpke <mail@jkroepke.de> * promqltest: Adjust more tests to "left-open" ranges This was an oversight because the old tests still happened to pass with the new behavior, but important test data was excluded at the left end of the interval, rendering some tests not actually testing what we want to test. In the past, we also applied different strategies to adjust the test (extend the range from 1m to 2m, or set the evaluation timestamp to 45s). This commit unifies things and reduces redundancy. Signed-off-by: beorn7 <beorn@grafana.com> * promql: Simplify avg aggregation and avg_over_time As it turns out, if we combine Kahan summation and incremental mean calculation properly, it works quite well and we do not need to switch between simple mean calculation and incremental calculation based on overflow. This simplifies the code quite a bit. Signed-off-by: beorn7 <beorn@grafana.com> * Export QuerySamples StartTimestamp and Interval to enable custom statistics implementations to populate these fields as needed. Signed-off-by: Innokentii Konstantinov <innokenty.konstantinov@grafana.com> * chore(docs/kubernetes SD): add a note about Endpoints API being deprecated in kubernetes 1.33+ (#16684) * chore(docs/kubernetes SD): add a note about Endpoints API being deprecated in kubernetes 1.33+ Signed-off-by: machine424 <ayoubmrini424@gmail.com> * chore(discovery/kubernetes): add Endpoints API deprecation comment Signed-off-by: machine424 <ayoubmrini424@gmail.com> --------- Signed-off-by: machine424 <ayoubmrini424@gmail.com> * PROM-39: Provide PromQL info annotations when rate()/increase() over series without `__type__`="counter" label (#16632) * Provide PromQL info annotations when rate()/increase() over series without counter label Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> * Address comments Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> --------- Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> * [Refactor] PromQL: Simplify detectHistogramStatsDecoding Restarting the depth-first walk on each leg of a binary expression is convoluted. ISTM the correct logic is to walk the path backwards to the first relevant function. Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * Add more avg_over_time test cases with extreme values These tests were initially created by @crush-on-anechka. I modified them slightly. Signed-off-by: beorn7 <beorn@grafana.com> * Enhancement: Reload all scrape pools concurrently (#16595) * Reload all scrape pools concurrently At the moment all scrape pools that need to be reloaded are reloaded one by one. While reloads are ongoing mtxScrape is locked. For each pool that's being reloaded we need to wait until all targets are updated. This whole process can take a while and the more scrape pools to reload the longer. At the same time all pools are independent and there's no real reason to do them one-by-one. Reload each pool in a seperate goroutine so we finish config reload as ASAP as possible and unlock the mtxScrape. Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com> * Address PR review feedback Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com> --------- Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com> * promql: Deactivate three failing tests for the time being These tests fail on darwin/arm64. One is expected, because the test demonstrates the limits of the numerical accuracy of our methods, and different inaccurate outcomes on different hardware are expected. The other two are mysterious at the moment, see https://github.com/prometheus/prometheus/issues/16714 for detailed discussion and debugging. Signed-off-by: beorn7 <beorn@grafana.com> * promql: typo fix in comment Signed-off-by: beorn7 <beorn@grafana.com> * engine: fix PossibleNonCounterInfo annotation for rate and increase. (#16718) * Comment exposed field Signed-off-by: Innokentii Konstantinov <innokenty.konstantinov@grafana.com> * Remove invalid test Signed-off-by: Innokentii Konstantinov <innokenty.konstantinov@grafana.com> * Fix PossibleNonCounterInfo annotation. Signed-off-by: Innokentii Konstantinov <innokenty.konstantinov@grafana.com> * lint Signed-off-by: Innokentii Konstantinov <innokenty.konstantinov@grafana.com> --------- Signed-off-by: Innokentii Konstantinov <innokenty.konstantinov@grafana.com> * fix(nhcb): do not loose NHCB after exponential with different labels The problem happens when we parse a standalone native histogram, which sets the p.lastHistogramExponential state flag. We never unset it. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> * refactor(nhcb): get rid of extra lastHistogramExponential state and make it a regular state of inhibiting NHCB conversion. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> * Enhance TestDropOldTimeSeries to test both v1 and v2 protocols (#16709) - Wrapped existing test logic in a loop to run with both protocol versions - Ensures consistent behavior across protocol versions for dropping old time series Signed-off-by: AxcelXander <tyz666@bu.edu> Co-authored-by: AxcelXander <tyz666@bu.edu> * mantine-ui: fix /alerts accordian theme colors (#16043) * fix-alerts-accordion-theme Signed-off-by: amanycodes <amanycodes@gmail.com> * fixes Signed-off-by: amanycodes <amanycodes@gmail.com> * added changes to /rules page Signed-off-by: amanycodes <amanycodes@gmail.com> * more fixes Signed-off-by: amanycodes <amanycodes@gmail.com> --------- Signed-off-by: amanycodes <amanycodes@gmail.com> Signed-off-by: Aman <95525722+amanycodes@users.noreply.github.com> * Fetch and display full query stats in hover tooltip in table query tab (#16723) Fixes https://github.com/prometheus/prometheus/issues/5857 Signed-off-by: Julius Volz <julius.volz@gmail.com> * promql: add ts_of_(max,min,last)_over_time functions This commit adds the ts_of_(max,min,last)_over_time functions behind the experimental feature flag. Signed-off-by: Michael Hoffmann <mhoffmann@cloudflare.com> * promql: Fix unary operator precedence in duration expressions Split unary operator handling in duration expressions into two specific cases to fix precedence conflicts: - Handle unary operators with number literals directly - Handle unary operators with parenthesized expressions separately This prevents unary minus from incorrectly binding to subsequent operators in expressions like `foo offset -1^1`, ensuring it parses as `(foo offset -1) ^ 1` rather than `foo offset (-1^1)`. Fixes #16711 Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com> * Update duration expression docs Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com> * promql: fix ts_of_last_over_time for histogram vectors This PR fixes a bug in ts_of_last_over_time where the float samples where used when computing the last timestamp of the histogram samples. Signed-off-by: Michael Hoffmann <mhoffmann@cloudflare.com> * config: fix typo in comments (#16734) Signed-off-by: Owen Williams <owen.williams@grafana.com> * discovery: add STACKIT SD (#16401) * Fix unit for Scrape failures Signed-off-by: datamuc <m@rbfh.de> * PromQL: Fix native histogram last_over_time with offset Signed-off-by: 🌲 Harry 🌊 John 🏔 <johrry@amazon.com> * OTLP: Support including scope metadata as metric labels (#16730) Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * promql: reuse `histogramStatsIterator` where possible, and expose it for other implementations to use (#16686) * Expose type * Add `Reset` method --------- Signed-off-by: Charles Korn <charles.korn@grafana.com> * Add script for converting PromQL tests to new syntax format (#16562) Signed-off-by: Kapil Lamba <kapillamba4@gmail.com> Co-authored-by: Neeraj Gartia <80708727+NeerajGartia21@users.noreply.github.com> * mantine-ui: add query warnings display to graph view Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com> * fix: prevent invalid array access in aggregate expression This commit fixes the evaluation of invalid expressions like `sum(rate(`. Before that, it would trigger a panic in the PromQL engine because it tried to access an index which is out of range. The bug was probably introduced by 06d0b063ea. Signed-off-by: Simon Pasquier <spasquie@redhat.com> * mantine-ui: add query info warnings display to graph view Followup of https://github.com/prometheus/prometheus/pull/16753 Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com> * OTLP receiver: Rename `otlp.convert_scope_metadata` to `otlp.promote_scope_metadata` (#16760) * Rename otlp.convert_scope_metadata to otlp.promote_scope_metadata --------- Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: Julius Hinze <juliusmh@proton.me> * [BUILD] Ran make update-go-deps Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * [BUILD] Wind back otlptranslator update which has breaking change Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * [BUILD] Wind back dependencies which require Go 1.24 We need Prometheus to build with the last 2 versions of Go. Remove Go toolchain line - it is redundant with the `go`` line above, and could mislead readers. Also don't update `fsnotify` which is pinned to an older version. Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * [TESTS] Rules: remove brittle TestNewGroup It broke on different implementation of 'NewNopLogger'. Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * [BUILD] Docker SD: Fix up deprecated types Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * Add tests for relabeling of type and unit labels (#16743) Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com> * Add documentation for custom_values usage in NHCB (#16757) Signed-off-by: rishi-jat <rishijat098@gmail.com> Signed-off-by: Rishi Jat <rishijat098@gmail.com> Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com> * fix(promql): histogram_quantile and histogram_fraction NaN observed in native histogram (#16724) * fix(promql): histogram_quantile NaN observed in native histogram Fixes: #16578 See the issue for detailed explanation. When a histogram had only NaN observations and no normal observations, we returned 0 from the quantile, which is completely wrong. If there were normal observations but we went over them, we returned the upper bound of the existing buckets, however that contradicts expectations on histogram_fraction. Now we return NaN if the quantile is calculated to be over all normal observations, falling into NaNs (in a virtual +Inf bucket). We also return info level annotations if we see any NaN observations. The annotation calls out if we returned NaN or even if we took the virtual +Inf bucket into account. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> * fix(promql): histogram_fraction NaN observed in native histogram Fixes: #16580 According to the specification we should not take NaN observations into account when calculating the fraction. This commit fixes that and adds an info level annotation to let the user know about this. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> * scrape: set validation and escaping defaults in default config vars (#16751) Fixes https://github.com/prometheus/prometheus/issues/16750 Signed-off-by: Owen Williams <owen.williams@grafana.com> * OTLP receiver: Don't append _total suffix to non-monotonic OTel sums (#16776) * OTLP receiver: Don't append _total suffix to non-monotonic OTel sums Fix the OTLP receiver so the suffix _total isn't appended to metrics converted from non-monotonic OTel sum metrics, if otlp.translation_strategy is UnderscoreEscapingWithSuffixes or NoUTF8EscapingWithSuffixes. Also add translation tests. --------- Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> * Prepare release 3.4.2 Signed-off-by: Jan-Otto Kröpke <mail@jkroepke.de> * Limit concurrency of scrape pool reloads (#16783) To avoid possible overload. As per https://github.com/prometheus/prometheus/pull/16595#issuecomment-3005027067 this changes scrape pool manager to limit the number of scrape pools that can reload at the same time. Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com> * promql: Add test cases for direct mean calculation These demonstrate that direct mean calculation has some merits after all. Signed-off-by: beorn7 <beorn@grafana.com> * promql: Remove falsified comment from test The test in question actually worked fine even before #16569. The finding reported in the comment has turned out to be caused by something else. Signed-off-by: beorn7 <beorn@grafana.com> * promql: Re-introduce direct mean calculation for better accuracy This commit brings back direct mean calculation (for `avg` and `avg_over_time`) but isn't an outright revert of #16569. It keeps the improved incremental mean calculation and features generally a bit cleaner code than before. Also, this commit... - ...updates the lengthy comment explaining the whole situation and trade-offs. - ...divides the running sum and the Kahan compensation term separately (in direct mean calculation) to avoid the (unlikely) possibility that sum and Kahan compensation together ovorflow float64. - ...uncomments the tests that should now work again on darwin/arm64. - ...uncomments the test that should now reliably yield the (inaccurate) value 0 on all hardware platforms. Also, the test description has been updated accordingly. - ...adds avg_over_time tests for zero and one sample in the range. Signed-off-by: beorn7 <beorn@grafana.com> * Prepare release 3.5.0-rc.0 (#16778) Signed-off-by: Bryan Boreham <bjboreham@gmail.com> Co-authored-by: Julius Volz <julius.volz@gmail.com> * Revert: OTLP Support including scope metadata as metric labels (#16842) Reverts #16730 and #16760 This is being done because we've noticed a problem in the spec that could lead to name collisions if attributes name, version or schema_url are added to the scope. They would collide with the already reserved labels otel_scope_name, otel_scope_version and otel_scope_schema_url. Since this new configuration option never made it into a release, we can safely remove it from the 3.5 release. We'll sort this out for the 3.6 release Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> * Prepare release 3.5.0-rc.1 (#16845) This RC reverts the feature "OTLP: Support promoting OTel scope attributes". Add the line back into the CHANGELOG for 3.5.0-rc.0, since we are not changing that version. Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * Prepare release 3.5.0 Signed-off-by: Bryan Boreham <bjboreham@gmail.com> * [bot] assets: generate Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * [create-pull-request] automated change Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * Update base image version to rhel_9_1.24 * Update promu build command in Containerfile.operator * Update Dockerfile.prow * Modify build command to include static assets directory * Update Containerfile.operator * Update PREBUILT_ASSETS_STATIC_DIR in Dockerfile.prow * Update buf-lint.yml * Modify .gitignore for UI asset tracking Updated .gitignore to track specific UI assets and exclude others. * update assets Signed-off-by: Coleen Iona Quadros <coleen.quadros27@gmail.com> * Update Dockerfile.prow * Refactor Containerfile by removing unused copies --------- Signed-off-by: Jan-Otto Kröpke <mail@jkroepke.de> Signed-off-by: Neeraj Gartia <neerajgartia211002@gmail.com> Signed-off-by: Arthur Silva Sens <arthursens2005@gmail.com> Signed-off-by: Michael Hoffmann <mhoffmann@cloudflare.com> Signed-off-by: Michael Hoffmann <mhoffm@posteo.de> Signed-off-by: machine424 <ayoubmrini424@gmail.com> Signed-off-by: Fiona Liao <fiona.liao@grafana.com> Signed-off-by: Zhang Zhanpeng <zhangzhanpeng.zzp@alibaba-inc.com> Signed-off-by: amanycodes <amanycodes@gmail.com> Signed-off-by: SungJin1212 <tjdwls1201@gmail.com> Signed-off-by: Yuchen Wang <yuchen.wang@databricks.com> Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com> Signed-off-by: dependabot[bot] <support@github.com> Signed-off-by: Bryan Boreham <bjboreham@gmail.com> Signed-off-by: Jonas Lammler <jonas.lammler@hetzner-cloud.de> Signed-off-by: Nicolas Peugnet <nicolas.peugnet@lip6.fr> Signed-off-by: Graham Reed <greed@hypervolt.co.uk> Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> Signed-off-by: Matthieu MOREL <matthieu.morel35@gmail.com> Signed-off-by: Charles Korn <charles.korn@grafana.com> Signed-off-by: chardch <otwordsne@gmail.com> Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com> Signed-off-by: Andre Branchizio <andrejbranch@gmail.com> Signed-off-by: dongjiang <dongjiang1989@126.com> Signed-off-by: Lukasz Mierzwa <l.mierzwa@gmail.com> Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com> Signed-off-by: Martin Danko <46035688+zepellin@users.noreply.github.com> Signed-off-by: Ryan Wu <rongjun0821@gmail.com> Signed-off-by: hardlydearly <799511800@qq.com> Signed-off-by: Julius Volz <julius.volz@gmail.com> Signed-off-by: Will Hegedus <whegedus@akamai.com> Signed-off-by: SuperQ <superq@gmail.com> Signed-off-by: marcoderama <marcoderamagit@gmail.com> Signed-off-by: bwplotka <bwplotka@gmail.com> Signed-off-by: jub0bs <jcretel-infosec+github@protonmail.com> Signed-off-by: carrychair <linghuchong404@gmail.com> Signed-off-by: Ayoub Mrini <ayoubmrini424@gmail.com> Signed-off-by: Kaveesh Dubey <kadubey@microsoft.com> Signed-off-by: bragi92 <kadubey@microsoft.com> Signed-off-by: Joe Harvey <51208233+jharvey10@users.noreply.github.com> Signed-off-by: Antonio Jimenez <antonjim@thousandEyes.com> Signed-off-by: Antonio Jimenez <123171955+antonjim-te@users.noreply.github.com> Signed-off-by: subhramit <subhramit.bb@live.in> Signed-off-by: zhengkezhou1 <madzhou1@gmail.com> Signed-off-by: tongjicoder <tongjicoder@icloud.com> Signed-off-by: Anand Rajagopal <anrajag@amazon.com> Signed-off-by: beorn7 <beorn@grafana.com> Signed-off-by: Innokentii Konstantinov <innokenty.konstantinov@grafana.com> Signed-off-by: AxcelXander <tyz666@bu.edu> Signed-off-by: Aman <95525722+amanycodes@users.noreply.github.com> Signed-off-by: Julien Pivotto <291750+roidelapluie@users.noreply.github.com> Signed-off-by: Owen Williams <owen.williams@grafana.com> Signed-off-by: datamuc <m@rbfh.de> Signed-off-by: 🌲 Harry 🌊 John 🏔 <johrry@amazon.com> Signed-off-by: Kapil Lamba <kapillamba4@gmail.com> Signed-off-by: Simon Pasquier <spasquie@redhat.com> Signed-off-by: Carrie Edwards <edwrdscarrie@gmail.com> Signed-off-by: rishi-jat <rishijat098@gmail.com> Signed-off-by: Rishi Jat <rishijat098@gmail.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Signed-off-by: Coleen Iona Quadros <coleen.quadros27@gmail.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Jan-Otto Kröpke <mail@jkroepke.de> Co-authored-by: Neeraj Gartia <80708727+NeerajGartia21@users.noreply.github.com> Co-authored-by: Julien <291750+roidelapluie@users.noreply.github.com> Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com> Co-authored-by: Björn Rabenstein <beorn@grafana.com> Co-authored-by: Bryan Boreham <bjboreham@gmail.com> Co-authored-by: Arthur Silva Sens <arthursens2005@gmail.com> Co-authored-by: Michael Hoffmann <mhoffm@posteo.de> Co-authored-by: Björn Rabenstein <github@rabenste.in> Co-authored-by: machine424 <ayoubmrini424@gmail.com> Co-authored-by: Fiona Liao <fiona.liao@grafana.com> Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com> Co-authored-by: Zhang Zhanpeng <zhangzhanpeng.zzp@alibaba-inc.com> Co-authored-by: Aman <95525722+amanycodes@users.noreply.github.com> Co-authored-by: amanycodes <amanycodes@gmail.com> Co-authored-by: Julius Volz <julius.volz@gmail.com> Co-authored-by: SungJin1212 <tjdwls1201@gmail.com> Co-authored-by: Yuchen Wang <162491048+yuchen-db@…
Fix for #16367
Adding a simple script to convert tests automatically to new syntax.