promql: Numerical accuracy issues with mean calculation #16714
Description
The following test failures were reported with go1.24.2 darwin/arm64:
--- FAIL: TestEvaluations/testdata/aggregators.test (0.14s)
--- FAIL: TestEvaluations/testdata/aggregators.test/line_596/avg(data{test="bigzero"}) (0.00s)
test.go:1321:
Error Trace: /Users/bryan/src/github.com/prometheus/prometheus/promql/promqltest/test.go:1321
Error: Received unexpected error:
error in eval avg(data{test="bigzero"}) (line 596): expected 0 for {} but got 2.4948003869184e+291
Test: TestEvaluations/testdata/aggregators.test/line_596/avg(data{test="bigzero"})
--- FAIL: TestEvaluations/testdata/functions.test (0.47s)
--- FAIL: TestEvaluations/testdata/functions.test/line_1013/avg_over_time(metric[2m]) (0.00s)
test.go:1321:
Error Trace: /Users/bryan/src/github.com/prometheus/prometheus/promql/promqltest/test.go:1321
Error: Received unexpected error:
error in eval avg_over_time(metric[2m]) (line 1013): expected 0.5 for {} but got -1.3877787807814457e+83
Test: TestEvaluations/testdata/functions.test/line_1013/avg_over_time(metric[2m])
These tests passed prior to #16569 , which was supposed to improve the accuracy. The tests also pass on linux/amd64.
The mystery here is that both failing tests test quite different aspects, while many other tests test similar aspects and do not fail. This needs to be debugged in detail.
The 1st test is using float64 numbers close to overflow, so that simple mean calculation would fail because the sum of the first two values would result in -Inf.
The 2nd test is more about masking small values by large values, so that Kahan summation saves the day.