Skip to content

Conversation

lamida
Copy link
Contributor

@lamida lamida commented Jan 27, 2025

What this PR does

Implement absent function. Added a new Operator: Absent. We also need to modify InstantVectorFunctionOperatorFactory to pass parser.Expressions object needed to evaluate the labels of the absent argument.

Benchmark

Step to create the benchmark:

  • Add test in pkg/streamingpromql/benchmarks/benchmarks.go
  • cd tools/benchmark-query-engine
  • go run . -bench=Query/absent -count=6 | tee absent-benchmark.txt
  • ./compare.sh absent-benchmark.txt
  • rm absent-benchmark.txt
goos: darwin
goarch: arm64
pkg: github.com/grafana/mimir/pkg/streamingpromql/benchmarks
cpu: Apple M1 Max
                                                           │  Prometheus   │                Mimir                │
                                                           │    sec/op     │    sec/op     vs base               │
Query/absent(a_1),_instant_query-10                           144.0µ ±  4%   144.2µ ±  8%        ~ (p=0.699 n=6)
Query/absent(a_1),_range_query_with_100_steps-10              154.3µ ±  5%   147.4µ ±  2%   -4.44% (p=0.004 n=6)
Query/absent(a_1),_range_query_with_1000_steps-10             250.2µ ± 12%   199.0µ ±  3%  -20.47% (p=0.002 n=6)
Query/absent(a_100),_instant_query-10                         752.1µ ±  5%   731.9µ ±  3%   -2.69% (p=0.015 n=6)
Query/absent(a_100),_range_query_with_100_steps-10            1.347m ±  3%   1.250m ±  1%   -7.21% (p=0.002 n=6)
Query/absent(a_100),_range_query_with_1000_steps-10           6.264m ±  1%   5.454m ±  1%  -12.93% (p=0.002 n=6)
Query/absent(a_2000),_instant_query-10                       10.416m ±  2%   9.895m ±  9%        ~ (p=0.065 n=6)
Query/absent(a_2000),_range_query_with_100_steps-10           21.42m ± 13%   19.24m ± 11%  -10.16% (p=0.015 n=6)
Query/absent(a_2000),_range_query_with_1000_steps-10         146.07m ±  1%   97.98m ±  3%  -32.92% (p=0.002 n=6)
Query/absent(a_1_>_Inf),_instant_query-10                     127.0µ ± 15%   117.8µ ±  9%   -7.25% (p=0.026 n=6)
Query/absent(a_1_>_Inf),_range_query_with_100_steps-10        176.2µ ±  3%   128.1µ ±  6%  -27.30% (p=0.002 n=6)
Query/absent(a_1_>_Inf),_range_query_with_1000_steps-10       578.7µ ±  4%   190.6µ ±  4%  -67.07% (p=0.002 n=6)
Query/absent(a_100_>_Inf),_instant_query-10                   768.5µ ±  7%   734.2µ ±  6%   -4.46% (p=0.041 n=6)
Query/absent(a_100_>_Inf),_range_query_with_100_steps-10      1.500m ±  3%   1.305m ±  2%  -13.03% (p=0.002 n=6)
Query/absent(a_100_>_Inf),_range_query_with_1000_steps-10     7.671m ±  4%   5.881m ±  1%  -23.34% (p=0.002 n=6)
Query/absent(a_2000_>_Inf),_instant_query-10                  10.49m ± 12%   10.05m ±  2%   -4.19% (p=0.002 n=6)
Query/absent(a_2000_>_Inf),_range_query_with_100_steps-10     23.52m ±  9%   20.16m ±  2%  -14.28% (p=0.002 n=6)
Query/absent(a_2000_>_Inf),_range_query_with_1000_steps-10    171.2m ±  5%   106.4m ±  5%  -37.86% (p=0.002 n=6)
geomean                                                       2.370m         1.929m        -18.58%

                                                           │  Prometheus   │                Mimir                │
                                                           │     B/op      │     B/op      vs base               │
Query/absent(a_1),_instant_query-10                           22.23Ki ± 0%   20.85Ki ± 0%   -6.22% (p=0.002 n=6)
Query/absent(a_1),_range_query_with_100_steps-10              25.65Ki ± 0%   21.65Ki ± 0%  -15.58% (p=0.002 n=6)
Query/absent(a_1),_range_query_with_1000_steps-10             51.70Ki ± 0%   26.31Ki ± 0%  -49.12% (p=0.002 n=6)
Query/absent(a_100),_instant_query-10                         194.9Ki ± 0%   150.5Ki ± 0%  -22.78% (p=0.002 n=6)
Query/absent(a_100),_range_query_with_100_steps-10            253.7Ki ± 0%   206.9Ki ± 0%  -18.44% (p=0.002 n=6)
Query/absent(a_100),_range_query_with_1000_steps-10           697.5Ki ± 0%   632.3Ki ± 0%   -9.35% (p=0.002 n=6)
Query/absent(a_2000),_instant_query-10                        3.536Mi ± 0%   2.577Mi ± 0%  -27.11% (p=0.002 n=6)
Query/absent(a_2000),_range_query_with_100_steps-10           4.660Mi ± 0%   3.679Mi ± 0%  -21.04% (p=0.002 n=6)
Query/absent(a_2000),_range_query_with_1000_steps-10          12.80Mi ± 1%   11.85Mi ± 1%   -7.49% (p=0.002 n=6)
Query/absent(a_1_>_Inf),_instant_query-10                     25.20Ki ± 0%   21.42Ki ± 0%  -15.02% (p=0.002 n=6)
Query/absent(a_1_>_Inf),_range_query_with_100_steps-10        56.87Ki ± 0%   22.16Ki ± 0%  -61.03% (p=0.002 n=6)
Query/absent(a_1_>_Inf),_range_query_with_1000_steps-10      338.08Ki ± 0%   26.59Ki ± 0%  -92.14% (p=0.002 n=6)
Query/absent(a_100_>_Inf),_instant_query-10                   198.0Ki ± 0%   153.3Ki ± 0%  -22.54% (p=0.002 n=6)
Query/absent(a_100_>_Inf),_range_query_with_100_steps-10      285.2Ki ± 0%   209.7Ki ± 0%  -26.49% (p=0.002 n=6)
Query/absent(a_100_>_Inf),_range_query_with_1000_steps-10     984.6Ki ± 0%   631.3Ki ± 0%  -35.88% (p=0.002 n=6)
Query/absent(a_2000_>_Inf),_instant_query-10                  3.542Mi ± 0%   2.655Mi ± 0%  -25.03% (p=0.002 n=6)
Query/absent(a_2000_>_Inf),_range_query_with_100_steps-10     4.694Mi ± 0%   3.758Mi ± 0%  -19.95% (p=0.002 n=6)
Query/absent(a_2000_>_Inf),_range_query_with_1000_steps-10    13.05Mi ± 1%   11.89Mi ± 1%   -8.90% (p=0.002 n=6)
geomean                                                       473.8Ki        314.8Ki       -33.56%

                                                           │  Prometheus  │               Mimir                │
                                                           │  allocs/op   │  allocs/op   vs base               │
Query/absent(a_1),_instant_query-10                            401.0 ± 0%    347.0 ± 0%  -13.47% (p=0.002 n=6)
Query/absent(a_1),_range_query_with_100_steps-10               511.0 ± 0%    354.0 ± 0%  -30.72% (p=0.002 n=6)
Query/absent(a_1),_range_query_with_1000_steps-10             1437.0 ± 0%    380.0 ± 0%  -73.56% (p=0.002 n=6)
Query/absent(a_100),_instant_query-10                         2.413k ± 0%   2.252k ± 0%   -6.67% (p=0.002 n=6)
Query/absent(a_100),_range_query_with_100_steps-10            2.830k ± 0%   2.661k ± 0%   -5.97% (p=0.002 n=6)
Query/absent(a_100),_range_query_with_1000_steps-10           6.344k ± 0%   5.275k ± 0%  -16.85% (p=0.002 n=6)
Query/absent(a_2000),_instant_query-10                        40.87k ± 0%   38.80k ± 0%   -5.08% (p=0.002 n=6)
Query/absent(a_2000),_range_query_with_100_steps-10           47.06k ± 0%   46.87k ± 0%   -0.42% (p=0.002 n=6)
Query/absent(a_2000),_range_query_with_1000_steps-10         100.04k ± 0%   98.94k ± 0%   -1.10% (p=0.002 n=6)
Query/absent(a_1_>_Inf),_instant_query-10                      462.0 ± 0%    361.0 ± 0%  -21.86% (p=0.002 n=6)
Query/absent(a_1_>_Inf),_range_query_with_100_steps-10         970.5 ± 0%    368.0 ± 0%  -62.08% (p=0.002 n=6)
Query/absent(a_1_>_Inf),_range_query_with_1000_steps-10       5499.0 ± 0%    394.0 ± 0%  -92.84% (p=0.002 n=6)
Query/absent(a_100_>_Inf),_instant_query-10                   2.474k ± 0%   2.269k ± 0%   -8.29% (p=0.002 n=6)
Query/absent(a_100_>_Inf),_range_query_with_100_steps-10      3.289k ± 0%   2.679k ± 0%  -18.55% (p=0.002 n=6)
Query/absent(a_100_>_Inf),_range_query_with_1000_steps-10    10.406k ± 0%   5.293k ± 0%  -49.14% (p=0.002 n=6)
Query/absent(a_2000_>_Inf),_instant_query-10                  40.93k ± 0%   38.82k ± 0%   -5.15% (p=0.002 n=6)
Query/absent(a_2000_>_Inf),_range_query_with_100_steps-10     47.50k ± 0%   46.88k ± 0%   -1.31% (p=0.002 n=6)
Query/absent(a_2000_>_Inf),_range_query_with_1000_steps-10   104.09k ± 0%   98.99k ± 0%   -4.90% (p=0.002 n=6)
geomean                                                       6.007k        4.035k       -32.82%

                                                           │  Prometheus   │                Mimir                │
                                                           │       B       │      B        vs base               │
Query/absent(a_1),_instant_query-10                           67.13Mi ± 1%   66.93Mi ± 1%        ~ (p=0.240 n=6)
Query/absent(a_1),_range_query_with_100_steps-10              67.01Mi ± 2%   66.65Mi ± 1%        ~ (p=0.686 n=6)
Query/absent(a_1),_range_query_with_1000_steps-10             64.52Mi ± 2%   65.21Mi ± 1%   +1.07% (p=0.041 n=6)
Query/absent(a_100),_instant_query-10                         62.32Mi ± 1%   61.69Mi ± 1%        ~ (p=0.093 n=6)
Query/absent(a_100),_range_query_with_100_steps-10            62.62Mi ± 1%   62.02Mi ± 2%        ~ (p=0.132 n=6)
Query/absent(a_100),_range_query_with_1000_steps-10           64.91Mi ± 1%   61.78Mi ± 1%   -4.83% (p=0.002 n=6)
Query/absent(a_2000),_instant_query-10                        63.10Mi ± 1%   63.23Mi ± 1%        ~ (p=0.786 n=6)
Query/absent(a_2000),_range_query_with_100_steps-10           71.98Mi ± 2%   62.62Mi ± 1%  -13.00% (p=0.002 n=6)
Query/absent(a_2000),_range_query_with_1000_steps-10         127.16Mi ± 3%   68.32Mi ± 2%  -46.27% (p=0.002 n=6)
Query/absent(a_1_>_Inf),_instant_query-10                     67.08Mi ± 2%   67.46Mi ± 1%        ~ (p=0.193 n=6)
Query/absent(a_1_>_Inf),_range_query_with_100_steps-10        65.09Mi ± 1%   67.28Mi ± 2%   +3.36% (p=0.002 n=6)
Query/absent(a_1_>_Inf),_range_query_with_1000_steps-10       63.39Mi ± 1%   65.66Mi ± 1%   +3.57% (p=0.002 n=6)
Query/absent(a_100_>_Inf),_instant_query-10                   62.13Mi ± 1%   62.17Mi ± 1%        ~ (p=1.000 n=6)
Query/absent(a_100_>_Inf),_range_query_with_100_steps-10      62.62Mi ± 1%   61.56Mi ± 1%   -1.68% (p=0.002 n=6)
Query/absent(a_100_>_Inf),_range_query_with_1000_steps-10     65.15Mi ± 0%   61.96Mi ± 0%   -4.89% (p=0.002 n=6)
Query/absent(a_2000_>_Inf),_instant_query-10                  63.38Mi ± 3%   63.34Mi ± 2%        ~ (p=0.732 n=6)
Query/absent(a_2000_>_Inf),_range_query_with_100_steps-10     71.33Mi ± 1%   62.75Mi ± 1%  -12.03% (p=0.002 n=6)
Query/absent(a_2000_>_Inf),_range_query_with_1000_steps-10   129.14Mi ± 3%   67.95Mi ± 2%  -47.39% (p=0.002 n=6)
geomean                                                       70.26Mi        64.32Mi        -8.45%

Which issue(s) this PR fixes or relates to

Fixes #

Checklist

  • Tests updated.
  • Documentation added.
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX].
  • about-versioning.md updated with experimental features.

@lamida lamida marked this pull request as ready for review January 27, 2025 14:57
@lamida lamida requested a review from a team as a code owner January 27, 2025 14:57
@lamida lamida requested a review from charleskorn January 27, 2025 17:31
@lamida lamida force-pushed the lamida/mqe-absent-function branch 2 times, most recently from d7818b2 to 51762d5 Compare February 10, 2025 03:21
@lamida lamida force-pushed the lamida/mqe-absent-function branch from 66776aa to 607c848 Compare February 13, 2025 19:47
@lamida lamida requested a review from charleskorn February 13, 2025 20:29
@lamida
Copy link
Contributor Author

lamida commented Feb 13, 2025

@charleskorn I have addressed all of your comments from the previous review. Please help to do another review. 🙏

@lamida lamida requested a review from charleskorn February 14, 2025 10:42
Copy link
Contributor

@jhesketh jhesketh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry if this has already been discussed/considered (if so, please disregard this comment).

However, I wonder if given the uniqueness of this operator if it makes sense to handle it as a special case instead of as an InstantVectorFunctionOperatorFactory?

Eg in convertFunctionCallToInstantVectorOperator in query.go we check if it's the absent function and then create the operator and pass it the full inner expression instead of looking it up in instantVectorFunctionOperatorFactories.

This way the absent operator can reason about the inner series. For example, if I'm understand the function correctly, there is nothing to do if the inner expression is something other than a selector (eg there's no point evaluating a sum).

Sorry if this has already been discussed or derails things too much!

@charleskorn
Copy link
Contributor

charleskorn commented Feb 17, 2025

This way the absent operator can reason about the inner series. For example, if I'm understand the function correctly, there is nothing to do if the inner expression is something other than a selector (eg there's no point evaluating a sum).

absent can be run over any expression that produces an instant vector. The only difference in behaviour between the case where absent is run over a instant vector selector and some other expression is the labels it will return:

  • if the argument is an instant vector selector, absent will infer the output labels from the selector
  • if the argument isn't an instant vector selector, absent will always produce a series with no labels (ie. {})

@jhesketh
Copy link
Contributor

This way the absent operator can reason about the inner series. For example, if I'm understand the function correctly, there is nothing to do if the inner expression is something other than a selector (eg there's no point evaluating a sum).

absent can be run over any expression that produces an instant vector. The only difference in behaviour between the case where absent is run over a instant vector selector and some other expression is the labels it will return:

* if the argument is an instant vector selector, `absent` will infer the output labels from the selector

* if the argument isn't an instant vector selector, `absent` will always produce a series with no labels (ie. `{}`)

Right, I see. However it may still be neater to have the operator called specifically in query.go to avoid passing both an evaluated and non-evaluated expression to all the functions (which feels odd).

@charleskorn
Copy link
Contributor

However it may still be neater to have the operator called specifically in query.go to avoid passing both an evaluated and non-evaluated expression to all the functions (which feels odd).

Are you referring to passing the expression to the function factory? If so, I don't mind this so much, but I don't hold this opinion strongly. Making absent a special case also feels a bit odd to me.

@jhesketh
Copy link
Contributor

However it may still be neater to have the operator called specifically in query.go to avoid passing both an evaluated and non-evaluated expression to all the functions (which feels odd).

Are you referring to passing the expression to the function factory? If so, I don't mind this so much, but I don't hold this opinion strongly. Making absent a special case also feels a bit odd to me.

Yes, that's what I was referring to.
It feels odd to me to pass both the parsed arguments, and unparsed expressions. Especially since the expressions are only used by absent.
I don't hold this strongly either, but find of the two oddities, handling absent in a special case is my personal preference.

@lamida lamida force-pushed the lamida/mqe-absent-function branch from c9de184 to c5a1d77 Compare February 25, 2025 07:30
@lamida
Copy link
Contributor Author

lamida commented Feb 25, 2025

It feels odd to me to pass both the parsed arguments, and unparsed expressions. Especially since the expressions are only used by absent.

@jhesketh I will consider to refactor this once I am making progress with absent_over_time. Probably with that absent_over_time, it will be more obvious, whether we should pull up passing the unparsed arguments, or instead we should just keep this way. But for now, I prefer just to keep this and have the PR merged once all other concerns are addressed.

But you have some valid points.

@lamida lamida requested a review from charleskorn February 27, 2025 17:37
Copy link
Contributor

@charleskorn charleskorn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Could you please add a benchmark for this and post a comparison with Prometheus' engine? Assuming the results look OK, I'm happy for this to be merged, but if there's any significant regression with MQE, then let's discuss before merging.

(let me know if you need some help with this)

Comment on lines 83 to 86
output := types.InstantVectorSeriesData{}
defer func() { a.called = true }()

if a.called {
return output, types.EOS
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] I think this can be simplified a bit:

Suggested change
output := types.InstantVectorSeriesData{}
defer func() { a.called = true }()
if a.called {
return output, types.EOS
}
if a.called {
return output, types.EOS
}
a.called = true
output := types.InstantVectorSeriesData{}

Also, I think a name like exhausted or used might be clearer - called is a little ambiguous (what was called?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renamed to exhausted and refactor as suggested by removing defer. Except that output initiliasation still need on the top of the line before the first return.

c1536e7

Copy link
Contributor

@charleskorn charleskorn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(meant to approve with the comment above 🤦)

@lamida
Copy link
Contributor Author

lamida commented Feb 28, 2025

Could you please add a benchmark for this and post a comparison with Prometheus' engine? Assuming the results look OK, I'm happy for this to be merged, but if there's any significant regression with MQE, then let's discuss before merging.

Sure I will do the benchmarking shortly. Thanks for checking.

lamida and others added 24 commits February 28, 2025 11:25
Signed-off-by: Jon Kartago Lamida <me@lamida.net>
Co-authored-by: Charles Korn <charleskorn@users.noreply.github.com>
Signed-off-by: Jon Kartago Lamida <me@lamida.net>
Signed-off-by: Jon Kartago Lamida <me@lamida.net>
Signed-off-by: Jon Kartago Lamida <me@lamida.net>
Signed-off-by: Jon Kartago Lamida <me@lamida.net>
Signed-off-by: Jon Kartago Lamida <me@lamida.net>
Signed-off-by: Jon Kartago Lamida <me@lamida.net>
Signed-off-by: Jon Kartago Lamida <me@lamida.net>
Signed-off-by: Jon Kartago Lamida <me@lamida.net>
Signed-off-by: Jon Kartago Lamida <me@lamida.net>
Signed-off-by: Jon Kartago Lamida <me@lamida.net>
Signed-off-by: Jon Kartago Lamida <me@lamida.net>
Signed-off-by: Jon Kartago Lamida <me@lamida.net>
Signed-off-by: Jon Kartago Lamida <me@lamida.net>
Signed-off-by: Jon Kartago Lamida <me@lamida.net>
Signed-off-by: Jon Kartago Lamida <me@lamida.net>
Signed-off-by: Jon Kartago Lamida <me@lamida.net>
Co-authored-by: Charles Korn <charleskorn@users.noreply.github.com>
Signed-off-by: Jon Kartago Lamida <me@lamida.net>
Signed-off-by: Jon Kartago Lamida <me@lamida.net>
Signed-off-by: Jon Kartago Lamida <me@lamida.net>
Signed-off-by: Jon Kartago Lamida <me@lamida.net>
Signed-off-by: Jon Kartago Lamida <me@lamida.net>
@lamida lamida force-pushed the lamida/mqe-absent-function branch from 3a3f44a to c1536e7 Compare February 28, 2025 03:25
Signed-off-by: Jon Kartago Lamida <me@lamida.net>
Signed-off-by: Jon Kartago Lamida <me@lamida.net>
@lamida lamida enabled auto-merge (squash) February 28, 2025 04:00
@lamida lamida merged commit 1065d86 into main Feb 28, 2025
28 checks passed
@lamida lamida deleted the lamida/mqe-absent-function branch February 28, 2025 04:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants