Skip to content

Do less Nexting #4753

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 9 commits into from
Feb 28, 2025
Merged

Do less Nexting #4753

merged 9 commits into from
Feb 28, 2025

Conversation

joe-elliott
Copy link
Collaborator

@joe-elliott joe-elliott commented Feb 25, 2025 •
edited
Loading

What this PR does:
Passes the max definition level into the RowNumber.Next() function to reduce the amount of required work. This allows for inlining and less work done when iterating at lower definition levels. This does mean that heavy iteration at higher definition levels is a bit slower. Putting up this PR b/c it shows a lot of promise, but I still want to test a version that reverts to the unrolled version for higher max definition levels.

This PR is a bit messy 😬. Look at RowNumber.Next (I can't link to it b/c it's in the collapsed iter file). All substantive changes flow from this.

Other Changes:

  • Refactored all benchmarks that work on local blocks to respect three env vars for easier testing. BENCH_BLOCKID, BENCH_PATH, BENCH_TENANTID
  • Passed a limit of 20 into the traceql benchmark to better mimic real world behavior.
  • Removed the unrolled RowNumber.Next() implementation although I may restore it.
  • Added a BenchmarkIterators function to better test raw iterator combinations. I find it useful for testing, but am having issues translating perf improvements found here to TraceQL queries.

Generally I feel like more work could be done on our benchmarks to better guide us when making performance improvements. This is a first step.

really long exhaustive benchmarks 
goos: darwin
goarch: arm64
pkg: github.com/grafana/tempo/tempodb/encoding/vparquet4
cpu: Apple M3 Pro
                                                                                                           │ before.txt  │              after.txt              │
                                                                                                           │   sec/op    │   sec/op     vs base                │
FetchTagValues/tag:_span.http.url,_query:_{resource.namespace="tempo-ops"}-11                                14.30m ± 1%   14.15m ± 0%   -1.03% (p=0.000 n=10)
FetchTagValues/tag:_span.component,_query:_{resource.namespace="tempo-ops"}-11                               14.41m ± 2%   14.10m ± 1%   -2.12% (p=0.000 n=10)
FetchTagValues/tag:_span.http.url,_query:_{resource.namespace="tempo-ops"_&&_span.http.status_code=200}-11   14.40m ± 1%   14.17m ± 2%   -1.62% (p=0.009 n=10)
FetchTagValues/tag:_resource.namespace,_query:_{span.http.status_code=200}-11                                549.8m ± 0%   489.1m ± 1%  -11.04% (p=0.000 n=10)
FetchTags/query:_{resource.namespace="tempo-ops"}_span-11                                                    14.40m ± 1%   14.22m ± 1%   -1.23% (p=0.000 n=10)
FetchTags/query:_{resource.namespace="tempo-ops"}_resource-11                                                14.43m ± 0%   14.22m ± 1%   -1.49% (p=0.000 n=10)
FetchTags/query:_{resource.namespace="tempo-ops"}_none-11                                                    14.62m ± 1%   14.28m ± 1%   -2.33% (p=0.000 n=10)
FetchTags/query:_{resource.k8s.node.name>"h"}_span-11                                                         2.129 ± 0%    2.058 ± 1%   -3.30% (p=0.000 n=10)
FetchTags/query:_{resource.k8s.node.name>"h"}_resource-11                                                    947.8m ± 0%   908.9m ± 0%   -4.10% (p=0.000 n=10)
FetchTags/query:_{resource.k8s.node.name>"h"}_none-11                                                         8.772 ± 0%    8.533 ± 0%   -2.72% (p=0.000 n=10)
FetchTags/query:_{span.http.status_code=200}_span-11                                                          1.492 ± 0%    1.386 ± 0%   -7.15% (p=0.000 n=10)
FetchTags/query:_{span.http.status_code=200}_resource-11                                                     856.5m ± 0%   778.0m ± 0%   -9.17% (p=0.000 n=10)
FetchTags/query:_{span.http.status_code=200}_none-11                                                          4.624 ± 0%    4.330 ± 0%   -6.36% (p=0.000 n=10)
FetchTags/query:_{nestedSetParent=-1}_span-11                                                                 1.427 ± 0%    1.322 ± 0%   -7.41% (p=0.000 n=10)
FetchTags/query:_{nestedSetParent=-1}_resource-11                                                            947.9m ± 0%   868.1m ± 0%   -8.42% (p=0.000 n=10)
FetchTags/query:_{nestedSetParent=-1}_none-11                                                                 3.968 ± 0%    3.761 ± 0%   -5.21% (p=0.000 n=10)
FetchTags/query:_{rootName="Memcache.Put"}_span-11                                                           34.58m ± 0%   32.63m ± 0%   -5.64% (p=0.000 n=10)
FetchTags/query:_{rootName="Memcache.Put"}_resource-11                                                       14.90m ± 0%   14.41m ± 0%   -3.29% (p=0.000 n=10)
FetchTags/query:_{rootName="Memcache.Put"}_none-11                                                           149.3m ± 0%   145.8m ± 0%   -2.33% (p=0.000 n=10)
BackendBlockTraceQL/spanAttValMatch-11                                                                       84.91m ± 1%   79.34m ± 0%   -6.56% (p=0.000 n=10)
BackendBlockTraceQL/spanAttValNoMatch-11                                                                     5.589m ± 1%   5.389m ± 1%   -3.58% (p=0.000 n=10)
BackendBlockTraceQL/spanAttIntrinsicMatch-11                                                                 59.21m ± 0%   56.19m ± 0%   -5.11% (p=0.000 n=10)
BackendBlockTraceQL/spanAttIntrinsicNoMatch-11                                                               5.488m ± 1%   5.277m ± 0%   -3.86% (p=0.000 n=10)
BackendBlockTraceQL/resourceAttValMatch-11                                                                   380.8m ± 1%   368.4m ± 0%   -3.26% (p=0.000 n=10)
BackendBlockTraceQL/resourceAttValNoMatch-11                                                                 5.202m ± 0%   5.004m ± 0%   -3.81% (p=0.000 n=10)
BackendBlockTraceQL/resourceAttIntrinsicMatch-11                                                             24.96m ± 0%   24.38m ± 0%   -2.33% (p=0.000 n=10)
BackendBlockTraceQL/resourceAttIntrinsicMatch#01-11                                                          5.003m ± 1%   5.011m ± 0%        ~ (p=0.481 n=10)
BackendBlockTraceQL/traceOrMatch-11                                                                          226.8m ± 0%   222.7m ± 1%   -1.83% (p=0.000 n=10)
BackendBlockTraceQL/traceOrNoMatch-11                                                                        226.6m ± 0%   223.0m ± 0%   -1.59% (p=0.000 n=10)
BackendBlockTraceQL/mixedValNoMatch-11                                                                       173.2m ± 0%   170.2m ± 0%   -1.72% (p=0.000 n=10)
BackendBlockTraceQL/mixedValMixedMatchAnd-11                                                                 4.876m ± 3%   4.869m ± 0%        ~ (p=0.123 n=10)
BackendBlockTraceQL/mixedValMixedMatchOr-11                                                                  128.7m ± 0%   135.8m ± 0%   +5.54% (p=0.000 n=10)
BackendBlockTraceQL/count-11                                                                                 284.4m ± 0%   297.7m ± 1%   +4.68% (p=0.000 n=10)
BackendBlockTraceQL/struct-11                                                                                384.5m ± 2%   411.7m ± 4%   +7.07% (p=0.000 n=10)
BackendBlockTraceQL/||-11                                                                                    131.1m ± 0%   137.1m ± 0%   +4.61% (p=0.000 n=10)
BackendBlockTraceQL/mixed-11                                                                                 24.81m ± 1%   24.74m ± 0%   -0.29% (p=0.004 n=10)
BackendBlockTraceQL/complex-11                                                                               4.678m ± 1%   4.982m ± 1%   +6.50% (p=0.002 n=10)
BackendBlockTraceQL/select-11                                                                                4.713m ± 3%   4.989m ± 1%   +5.86% (p=0.002 n=10)
BackendBlockQueryRange/{}_|_rate()/5-11                                                                      818.5m ± 6%   850.7m ± 1%   +3.93% (p=0.043 n=10)
BackendBlockQueryRange/{}_|_rate()/7-11                                                                      854.2m ± 1%   859.7m ± 0%        ~ (p=0.353 n=10)
BackendBlockQueryRange/{}_|_rate()_by_(name)/5-11                                                             1.707 ± 3%    1.748 ± 5%        ~ (p=0.165 n=10)
BackendBlockQueryRange/{}_|_rate()_by_(name)/7-11                                                             1.879 ± 2%    1.931 ± 1%   +2.78% (p=0.001 n=10)
BackendBlockQueryRange/{}_|_rate()_by_(resource.service.name)/5-11                                            1.163 ± 8%    1.184 ± 7%        ~ (p=0.579 n=10)
BackendBlockQueryRange/{}_|_rate()_by_(resource.service.name)/7-11                                            1.267 ± 1%    1.299 ± 1%   +2.59% (p=0.000 n=10)
BackendBlockQueryRange/{}_|_rate()_by_(span.http.url)/5-11                                                    1.917 ± 2%    1.930 ± 1%        ~ (p=0.315 n=10)
BackendBlockQueryRange/{}_|_rate()_by_(span.http.url)/7-11                                                    1.996 ± 3%    2.028 ± 4%   +1.59% (p=0.009 n=10)
BackendBlockQueryRange/{resource.service.name=`loki-ingester`}_|_rate()/5-11                                 60.70m ± 1%   61.38m ± 0%   +1.12% (p=0.015 n=10)
BackendBlockQueryRange/{resource.service.name=`loki-ingester`}_|_rate()/7-11                                 61.07m ± 5%   61.54m ± 0%   +0.78% (p=0.023 n=10)
BackendBlockQueryRange/{status=error}_|_rate()/5-11                                                          86.87m ± 0%   80.88m ± 0%   -6.89% (p=0.000 n=10)
BackendBlockQueryRange/{status=error}_|_rate()/7-11                                                          87.38m ± 7%   80.97m ± 0%   -7.34% (p=0.000 n=10)
geomean                                                                                                      138.7m        136.3m        -1.72%

                                                                                                           │  before.txt   │               after.txt               │
                                                                                                           │     B/op      │      B/op       vs base               │
FetchTagValues/tag:_span.http.url,_query:_{resource.namespace="tempo-ops"}-11                                14.69Mi ±  1%    14.77Mi ±  2%       ~ (p=0.631 n=10)
FetchTagValues/tag:_span.component,_query:_{resource.namespace="tempo-ops"}-11                               14.63Mi ±  1%    14.71Mi ±  2%       ~ (p=0.971 n=10)
FetchTagValues/tag:_span.http.url,_query:_{resource.namespace="tempo-ops"_&&_span.http.status_code=200}-11   14.63Mi ±  2%    14.72Mi ±  2%       ~ (p=0.579 n=10)
FetchTagValues/tag:_resource.namespace,_query:_{span.http.status_code=200}-11                                8.268Mi ± 16%    7.467Mi ±  8%  -9.68% (p=0.035 n=10)
FetchTags/query:_{resource.namespace="tempo-ops"}_span-11                                                    14.69Mi ±  2%    14.67Mi ±  2%       ~ (p=0.971 n=10)
FetchTags/query:_{resource.namespace="tempo-ops"}_resource-11                                                14.72Mi ±  2%    14.75Mi ±  2%       ~ (p=0.796 n=10)
FetchTags/query:_{resource.namespace="tempo-ops"}_none-11                                                    14.74Mi ±  1%    14.68Mi ±  2%       ~ (p=0.481 n=10)
FetchTags/query:_{resource.k8s.node.name>"h"}_span-11                                                        25.91Mi ±  9%    25.57Mi ±  3%       ~ (p=0.436 n=10)
FetchTags/query:_{resource.k8s.node.name>"h"}_resource-11                                                    18.16Mi ±  2%    18.31Mi ±  3%       ~ (p=0.315 n=10)
FetchTags/query:_{resource.k8s.node.name>"h"}_none-11                                                        54.81Mi ± 22%    45.75Mi ± 22%       ~ (p=0.075 n=10)
FetchTags/query:_{span.http.status_code=200}_span-11                                                         12.78Mi ± 15%    13.40Mi ±  7%       ~ (p=0.165 n=10)
FetchTags/query:_{span.http.status_code=200}_resource-11                                                     6.697Mi ± 10%    7.097Mi ± 20%       ~ (p=0.190 n=10)
FetchTags/query:_{span.http.status_code=200}_none-11                                                         38.62Mi ± 34%    41.24Mi ± 23%       ~ (p=0.684 n=10)
FetchTags/query:_{nestedSetParent=-1}_span-11                                                                11.63Mi ± 18%    12.16Mi ± 27%       ~ (p=0.684 n=10)
FetchTags/query:_{nestedSetParent=-1}_resource-11                                                            9.454Mi ± 26%    9.880Mi ± 24%       ~ (p=0.739 n=10)
FetchTags/query:_{nestedSetParent=-1}_none-11                                                                42.20Mi ± 34%    40.34Mi ± 33%       ~ (p=0.529 n=10)
FetchTags/query:_{rootName="Memcache.Put"}_span-11                                                           7.232Mi ±  3%    7.121Mi ±  1%       ~ (p=0.052 n=10)
FetchTags/query:_{rootName="Memcache.Put"}_resource-11                                                       6.738Mi ±  1%    6.747Mi ±  1%       ~ (p=0.853 n=10)
FetchTags/query:_{rootName="Memcache.Put"}_none-11                                                           15.27Mi ± 12%    15.39Mi ± 22%       ~ (p=1.000 n=10)
BackendBlockTraceQL/spanAttValMatch-11                                                                       45.34Mi ±  1%    45.36Mi ±  1%       ~ (p=0.436 n=10)
BackendBlockTraceQL/spanAttValNoMatch-11                                                                     6.761Mi ±  2%    6.859Mi ±  1%  +1.44% (p=0.004 n=10)
BackendBlockTraceQL/spanAttIntrinsicMatch-11                                                                 39.76Mi ±  1%    39.66Mi ±  0%       ~ (p=0.063 n=10)
BackendBlockTraceQL/spanAttIntrinsicNoMatch-11                                                               7.078Mi ±  2%    7.054Mi ±  2%       ~ (p=0.529 n=10)
BackendBlockTraceQL/resourceAttValMatch-11                                                                   576.6Mi ±  0%    576.1Mi ±  0%       ~ (p=0.481 n=10)
BackendBlockTraceQL/resourceAttValNoMatch-11                                                                 5.514Mi ±  1%    5.349Mi ±  2%  -2.99% (p=0.000 n=10)
BackendBlockTraceQL/resourceAttIntrinsicMatch-11                                                             10.49Mi ±  2%    10.40Mi ±  1%       ~ (p=0.105 n=10)
BackendBlockTraceQL/resourceAttIntrinsicMatch#01-11                                                          6.833Mi ±  0%    6.864Mi ±  1%       ~ (p=0.143 n=10)
BackendBlockTraceQL/traceOrMatch-11                                                                          9.157Mi ± 18%    9.621Mi ± 24%       ~ (p=0.315 n=10)
BackendBlockTraceQL/traceOrNoMatch-11                                                                        9.226Mi ± 12%   10.402Mi ± 15%       ~ (p=0.123 n=10)
BackendBlockTraceQL/mixedValNoMatch-11                                                                       8.225Mi ±  9%    8.088Mi ± 13%       ~ (p=0.853 n=10)
BackendBlockTraceQL/mixedValMixedMatchAnd-11                                                                 5.976Mi ±  1%    5.945Mi ±  1%       ~ (p=0.218 n=10)
BackendBlockTraceQL/mixedValMixedMatchOr-11                                                                  7.859Mi ± 11%    8.189Mi ±  8%       ~ (p=0.684 n=10)
BackendBlockTraceQL/count-11                                                                                 418.3Mi ±  0%    417.8Mi ±  0%       ~ (p=0.105 n=10)
BackendBlockTraceQL/struct-11                                                                                14.30Mi ± 25%    13.95Mi ± 14%       ~ (p=1.000 n=10)
BackendBlockTraceQL/||-11                                                                                    16.65Mi ±  4%    16.55Mi ±  3%       ~ (p=1.000 n=10)
BackendBlockTraceQL/mixed-11                                                                                 6.942Mi ±  2%    6.949Mi ±  1%       ~ (p=0.631 n=10)
BackendBlockTraceQL/complex-11                                                                               6.150Mi ±  2%    5.803Mi ±  3%  -5.65% (p=0.000 n=10)
BackendBlockTraceQL/select-11                                                                                5.883Mi ±  3%    5.769Mi ±  1%  -1.94% (p=0.004 n=10)
BackendBlockQueryRange/{}_|_rate()/5-11                                                                      28.90Mi ±  9%    29.13Mi ±  7%       ~ (p=0.912 n=10)
BackendBlockQueryRange/{}_|_rate()/7-11                                                                      27.98Mi ± 24%    28.68Mi ± 23%       ~ (p=0.579 n=10)
BackendBlockQueryRange/{}_|_rate()_by_(name)/5-11                                                            32.83Mi ± 57%    32.89Mi ± 63%       ~ (p=0.739 n=10)
BackendBlockQueryRange/{}_|_rate()_by_(name)/7-11                                                            31.80Mi ± 43%    32.27Mi ± 11%       ~ (p=0.971 n=10)
BackendBlockQueryRange/{}_|_rate()_by_(resource.service.name)/5-11                                           31.94Mi ± 52%    29.54Mi ± 47%       ~ (p=0.353 n=10)
BackendBlockQueryRange/{}_|_rate()_by_(resource.service.name)/7-11                                           27.13Mi ± 43%    30.95Mi ± 12%       ~ (p=0.165 n=10)
BackendBlockQueryRange/{}_|_rate()_by_(span.http.url)/5-11                                                   355.0Mi ±  4%    360.7Mi ±  3%       ~ (p=0.165 n=10)
BackendBlockQueryRange/{}_|_rate()_by_(span.http.url)/7-11                                                   390.1Mi ±  3%    382.5Mi ±  3%  -1.95% (p=0.035 n=10)
BackendBlockQueryRange/{resource.service.name=`loki-ingester`}_|_rate()/5-11                                 23.61Mi ±  2%    23.07Mi ±  3%  -2.31% (p=0.043 n=10)
BackendBlockQueryRange/{resource.service.name=`loki-ingester`}_|_rate()/7-11                                 23.68Mi ±  4%    23.35Mi ±  2%       ~ (p=0.631 n=10)
BackendBlockQueryRange/{status=error}_|_rate()/5-11                                                          6.389Mi ±  3%    6.711Mi ±  5%       ~ (p=0.063 n=10)
BackendBlockQueryRange/{status=error}_|_rate()/7-11                                                          6.493Mi ±  3%    6.571Mi ±  6%       ~ (p=0.247 n=10)
geomean                                                                                                      18.17Mi          18.21Mi        +0.22%

                                                                                                           │  before.txt  │              after.txt              │
                                                                                                           │  allocs/op   │  allocs/op    vs base               │
FetchTagValues/tag:_span.http.url,_query:_{resource.namespace="tempo-ops"}-11                                81.37k ±  0%   81.38k ±  0%       ~ (p=0.469 n=10)
FetchTagValues/tag:_span.component,_query:_{resource.namespace="tempo-ops"}-11                               81.37k ±  0%   81.37k ±  0%       ~ (p=0.785 n=10)
FetchTagValues/tag:_span.http.url,_query:_{resource.namespace="tempo-ops"_&&_span.http.status_code=200}-11   81.39k ±  0%   81.39k ±  0%       ~ (p=0.324 n=10)
FetchTagValues/tag:_resource.namespace,_query:_{span.http.status_code=200}-11                                89.72k ±  0%   89.68k ±  0%  -0.05% (p=0.001 n=10)
FetchTags/query:_{resource.namespace="tempo-ops"}_span-11                                                    81.39k ±  0%   81.39k ±  0%       ~ (p=0.810 n=10)
FetchTags/query:_{resource.namespace="tempo-ops"}_resource-11                                                81.39k ±  0%   81.39k ±  0%       ~ (p=0.724 n=10)
FetchTags/query:_{resource.namespace="tempo-ops"}_none-11                                                    81.52k ±  0%   81.51k ±  0%       ~ (p=0.446 n=10)
FetchTags/query:_{resource.k8s.node.name>"h"}_span-11                                                        165.7k ±  0%   165.7k ±  0%       ~ (p=0.591 n=10)
FetchTags/query:_{resource.k8s.node.name>"h"}_resource-11                                                    89.14k ±  0%   89.14k ±  0%       ~ (p=0.306 n=10)
FetchTags/query:_{resource.k8s.node.name>"h"}_none-11                                                        258.9k ±  0%   258.9k ±  0%       ~ (p=0.984 n=10)
FetchTags/query:_{span.http.status_code=200}_span-11                                                         174.8k ±  0%   174.9k ±  0%       ~ (p=0.118 n=10)
FetchTags/query:_{span.http.status_code=200}_resource-11                                                     92.10k ±  0%   92.12k ±  0%  +0.02% (p=0.000 n=10)
FetchTags/query:_{span.http.status_code=200}_none-11                                                         314.5k ±  0%   314.5k ±  0%       ~ (p=0.529 n=10)
FetchTags/query:_{nestedSetParent=-1}_span-11                                                                143.3k ±  0%   143.3k ±  0%       ~ (p=0.493 n=10)
FetchTags/query:_{nestedSetParent=-1}_resource-11                                                            98.18k ±  0%   98.18k ±  0%       ~ (p=0.897 n=10)
FetchTags/query:_{nestedSetParent=-1}_none-11                                                                301.7k ±  0%   301.7k ±  0%       ~ (p=0.085 n=10)
FetchTags/query:_{rootName="Memcache.Put"}_span-11                                                           82.35k ±  0%   82.34k ±  0%       ~ (p=0.065 n=10)
FetchTags/query:_{rootName="Memcache.Put"}_resource-11                                                       81.79k ±  0%   81.79k ±  0%       ~ (p=0.946 n=10)
FetchTags/query:_{rootName="Memcache.Put"}_none-11                                                           91.07k ±  0%   91.08k ±  0%       ~ (p=0.669 n=10)
BackendBlockTraceQL/spanAttValMatch-11                                                                       503.5k ±  0%   503.5k ±  0%       ~ (p=0.183 n=10)
BackendBlockTraceQL/spanAttValNoMatch-11                                                                     79.48k ±  0%   79.48k ±  0%  +0.00% (p=0.007 n=10)
BackendBlockTraceQL/spanAttIntrinsicMatch-11                                                                 288.1k ±  0%   288.1k ±  0%       ~ (p=0.424 n=10)
BackendBlockTraceQL/spanAttIntrinsicNoMatch-11                                                               79.44k ±  0%   79.44k ±  0%       ~ (p=1.000 n=10)
BackendBlockTraceQL/resourceAttValMatch-11                                                                   3.450M ±  0%   3.450M ±  0%       ~ (p=0.898 n=10)
BackendBlockTraceQL/resourceAttValNoMatch-11                                                                 79.48k ±  0%   79.48k ±  0%  -0.00% (p=0.002 n=10)
BackendBlockTraceQL/resourceAttIntrinsicMatch-11                                                             126.1k ±  0%   126.1k ±  0%       ~ (p=0.098 n=10)
BackendBlockTraceQL/resourceAttIntrinsicMatch#01-11                                                          79.47k ±  0%   79.47k ±  0%       ~ (p=0.084 n=10)
BackendBlockTraceQL/traceOrMatch-11                                                                          86.47k ±  2%   86.40k ±  1%       ~ (p=0.739 n=10)
BackendBlockTraceQL/traceOrNoMatch-11                                                                        86.36k ±  0%   86.66k ±  1%  +0.35% (p=0.002 n=10)
BackendBlockTraceQL/mixedValNoMatch-11                                                                       80.31k ±  0%   80.30k ±  0%       ~ (p=0.926 n=10)
BackendBlockTraceQL/mixedValMixedMatchAnd-11                                                                 79.48k ±  0%   79.48k ±  0%       ~ (p=0.592 n=10)
BackendBlockTraceQL/mixedValMixedMatchOr-11                                                                  80.28k ±  0%   80.28k ±  0%       ~ (p=1.000 n=10)
BackendBlockTraceQL/count-11                                                                                 1.990M ±  0%   1.990M ±  0%  -0.00% (p=0.022 n=10)
BackendBlockTraceQL/struct-11                                                                                93.98k ±  7%   93.66k ±  3%       ~ (p=0.481 n=10)
BackendBlockTraceQL/||-11                                                                                    161.9k ±  0%   161.9k ±  0%       ~ (p=0.956 n=10)
BackendBlockTraceQL/mixed-11                                                                                 87.96k ±  0%   87.96k ±  0%       ~ (p=0.180 n=10)
BackendBlockTraceQL/complex-11                                                                               79.85k ±  0%   79.84k ±  0%  -0.01% (p=0.000 n=10)
BackendBlockTraceQL/select-11                                                                                79.72k ±  0%   79.71k ±  0%  -0.00% (p=0.009 n=10)
BackendBlockQueryRange/{}_|_rate()/5-11                                                                      82.28k ±  9%   89.48k ±  8%       ~ (p=0.050 n=10)
BackendBlockQueryRange/{}_|_rate()/7-11                                                                      89.50k ±  8%   82.37k ±  9%       ~ (p=0.085 n=10)
BackendBlockQueryRange/{}_|_rate()_by_(name)/5-11                                                            90.80k ± 61%   90.80k ± 61%       ~ (p=0.853 n=10)
BackendBlockQueryRange/{}_|_rate()_by_(name)/7-11                                                            92.14k ± 32%   92.14k ±  0%       ~ (p=0.986 n=10)
BackendBlockQueryRange/{}_|_rate()_by_(resource.service.name)/5-11                                           83.82k ±  0%   83.80k ±  0%       ~ (p=0.288 n=10)
BackendBlockQueryRange/{}_|_rate()_by_(resource.service.name)/7-11                                           83.97k ±  0%   84.01k ±  0%  +0.05% (p=0.009 n=10)
BackendBlockQueryRange/{}_|_rate()_by_(span.http.url)/5-11                                                   534.5k ±  5%   557.6k ±  3%       ~ (p=0.063 n=10)
BackendBlockQueryRange/{}_|_rate()_by_(span.http.url)/7-11                                                   683.7k ±  0%   682.1k ±  3%       ~ (p=0.353 n=10)
BackendBlockQueryRange/{resource.service.name=`loki-ingester`}_|_rate()/5-11                                 156.0k ±  0%   155.9k ±  0%  -0.01% (p=0.037 n=10)
BackendBlockQueryRange/{resource.service.name=`loki-ingester`}_|_rate()/7-11                                 156.0k ±  0%   155.9k ±  0%       ~ (p=0.566 n=10)
BackendBlockQueryRange/{status=error}_|_rate()/5-11                                                          88.97k ±  0%   88.98k ±  0%       ~ (p=0.078 n=10)
BackendBlockQueryRange/{status=error}_|_rate()/7-11                                                          88.97k ±  0%   88.98k ±  0%       ~ (p=0.493 n=10)
geomean                                                                                                      130.7k         130.8k        +0.08%

                                                    │  before.txt  │              after.txt              │
                                                    │     B/s      │     B/s       vs base               │
BackendBlockTraceQL/spanAttValMatch-11                263.7Mi ± 1%   282.2Mi ± 0%  +7.02% (p=0.000 n=10)
BackendBlockTraceQL/spanAttValNoMatch-11              302.3Mi ± 1%   313.6Mi ± 1%  +3.71% (p=0.000 n=10)
BackendBlockTraceQL/spanAttIntrinsicMatch-11          393.4Mi ± 0%   414.6Mi ± 0%  +5.38% (p=0.000 n=10)
BackendBlockTraceQL/spanAttIntrinsicNoMatch-11        451.4Mi ± 1%   469.5Mi ± 0%  +4.01% (p=0.000 n=10)
BackendBlockTraceQL/resourceAttValMatch-11            57.70Mi ± 1%   59.65Mi ± 0%  +3.38% (p=0.000 n=10)
BackendBlockTraceQL/resourceAttValNoMatch-11          172.9Mi ± 0%   179.8Mi ± 0%  +3.95% (p=0.000 n=10)
BackendBlockTraceQL/resourceAttIntrinsicMatch-11      882.2Mi ± 0%   903.2Mi ± 0%  +2.38% (p=0.000 n=10)
BackendBlockTraceQL/resourceAttIntrinsicMatch#01-11   186.6Mi ± 1%   186.3Mi ± 0%       ~ (p=0.468 n=10)
BackendBlockTraceQL/traceOrMatch-11                   7.372Mi ± 0%   7.505Mi ± 1%  +1.81% (p=0.000 n=10)
BackendBlockTraceQL/traceOrNoMatch-11                 7.377Mi ± 0%   7.496Mi ± 1%  +1.62% (p=0.000 n=10)
BackendBlockTraceQL/mixedValNoMatch-11                11.38Mi ± 0%   11.58Mi ± 0%  +1.72% (p=0.000 n=10)
BackendBlockTraceQL/mixedValMixedMatchAnd-11          183.1Mi ± 3%   183.3Mi ± 0%       ~ (p=0.110 n=10)
BackendBlockTraceQL/mixedValMixedMatchOr-11           21.44Mi ± 0%   20.32Mi ± 0%  -5.25% (p=0.000 n=10)
BackendBlockTraceQL/count-11                          77.24Mi ± 0%   73.78Mi ± 1%  -4.48% (p=0.000 n=10)
BackendBlockTraceQL/struct-11                         14.20Mi ± 2%   13.27Mi ± 4%  -6.58% (p=0.000 n=10)
BackendBlockTraceQL/||-11                             168.4Mi ± 0%   161.0Mi ± 0%  -4.41% (p=0.000 n=10)
BackendBlockTraceQL/mixed-11                          861.0Mi ± 1%   863.5Mi ± 0%  +0.29% (p=0.004 n=10)
BackendBlockTraceQL/complex-11                        192.4Mi ± 1%   180.7Mi ± 1%  -6.10% (p=0.002 n=10)
BackendBlockTraceQL/select-11                         191.0Mi ± 3%   180.4Mi ± 1%  -5.54% (p=0.002 n=10)
geomean                                               106.5Mi        106.6Mi       +0.07%

                                                    │ before.txt  │              after.txt               │
                                                    │  MB_io/op   │  MB_io/op    vs base                 │
BackendBlockTraceQL/spanAttValMatch-11                 23.48 ± 0%    23.48 ± 0%       ~ (p=1.000 n=10) ¹
BackendBlockTraceQL/spanAttValNoMatch-11               1.772 ± 0%    1.772 ± 0%       ~ (p=1.000 n=10) ¹
BackendBlockTraceQL/spanAttIntrinsicMatch-11           24.43 ± 0%    24.43 ± 0%       ~ (p=1.000 n=10) ¹
BackendBlockTraceQL/spanAttIntrinsicNoMatch-11         2.598 ± 0%    2.598 ± 0%       ~ (p=1.000 n=10) ¹
BackendBlockTraceQL/resourceAttValMatch-11             23.04 ± 0%    23.04 ± 0%       ~ (p=1.000 n=10) ¹
BackendBlockTraceQL/resourceAttValNoMatch-11          943.2m ± 0%   943.2m ± 0%       ~ (p=1.000 n=10) ¹
BackendBlockTraceQL/resourceAttIntrinsicMatch-11       23.09 ± 0%    23.09 ± 0%       ~ (p=1.000 n=10) ¹
BackendBlockTraceQL/resourceAttIntrinsicMatch#01-11   979.0m ± 0%   979.0m ± 0%       ~ (p=1.000 n=10) ¹
BackendBlockTraceQL/traceOrMatch-11                    1.753 ± 0%    1.753 ± 0%       ~ (p=1.000 n=10) ¹
BackendBlockTraceQL/traceOrNoMatch-11                  1.753 ± 0%    1.753 ± 0%       ~ (p=1.000 n=10) ¹
BackendBlockTraceQL/mixedValNoMatch-11                 2.067 ± 0%    2.067 ± 0%       ~ (p=1.000 n=10) ¹
BackendBlockTraceQL/mixedValMixedMatchAnd-11          936.1m ± 0%   936.1m ± 0%       ~ (p=1.000 n=10) ¹
BackendBlockTraceQL/mixedValMixedMatchOr-11            2.893 ± 0%    2.893 ± 0%       ~ (p=1.000 n=10) ¹
BackendBlockTraceQL/count-11                           23.03 ± 0%    23.03 ± 0%       ~ (p=1.000 n=10) ¹
BackendBlockTraceQL/struct-11                          5.726 ± 0%    5.726 ± 0%       ~ (p=1.000 n=10) ¹
BackendBlockTraceQL/||-11                              23.14 ± 0%    23.14 ± 0%       ~ (p=1.000 n=10) ¹
BackendBlockTraceQL/mixed-11                           22.40 ± 0%    22.40 ± 0%       ~ (p=1.000 n=10) ¹
BackendBlockTraceQL/complex-11                        943.7m ± 0%   943.7m ± 0%       ~ (p=1.000 n=10) ¹
BackendBlockTraceQL/select-11                         943.7m ± 0%   943.7m ± 0%       ~ (p=1.000 n=10) ¹
geomean                                                4.351         4.351       +0.00%
¹ all samples are equal

                                                                             │ before.txt │              after.txt              │
                                                                             │  MB_IO/op  │  MB_IO/op   vs base                 │
BackendBlockQueryRange/{}_|_rate()/5-11                                        26.19 ± 0%   26.19 ± 0%       ~ (p=1.000 n=10) ¹
BackendBlockQueryRange/{}_|_rate()/7-11                                        26.19 ± 0%   26.19 ± 0%       ~ (p=1.000 n=10) ¹
BackendBlockQueryRange/{}_|_rate()_by_(name)/5-11                              34.40 ± 0%   34.40 ± 0%       ~ (p=1.000 n=10) ¹
BackendBlockQueryRange/{}_|_rate()_by_(name)/7-11                              34.40 ± 0%   34.40 ± 0%       ~ (p=1.000 n=10) ¹
BackendBlockQueryRange/{}_|_rate()_by_(resource.service.name)/5-11             26.51 ± 0%   26.51 ± 0%       ~ (p=1.000 n=10) ¹
BackendBlockQueryRange/{}_|_rate()_by_(resource.service.name)/7-11             26.51 ± 0%   26.51 ± 0%       ~ (p=1.000 n=10) ¹
BackendBlockQueryRange/{}_|_rate()_by_(span.http.url)/5-11                     29.01 ± 0%   29.01 ± 0%       ~ (p=1.000 n=10) ¹
BackendBlockQueryRange/{}_|_rate()_by_(span.http.url)/7-11                     29.01 ± 0%   29.01 ± 0%       ~ (p=1.000 n=10) ¹
BackendBlockQueryRange/{resource.service.name=`loki-ingester`}_|_rate()/5-11   26.43 ± 0%   26.43 ± 0%       ~ (p=1.000 n=10) ¹
BackendBlockQueryRange/{resource.service.name=`loki-ingester`}_|_rate()/7-11   26.43 ± 0%   26.43 ± 0%       ~ (p=1.000 n=10) ¹
BackendBlockQueryRange/{status=error}_|_rate()/5-11                            27.55 ± 0%   27.55 ± 0%       ~ (p=1.000 n=10) ¹
BackendBlockQueryRange/{status=error}_|_rate()/7-11                            27.55 ± 0%   27.55 ± 0%       ~ (p=1.000 n=10) ¹
geomean                                                                        28.21        28.21       +0.00%
¹ all samples are equal

                                                                             │ before.txt  │              after.txt               │
                                                                             │  spans/op   │  spans/op    vs base                 │
BackendBlockQueryRange/{}_|_rate()/5-11                                        4.916M ± 0%   4.916M ± 0%       ~ (p=1.000 n=10) ¹
BackendBlockQueryRange/{}_|_rate()/7-11                                        7.271M ± 0%   7.271M ± 0%       ~ (p=1.000 n=10) ¹
BackendBlockQueryRange/{}_|_rate()_by_(name)/5-11                              4.916M ± 0%   4.916M ± 0%       ~ (p=1.000 n=10) ¹
BackendBlockQueryRange/{}_|_rate()_by_(name)/7-11                              7.271M ± 0%   7.271M ± 0%       ~ (p=1.000 n=10) ¹
BackendBlockQueryRange/{}_|_rate()_by_(resource.service.name)/5-11             4.916M ± 0%   4.916M ± 0%       ~ (p=1.000 n=10) ¹
BackendBlockQueryRange/{}_|_rate()_by_(resource.service.name)/7-11             7.271M ± 0%   7.271M ± 0%       ~ (p=1.000 n=10) ¹
BackendBlockQueryRange/{}_|_rate()_by_(span.http.url)/5-11                     4.916M ± 0%   4.916M ± 0%       ~ (p=1.000 n=10) ¹
BackendBlockQueryRange/{}_|_rate()_by_(span.http.url)/7-11                     7.271M ± 0%   7.271M ± 0%       ~ (p=1.000 n=10) ¹
BackendBlockQueryRange/{resource.service.name=`loki-ingester`}_|_rate()/5-11   60.97k ± 0%   60.97k ± 0%       ~ (p=1.000 n=10) ¹
BackendBlockQueryRange/{resource.service.name=`loki-ingester`}_|_rate()/7-11   89.74k ± 0%   89.74k ± 0%       ~ (p=1.000 n=10) ¹
BackendBlockQueryRange/{status=error}_|_rate()/5-11                            16.93k ± 0%   16.93k ± 0%       ~ (p=1.000 n=10) ¹
BackendBlockQueryRange/{status=error}_|_rate()/7-11                            23.46k ± 0%   23.46k ± 0%       ~ (p=1.000 n=10) ¹
geomean                                                                        1.111M        1.111M       +0.00%
¹ all samples are equal

                                                                             │  before.txt  │             after.txt              │
                                                                             │   spans/s    │   spans/s    vs base               │
BackendBlockQueryRange/{}_|_rate()/5-11                                         6.006M ± 5%   5.779M ± 1%  -3.78% (p=0.043 n=10)
BackendBlockQueryRange/{}_|_rate()/7-11                                         8.512M ± 1%   8.458M ± 0%       ~ (p=0.353 n=10)
BackendBlockQueryRange/{}_|_rate()_by_(name)/5-11                               2.880M ± 3%   2.813M ± 5%       ~ (p=0.165 n=10)
BackendBlockQueryRange/{}_|_rate()_by_(name)/7-11                               3.870M ± 2%   3.765M ± 1%  -2.71% (p=0.001 n=10)
BackendBlockQueryRange/{}_|_rate()_by_(resource.service.name)/5-11              4.227M ± 8%   4.153M ± 7%       ~ (p=0.579 n=10)
BackendBlockQueryRange/{}_|_rate()_by_(resource.service.name)/7-11              5.740M ± 1%   5.595M ± 1%  -2.53% (p=0.000 n=10)
BackendBlockQueryRange/{}_|_rate()_by_(span.http.url)/5-11                      2.564M ± 2%   2.548M ± 1%       ~ (p=0.315 n=10)
BackendBlockQueryRange/{}_|_rate()_by_(span.http.url)/7-11                      3.642M ± 3%   3.585M ± 4%  -1.56% (p=0.009 n=10)
BackendBlockQueryRange/{resource.service.name=`loki-ingester`}_|_rate()/5-11   1004.5k ± 1%   993.3k ± 0%  -1.11% (p=0.015 n=10)
BackendBlockQueryRange/{resource.service.name=`loki-ingester`}_|_rate()/7-11    1.470M ± 5%   1.458M ± 0%  -0.77% (p=0.023 n=10)
BackendBlockQueryRange/{status=error}_|_rate()/5-11                             194.9k ± 0%   209.3k ± 0%  +7.41% (p=0.000 n=10)
BackendBlockQueryRange/{status=error}_|_rate()/7-11                             268.5k ± 6%   289.8k ± 0%  +7.92% (p=0.000 n=10)
geomean                                                                         2.154M        2.148M       -0.27%

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

@joe-elliott
this is garbage
b79f49d 
Signed-off-by: Joe Elliott <number101010@gmail.com>
@joe-elliott
filtery stuff
0fe8973 
Signed-off-by: Joe Elliott <number101010@gmail.com>
@joe-elliott
fix
168589f 
Signed-off-by: Joe Elliott <number101010@gmail.com>
@joe-elliott
max def everywhere
55854da 
Signed-off-by: Joe Elliott <number101010@gmail.com>
@joe-elliott
clean up benches
a99ae40 
Signed-off-by: Joe Elliott <number101010@gmail.com>
@joe-elliott
clean up
1b816d2 
Signed-off-by: Joe Elliott <number101010@gmail.com>
@joe-elliott
remove vendor chagnes
07a4e00 
Signed-off-by: Joe Elliott <number101010@gmail.com>
@joe-elliott joe-elliott mentioned this pull request Feb 25, 2025
Closed
3 tasks
stoewer
stoewer reviewed Feb 26, 2025
View reviewed changes
@stoewer
Copy link
Contributor

stoewer commented Feb 26, 2025

Looks good

but I still want to test a version that reverts to the unrolled version for higher max definition levels

Definitely interested to see the benchmark results for this approach 👍

@joe-elliott
changelog
ea572fc 
Signed-off-by: Joe Elliott <number101010@gmail.com>
@joe-elliott
add details about bench env vars
3a56ceb 
Signed-off-by: Joe Elliott <number101010@gmail.com>
@joe-elliott
Copy link
Collaborator Author

I spent quite a bit of time trying to find a breakpoint at which to run the unrolled RowNumber.Next without luck. I reran 10 more iterations of the benches on main and this branch and the final results are looking a lot better. These benches include the previous and show only improvements.

even more exhaustive benches! 
> benchstat before-all.txt after-all.txt
goos: darwin
goarch: arm64
pkg: github.com/grafana/tempo/tempodb/encoding/vparquet4
cpu: Apple M3 Pro
                                                                                                           │ before-all.txt │           after-all.txt            │
                                                                                                           │     sec/op     │   sec/op     vs base               │
FetchTagValues/tag:_span.http.url,_query:_{resource.namespace="tempo-ops"}-11                                   14.24m ± 1%   14.24m ± 1%       ~ (p=0.495 n=20)
FetchTagValues/tag:_span.component,_query:_{resource.namespace="tempo-ops"}-11                                  14.30m ± 1%   14.31m ± 2%       ~ (p=0.718 n=20)
FetchTagValues/tag:_span.http.url,_query:_{resource.namespace="tempo-ops"_&&_span.http.status_code=200}-11      14.38m ± 1%   14.42m ± 2%       ~ (p=0.820 n=20)
FetchTagValues/tag:_resource.namespace,_query:_{span.http.status_code=200}-11                                   547.7m ± 1%   497.3m ± 2%  -9.20% (p=0.000 n=20)
FetchTags/query:_{resource.namespace="tempo-ops"}_span-11                                                       14.36m ± 0%   14.39m ± 1%       ~ (p=1.000 n=20)
FetchTags/query:_{resource.namespace="tempo-ops"}_resource-11                                                   14.39m ± 0%   14.44m ± 2%       ~ (p=0.925 n=20)
FetchTags/query:_{resource.namespace="tempo-ops"}_none-11                                                       14.49m ± 1%   14.43m ± 2%       ~ (p=0.565 n=20)
FetchTags/query:_{resource.k8s.node.name>"h"}_span-11                                                            2.126 ± 1%    2.075 ± 1%  -2.41% (p=0.000 n=20)
FetchTags/query:_{resource.k8s.node.name>"h"}_resource-11                                                       943.7m ± 1%   912.0m ± 0%  -3.36% (p=0.000 n=20)
FetchTags/query:_{resource.k8s.node.name>"h"}_none-11                                                            8.758 ± 1%    8.560 ± 0%  -2.26% (p=0.000 n=20)
FetchTags/query:_{span.http.status_code=200}_span-11                                                             1.484 ± 1%    1.390 ± 0%  -6.32% (p=0.000 n=20)
FetchTags/query:_{span.http.status_code=200}_resource-11                                                        856.0m ± 1%   779.7m ± 0%  -8.92% (p=0.000 n=20)
FetchTags/query:_{span.http.status_code=200}_none-11                                                             4.582 ± 1%    4.340 ± 0%  -5.28% (p=0.000 n=20)
FetchTags/query:_{nestedSetParent=-1}_span-11                                                                    1.426 ± 2%    1.327 ± 0%  -6.94% (p=0.000 n=20)
FetchTags/query:_{nestedSetParent=-1}_resource-11                                                               945.7m ± 0%   875.1m ± 1%  -7.47% (p=0.000 n=20)
FetchTags/query:_{nestedSetParent=-1}_none-11                                                                    3.964 ± 1%    3.778 ± 0%  -4.70% (p=0.000 n=20)
FetchTags/query:_{rootName="Memcache.Put"}_span-11                                                              34.21m ± 1%   32.72m ± 0%  -4.37% (p=0.000 n=20)
FetchTags/query:_{rootName="Memcache.Put"}_resource-11                                                          14.93m ± 0%   14.46m ± 0%  -3.14% (p=0.000 n=20)
FetchTags/query:_{rootName="Memcache.Put"}_none-11                                                              149.2m ± 0%   146.3m ± 0%  -2.00% (p=0.000 n=20)
BackendBlockTraceQL/spanAttValMatch-11                                                                          84.32m ± 1%   80.60m ± 2%  -4.41% (p=0.000 n=20)
BackendBlockTraceQL/spanAttValNoMatch-11                                                                        5.576m ± 0%   5.495m ± 2%       ~ (p=0.121 n=20)
BackendBlockTraceQL/spanAttIntrinsicMatch-11                                                                    59.23m ± 0%   56.84m ± 1%  -4.04% (p=0.000 n=20)
BackendBlockTraceQL/spanAttIntrinsicNoMatch-11                                                                  5.476m ± 0%   5.367m ± 2%  -1.99% (p=0.001 n=20)
BackendBlockTraceQL/resourceAttValMatch-11                                                                      381.5m ± 0%   372.9m ± 2%  -2.24% (p=0.000 n=20)
BackendBlockTraceQL/resourceAttValNoMatch-11                                                                    5.198m ± 0%   5.107m ± 2%  -1.74% (p=0.000 n=20)
BackendBlockTraceQL/resourceAttIntrinsicMatch-11                                                                24.98m ± 0%   24.42m ± 0%  -2.24% (p=0.000 n=20)
BackendBlockTraceQL/resourceAttIntrinsicMatch#01-11                                                             5.181m ± 3%   5.060m ± 3%       ~ (p=0.341 n=20)
BackendBlockTraceQL/traceOrMatch-11                                                                             226.6m ± 0%   222.6m ± 0%  -1.77% (p=0.000 n=20)
BackendBlockTraceQL/traceOrNoMatch-11                                                                           226.5m ± 0%   222.7m ± 0%  -1.70% (p=0.000 n=20)
BackendBlockTraceQL/mixedValNoMatch-11                                                                          172.9m ± 0%   170.1m ± 0%  -1.61% (p=0.000 n=20)
BackendBlockTraceQL/mixedValMixedMatchAnd-11                                                                    5.038m ± 3%   4.940m ± 2%       ~ (p=0.091 n=20)
BackendBlockTraceQL/mixedValMixedMatchOr-11                                                                     132.8m ± 3%   135.7m ± 0%       ~ (p=0.779 n=20)
BackendBlockTraceQL/count-11                                                                                    298.2m ± 5%   306.1m ± 3%       ~ (p=0.583 n=20)
BackendBlockTraceQL/struct-11                                                                                   400.7m ± 4%   410.1m ± 0%       ~ (p=0.068 n=20)
BackendBlockTraceQL/||-11                                                                                       135.5m ± 3%   137.4m ± 0%       ~ (p=1.000 n=20)
BackendBlockTraceQL/mixed-11                                                                                    25.59m ± 3%   24.93m ± 1%  -2.60% (p=0.004 n=20)
BackendBlockTraceQL/complex-11                                                                                  5.007m ± 7%   5.019m ± 1%       ~ (p=0.341 n=20)
BackendBlockTraceQL/select-11                                                                                   5.034m ± 6%   4.997m ± 1%       ~ (p=0.968 n=20)
BackendBlockQueryRange/{}_|_rate()/5-11                                                                         852.7m ± 4%   848.8m ± 1%       ~ (p=0.862 n=20)
BackendBlockQueryRange/{}_|_rate()/7-11                                                                         869.7m ± 1%   860.1m ± 0%       ~ (p=0.086 n=20)
BackendBlockQueryRange/{}_|_rate()_by_(name)/5-11                                                                1.734 ± 3%    1.751 ± 2%       ~ (p=0.698 n=20)
BackendBlockQueryRange/{}_|_rate()_by_(name)/7-11                                                                1.914 ± 1%    1.918 ± 1%       ~ (p=0.640 n=20)
BackendBlockQueryRange/{}_|_rate()_by_(resource.service.name)/5-11                                               1.177 ± 2%    1.176 ± 1%       ~ (p=0.461 n=20)
BackendBlockQueryRange/{}_|_rate()_by_(resource.service.name)/7-11                                               1.299 ± 2%    1.288 ± 1%       ~ (p=0.758 n=20)
BackendBlockQueryRange/{}_|_rate()_by_(span.http.url)/5-11                                                       1.926 ± 1%    1.926 ± 0%       ~ (p=0.620 n=20)
BackendBlockQueryRange/{}_|_rate()_by_(span.http.url)/7-11                                                       2.019 ± 1%    2.017 ± 1%       ~ (p=0.583 n=20)
BackendBlockQueryRange/{resource.service.name=`loki-ingester`}_|_rate()/5-11                                    62.85m ± 3%   61.27m ± 0%       ~ (p=0.478 n=20)
BackendBlockQueryRange/{resource.service.name=`loki-ingester`}_|_rate()/7-11                                    63.16m ± 3%   61.54m ± 0%       ~ (p=0.265 n=20)
BackendBlockQueryRange/{status=error}_|_rate()/5-11                                                             87.58m ± 1%   81.17m ± 0%  -7.32% (p=0.000 n=20)
BackendBlockQueryRange/{status=error}_|_rate()/7-11                                                             87.55m ± 0%   81.01m ± 0%  -7.47% (p=0.000 n=20)
geomean                                                                                                         140.2m        137.1m       -2.22%

                                                                                                           │ before-all.txt │             after-all.txt              │
                                                                                                           │      B/op      │      B/op       vs base                │
FetchTagValues/tag:_span.http.url,_query:_{resource.namespace="tempo-ops"}-11                                 14.66Mi ±  1%    14.71Mi ±  1%        ~ (p=0.429 n=20)
FetchTagValues/tag:_span.component,_query:_{resource.namespace="tempo-ops"}-11                                14.62Mi ±  1%    14.71Mi ±  1%        ~ (p=0.565 n=20)
FetchTagValues/tag:_span.http.url,_query:_{resource.namespace="tempo-ops"_&&_span.http.status_code=200}-11    14.64Mi ±  1%    14.65Mi ±  1%        ~ (p=0.989 n=20)
FetchTagValues/tag:_resource.namespace,_query:_{span.http.status_code=200}-11                                 8.088Mi ±  9%    7.316Mi ±  7%   -9.55% (p=0.011 n=20)
FetchTags/query:_{resource.namespace="tempo-ops"}_span-11                                                     14.66Mi ±  1%    14.61Mi ±  1%        ~ (p=0.758 n=20)
FetchTags/query:_{resource.namespace="tempo-ops"}_resource-11                                                 14.69Mi ±  1%    14.70Mi ±  1%        ~ (p=0.841 n=20)
FetchTags/query:_{resource.namespace="tempo-ops"}_none-11                                                     14.73Mi ±  1%    14.69Mi ±  1%        ~ (p=0.565 n=20)
FetchTags/query:_{resource.k8s.node.name>"h"}_span-11                                                         25.69Mi ±  2%    25.71Mi ±  4%        ~ (p=0.678 n=20)
FetchTags/query:_{resource.k8s.node.name>"h"}_resource-11                                                     18.41Mi ±  3%    18.22Mi ±  2%        ~ (p=0.620 n=20)
FetchTags/query:_{resource.k8s.node.name>"h"}_none-11                                                         51.27Mi ± 16%    49.44Mi ± 23%        ~ (p=0.989 n=20)
FetchTags/query:_{span.http.status_code=200}_span-11                                                          12.91Mi ± 10%    13.03Mi ±  8%        ~ (p=0.327 n=20)
FetchTags/query:_{span.http.status_code=200}_resource-11                                                      7.014Mi ±  9%    7.163Mi ± 16%        ~ (p=0.114 n=20)
FetchTags/query:_{span.http.status_code=200}_none-11                                                          41.12Mi ± 21%    42.23Mi ± 11%        ~ (p=0.862 n=20)
FetchTags/query:_{nestedSetParent=-1}_span-11                                                                 11.40Mi ±  5%    12.69Mi ±  5%  +11.34% (p=0.043 n=20)
FetchTags/query:_{nestedSetParent=-1}_resource-11                                                             9.454Mi ± 13%    9.981Mi ±  9%        ~ (p=0.678 n=20)
FetchTags/query:_{nestedSetParent=-1}_none-11                                                                 40.52Mi ± 13%    39.45Mi ± 13%        ~ (p=0.602 n=20)
FetchTags/query:_{rootName="Memcache.Put"}_span-11                                                            7.226Mi ±  2%    7.119Mi ±  1%   -1.48% (p=0.040 n=20)
FetchTags/query:_{rootName="Memcache.Put"}_resource-11                                                        6.729Mi ±  1%    6.747Mi ±  1%        ~ (p=0.529 n=20)
FetchTags/query:_{rootName="Memcache.Put"}_none-11                                                            15.76Mi ±  8%    14.25Mi ± 12%        ~ (p=0.231 n=20)
BackendBlockTraceQL/spanAttValMatch-11                                                                        45.15Mi ±  0%    45.24Mi ±  0%        ~ (p=0.231 n=20)
BackendBlockTraceQL/spanAttValNoMatch-11                                                                      6.738Mi ±  1%    6.803Mi ±  1%   +0.98% (p=0.017 n=20)
BackendBlockTraceQL/spanAttIntrinsicMatch-11                                                                  39.81Mi ±  0%    39.60Mi ±  0%   -0.52% (p=0.001 n=20)
BackendBlockTraceQL/spanAttIntrinsicNoMatch-11                                                                7.069Mi ±  1%    7.108Mi ±  1%        ~ (p=0.678 n=20)
BackendBlockTraceQL/resourceAttValMatch-11                                                                    576.0Mi ±  0%    576.0Mi ±  0%        ~ (p=0.678 n=20)
BackendBlockTraceQL/resourceAttValNoMatch-11                                                                  5.501Mi ±  1%    5.427Mi ±  1%   -1.35% (p=0.040 n=20)
BackendBlockTraceQL/resourceAttIntrinsicMatch-11                                                              10.49Mi ±  1%    10.44Mi ±  1%        ~ (p=0.076 n=20)
BackendBlockTraceQL/resourceAttIntrinsicMatch#01-11                                                           6.837Mi ±  0%    6.859Mi ±  1%        ~ (p=0.231 n=20)
BackendBlockTraceQL/traceOrMatch-11                                                                           9.915Mi ±  9%    9.980Mi ±  7%        ~ (p=0.495 n=20)
BackendBlockTraceQL/traceOrNoMatch-11                                                                         9.226Mi ±  7%   10.326Mi ±  6%  +11.92% (p=0.030 n=20)
BackendBlockTraceQL/mixedValNoMatch-11                                                                        8.236Mi ±  6%    8.031Mi ±  3%        ~ (p=0.253 n=20)
BackendBlockTraceQL/mixedValMixedMatchAnd-11                                                                  5.906Mi ±  1%    5.918Mi ±  1%        ~ (p=0.841 n=20)
BackendBlockTraceQL/mixedValMixedMatchOr-11                                                                   7.919Mi ±  4%    8.027Mi ±  5%        ~ (p=0.265 n=20)
BackendBlockTraceQL/count-11                                                                                  418.0Mi ±  0%    418.4Mi ±  0%        ~ (p=1.000 n=20)
BackendBlockTraceQL/struct-11                                                                                 13.43Mi ± 10%    13.79Mi ±  7%        ~ (p=0.547 n=20)
BackendBlockTraceQL/||-11                                                                                     16.68Mi ±  1%    16.53Mi ±  1%        ~ (p=0.192 n=20)
BackendBlockTraceQL/mixed-11                                                                                  6.957Mi ±  1%    6.921Mi ±  1%        ~ (p=0.242 n=20)
BackendBlockTraceQL/complex-11                                                                                6.052Mi ±  4%    5.802Mi ±  2%   -4.13% (p=0.003 n=20)
BackendBlockTraceQL/select-11                                                                                 5.883Mi ±  2%    5.743Mi ±  1%   -2.39% (p=0.000 n=20)
BackendBlockQueryRange/{}_|_rate()/5-11                                                                       28.30Mi ±  5%    28.40Mi ±  8%        ~ (p=0.989 n=20)
BackendBlockQueryRange/{}_|_rate()/7-11                                                                       28.96Mi ±  8%    28.68Mi ±  3%        ~ (p=0.799 n=20)
BackendBlockQueryRange/{}_|_rate()_by_(name)/5-11                                                             32.83Mi ± 16%    31.69Mi ± 15%        ~ (p=0.883 n=20)
BackendBlockQueryRange/{}_|_rate()_by_(name)/7-11                                                             32.17Mi ± 10%    31.82Mi ±  9%        ~ (p=0.547 n=20)
BackendBlockQueryRange/{}_|_rate()_by_(resource.service.name)/5-11                                            31.94Mi ±  9%    29.64Mi ±  5%   -7.20% (p=0.035 n=20)
BackendBlockQueryRange/{}_|_rate()_by_(resource.service.name)/7-11                                            30.16Mi ± 18%    29.32Mi ± 13%        ~ (p=0.904 n=20)
BackendBlockQueryRange/{}_|_rate()_by_(span.http.url)/5-11                                                    355.5Mi ±  2%    355.5Mi ±  3%        ~ (p=0.355 n=20)
BackendBlockQueryRange/{}_|_rate()_by_(span.http.url)/7-11                                                    388.7Mi ±  1%    381.3Mi ±  1%   -1.89% (p=0.004 n=20)
BackendBlockQueryRange/{resource.service.name=`loki-ingester`}_|_rate()/5-11                                  23.37Mi ±  2%    23.18Mi ±  1%        ~ (p=0.081 n=20)
BackendBlockQueryRange/{resource.service.name=`loki-ingester`}_|_rate()/7-11                                  23.32Mi ±  2%    23.34Mi ±  2%        ~ (p=0.678 n=20)
BackendBlockQueryRange/{status=error}_|_rate()/5-11                                                           6.418Mi ±  3%    6.692Mi ±  2%   +4.27% (p=0.003 n=20)
BackendBlockQueryRange/{status=error}_|_rate()/7-11                                                           6.462Mi ±  1%    6.591Mi ±  3%        ~ (p=0.086 n=20)
geomean                                                                                                       18.20Mi          18.15Mi         -0.26%

                                                                                                           │ before-all.txt │           after-all.txt            │
                                                                                                           │   allocs/op    │  allocs/op   vs base               │
FetchTagValues/tag:_span.http.url,_query:_{resource.namespace="tempo-ops"}-11                                   81.37k ± 0%   81.37k ± 0%       ~ (p=0.425 n=20)
FetchTagValues/tag:_span.component,_query:_{resource.namespace="tempo-ops"}-11                                  81.37k ± 0%   81.37k ± 0%       ~ (p=0.722 n=20)
FetchTagValues/tag:_span.http.url,_query:_{resource.namespace="tempo-ops"_&&_span.http.status_code=200}-11      81.39k ± 0%   81.39k ± 0%       ~ (p=0.654 n=20)
FetchTagValues/tag:_resource.namespace,_query:_{span.http.status_code=200}-11                                   89.72k ± 0%   89.68k ± 0%  -0.05% (p=0.000 n=20)
FetchTags/query:_{resource.namespace="tempo-ops"}_span-11                                                       81.39k ± 0%   81.39k ± 0%       ~ (p=0.693 n=20)
FetchTags/query:_{resource.namespace="tempo-ops"}_resource-11                                                   81.39k ± 0%   81.39k ± 0%       ~ (p=0.963 n=20)
FetchTags/query:_{resource.namespace="tempo-ops"}_none-11                                                       81.52k ± 0%   81.51k ± 0%       ~ (p=0.498 n=20)
FetchTags/query:_{resource.k8s.node.name>"h"}_span-11                                                           165.7k ± 0%   165.7k ± 0%       ~ (p=0.606 n=20)
FetchTags/query:_{resource.k8s.node.name>"h"}_resource-11                                                       89.14k ± 0%   89.14k ± 0%       ~ (p=0.310 n=20)
FetchTags/query:_{resource.k8s.node.name>"h"}_none-11                                                           258.9k ± 0%   258.9k ± 0%       ~ (p=0.136 n=20)
FetchTags/query:_{span.http.status_code=200}_span-11                                                            174.8k ± 0%   174.8k ± 0%       ~ (p=0.234 n=20)
FetchTags/query:_{span.http.status_code=200}_resource-11                                                        92.11k ± 0%   92.13k ± 0%  +0.02% (p=0.006 n=20)
FetchTags/query:_{span.http.status_code=200}_none-11                                                            314.5k ± 0%   314.5k ± 0%       ~ (p=0.713 n=20)
FetchTags/query:_{nestedSetParent=-1}_span-11                                                                   143.3k ± 0%   143.3k ± 0%  +0.01% (p=0.044 n=20)
FetchTags/query:_{nestedSetParent=-1}_resource-11                                                               98.18k ± 0%   98.18k ± 0%       ~ (p=0.857 n=20)
FetchTags/query:_{nestedSetParent=-1}_none-11                                                                   301.7k ± 0%   301.7k ± 0%       ~ (p=0.147 n=20)
FetchTags/query:_{rootName="Memcache.Put"}_span-11                                                              82.35k ± 0%   82.34k ± 0%       ~ (p=0.165 n=20)
FetchTags/query:_{rootName="Memcache.Put"}_resource-11                                                          81.79k ± 0%   81.79k ± 0%       ~ (p=0.965 n=20)
FetchTags/query:_{rootName="Memcache.Put"}_none-11                                                              91.07k ± 0%   91.06k ± 0%       ~ (p=0.323 n=20)
BackendBlockTraceQL/spanAttValMatch-11                                                                          503.5k ± 0%   503.5k ± 0%       ~ (p=0.167 n=20)
BackendBlockTraceQL/spanAttValNoMatch-11                                                                        79.48k ± 0%   79.48k ± 0%  +0.00% (p=0.040 n=20)
BackendBlockTraceQL/spanAttIntrinsicMatch-11                                                                    288.1k ± 0%   288.1k ± 0%       ~ (p=0.106 n=20)
BackendBlockTraceQL/spanAttIntrinsicNoMatch-11                                                                  79.44k ± 0%   79.44k ± 0%       ~ (p=0.214 n=20)
BackendBlockTraceQL/resourceAttValMatch-11                                                                      3.450M ± 0%   3.450M ± 0%       ~ (p=0.644 n=20)
BackendBlockTraceQL/resourceAttValNoMatch-11                                                                    79.48k ± 0%   79.48k ± 0%       ~ (p=0.098 n=20)
BackendBlockTraceQL/resourceAttIntrinsicMatch-11                                                                126.1k ± 0%   126.1k ± 0%  -0.00% (p=0.028 n=20)
BackendBlockTraceQL/resourceAttIntrinsicMatch#01-11                                                             79.47k ± 0%   79.47k ± 0%       ~ (p=0.378 n=20)
BackendBlockTraceQL/traceOrMatch-11                                                                             86.47k ± 0%   86.41k ± 1%       ~ (p=0.984 n=20)
BackendBlockTraceQL/traceOrNoMatch-11                                                                           86.35k ± 0%   86.49k ± 0%  +0.16% (p=0.000 n=20)
BackendBlockTraceQL/mixedValNoMatch-11                                                                          80.31k ± 0%   80.30k ± 0%       ~ (p=0.402 n=20)
BackendBlockTraceQL/mixedValMixedMatchAnd-11                                                                    79.48k ± 0%   79.48k ± 0%       ~ (p=0.845 n=20)
BackendBlockTraceQL/mixedValMixedMatchOr-11                                                                     80.28k ± 0%   80.28k ± 0%       ~ (p=0.743 n=20)
BackendBlockTraceQL/count-11                                                                                    1.990M ± 0%   1.990M ± 0%       ~ (p=0.185 n=20)
BackendBlockTraceQL/struct-11                                                                                   93.67k ± 1%   93.70k ± 0%       ~ (p=0.925 n=20)
BackendBlockTraceQL/||-11                                                                                       161.9k ± 0%   161.9k ± 0%       ~ (p=0.525 n=20)
BackendBlockTraceQL/mixed-11                                                                                    87.96k ± 0%   87.96k ± 0%       ~ (p=0.813 n=20)
BackendBlockTraceQL/complex-11                                                                                  79.84k ± 0%   79.84k ± 0%  -0.01% (p=0.004 n=20)
BackendBlockTraceQL/select-11                                                                                   79.72k ± 0%   79.71k ± 0%  -0.01% (p=0.000 n=20)
BackendBlockQueryRange/{}_|_rate()/5-11                                                                         82.47k ± 8%   86.89k ± 5%       ~ (p=0.449 n=20)
BackendBlockQueryRange/{}_|_rate()/7-11                                                                         89.49k ± 8%   82.28k ± 9%       ~ (p=0.059 n=20)
BackendBlockQueryRange/{}_|_rate()_by_(name)/5-11                                                               90.81k ± 0%   90.79k ± 0%       ~ (p=0.723 n=20)
BackendBlockQueryRange/{}_|_rate()_by_(name)/7-11                                                               92.14k ± 0%   92.12k ± 0%       ~ (p=0.331 n=20)
BackendBlockQueryRange/{}_|_rate()_by_(resource.service.name)/5-11                                              83.82k ± 0%   83.80k ± 0%       ~ (p=0.119 n=20)
BackendBlockQueryRange/{}_|_rate()_by_(resource.service.name)/7-11                                              83.98k ± 0%   83.99k ± 0%       ~ (p=0.372 n=20)
BackendBlockQueryRange/{}_|_rate()_by_(span.http.url)/5-11                                                      554.4k ± 4%   556.6k ± 2%       ~ (p=0.142 n=20)
BackendBlockQueryRange/{}_|_rate()_by_(span.http.url)/7-11                                                      684.1k ± 0%   682.8k ± 3%       ~ (p=0.142 n=20)
BackendBlockQueryRange/{resource.service.name=`loki-ingester`}_|_rate()/5-11                                    156.0k ± 0%   155.9k ± 0%       ~ (p=0.213 n=20)
BackendBlockQueryRange/{resource.service.name=`loki-ingester`}_|_rate()/7-11                                    155.9k ± 0%   155.9k ± 0%       ~ (p=0.952 n=20)
BackendBlockQueryRange/{status=error}_|_rate()/5-11                                                             88.97k ± 0%   88.98k ± 0%       ~ (p=0.193 n=20)
BackendBlockQueryRange/{status=error}_|_rate()/7-11                                                             88.97k ± 0%   88.97k ± 0%       ~ (p=0.973 n=20)
geomean                                                                                                         130.8k        130.7k       -0.06%

                                                    │ before-all.txt │            after-all.txt            │
                                                    │      B/s       │     B/s       vs base               │
BackendBlockTraceQL/spanAttValMatch-11                  265.6Mi ± 1%   277.8Mi ± 2%  +4.62% (p=0.000 n=20)
BackendBlockTraceQL/spanAttValNoMatch-11                303.0Mi ± 0%   307.5Mi ± 2%       ~ (p=0.116 n=20)
BackendBlockTraceQL/spanAttIntrinsicMatch-11            393.3Mi ± 0%   409.8Mi ± 1%  +4.21% (p=0.000 n=20)
BackendBlockTraceQL/spanAttIntrinsicNoMatch-11          452.4Mi ± 0%   461.6Mi ± 2%  +2.04% (p=0.001 n=20)
BackendBlockTraceQL/resourceAttValMatch-11              57.61Mi ± 0%   58.93Mi ± 1%  +2.30% (p=0.000 n=20)
BackendBlockTraceQL/resourceAttValNoMatch-11            173.1Mi ± 0%   176.1Mi ± 2%  +1.78% (p=0.000 n=20)
BackendBlockTraceQL/resourceAttIntrinsicMatch-11        881.4Mi ± 0%   901.6Mi ± 0%  +2.29% (p=0.000 n=20)
BackendBlockTraceQL/resourceAttIntrinsicMatch#01-11     180.2Mi ± 3%   184.5Mi ± 3%       ~ (p=0.337 n=20)
BackendBlockTraceQL/traceOrMatch-11                     7.372Mi ± 0%   7.510Mi ± 0%  +1.88% (p=0.000 n=20)
BackendBlockTraceQL/traceOrNoMatch-11                   7.381Mi ± 0%   7.510Mi ± 0%  +1.74% (p=0.000 n=20)
BackendBlockTraceQL/mixedValNoMatch-11                  11.40Mi ± 0%   11.59Mi ± 0%  +1.63% (p=0.000 n=20)
BackendBlockTraceQL/mixedValMixedMatchAnd-11            177.2Mi ± 3%   180.7Mi ± 2%       ~ (p=0.087 n=20)
BackendBlockTraceQL/mixedValMixedMatchOr-11             20.79Mi ± 3%   20.32Mi ± 0%       ~ (p=0.773 n=20)
BackendBlockTraceQL/count-11                            73.74Mi ± 5%   71.77Mi ± 3%       ~ (p=0.579 n=20)
BackendBlockTraceQL/struct-11                           13.63Mi ± 4%   13.31Mi ± 0%       ~ (p=0.066 n=20)
BackendBlockTraceQL/||-11                               163.0Mi ± 3%   160.6Mi ± 0%       ~ (p=1.000 n=20)
BackendBlockTraceQL/mixed-11                            834.9Mi ± 3%   857.1Mi ± 1%  +2.67% (p=0.004 n=20)
BackendBlockTraceQL/complex-11                          179.8Mi ± 7%   179.3Mi ± 1%       ~ (p=0.341 n=20)
BackendBlockTraceQL/select-11                           178.8Mi ± 7%   180.1Mi ± 1%       ~ (p=0.968 n=20)
geomean                                                 104.4Mi        105.7Mi       +1.18%

                                                    │ before-all.txt │            after-all.txt             │
                                                    │    MB_io/op    │  MB_io/op    vs base                 │
BackendBlockTraceQL/spanAttValMatch-11                    23.48 ± 0%    23.48 ± 0%       ~ (p=1.000 n=20) ¹
BackendBlockTraceQL/spanAttValNoMatch-11                  1.772 ± 0%    1.772 ± 0%       ~ (p=1.000 n=20) ¹
BackendBlockTraceQL/spanAttIntrinsicMatch-11              24.43 ± 0%    24.43 ± 0%       ~ (p=1.000 n=20) ¹
BackendBlockTraceQL/spanAttIntrinsicNoMatch-11            2.598 ± 0%    2.598 ± 0%       ~ (p=1.000 n=20) ¹
BackendBlockTraceQL/resourceAttValMatch-11                23.04 ± 0%    23.04 ± 0%       ~ (p=1.000 n=20) ¹
BackendBlockTraceQL/resourceAttValNoMatch-11             943.2m ± 0%   943.2m ± 0%       ~ (p=1.000 n=20) ¹
BackendBlockTraceQL/resourceAttIntrinsicMatch-11          23.09 ± 0%    23.09 ± 0%       ~ (p=1.000 n=20) ¹
BackendBlockTraceQL/resourceAttIntrinsicMatch#01-11      979.0m ± 0%   979.0m ± 0%       ~ (p=1.000 n=20) ¹
BackendBlockTraceQL/traceOrMatch-11                       1.753 ± 0%    1.753 ± 0%       ~ (p=1.000 n=20) ¹
BackendBlockTraceQL/traceOrNoMatch-11                     1.753 ± 0%    1.753 ± 0%       ~ (p=1.000 n=20) ¹
BackendBlockTraceQL/mixedValNoMatch-11                    2.067 ± 0%    2.067 ± 0%       ~ (p=1.000 n=20) ¹
BackendBlockTraceQL/mixedValMixedMatchAnd-11             936.1m ± 0%   936.1m ± 0%       ~ (p=1.000 n=20) ¹
BackendBlockTraceQL/mixedValMixedMatchOr-11               2.893 ± 0%    2.893 ± 0%       ~ (p=1.000 n=20) ¹
BackendBlockTraceQL/count-11                              23.03 ± 0%    23.03 ± 0%       ~ (p=1.000 n=20) ¹
BackendBlockTraceQL/struct-11                             5.726 ± 0%    5.726 ± 0%       ~ (p=1.000 n=20) ¹
BackendBlockTraceQL/||-11                                 23.14 ± 0%    23.14 ± 0%       ~ (p=1.000 n=20) ¹
BackendBlockTraceQL/mixed-11                              22.40 ± 0%    22.40 ± 0%       ~ (p=1.000 n=20) ¹
BackendBlockTraceQL/complex-11                           943.7m ± 0%   943.7m ± 0%       ~ (p=1.000 n=20) ¹
BackendBlockTraceQL/select-11                            943.7m ± 0%   943.7m ± 0%       ~ (p=1.000 n=20) ¹
geomean                                                   4.351         4.351       +0.00%
¹ all samples are equal

                                                                             │ before-all.txt │            after-all.txt            │
                                                                             │    MB_IO/op    │  MB_IO/op   vs base                 │
BackendBlockQueryRange/{}_|_rate()/5-11                                            26.19 ± 0%   26.19 ± 0%       ~ (p=1.000 n=20) ¹
BackendBlockQueryRange/{}_|_rate()/7-11                                            26.19 ± 0%   26.19 ± 0%       ~ (p=1.000 n=20) ¹
BackendBlockQueryRange/{}_|_rate()_by_(name)/5-11                                  34.40 ± 0%   34.40 ± 0%       ~ (p=1.000 n=20) ¹
BackendBlockQueryRange/{}_|_rate()_by_(name)/7-11                                  34.40 ± 0%   34.40 ± 0%       ~ (p=1.000 n=20) ¹
BackendBlockQueryRange/{}_|_rate()_by_(resource.service.name)/5-11                 26.51 ± 0%   26.51 ± 0%       ~ (p=1.000 n=20) ¹
BackendBlockQueryRange/{}_|_rate()_by_(resource.service.name)/7-11                 26.51 ± 0%   26.51 ± 0%       ~ (p=1.000 n=20) ¹
BackendBlockQueryRange/{}_|_rate()_by_(span.http.url)/5-11                         29.01 ± 0%   29.01 ± 0%       ~ (p=1.000 n=20) ¹
BackendBlockQueryRange/{}_|_rate()_by_(span.http.url)/7-11                         29.01 ± 0%   29.01 ± 0%       ~ (p=1.000 n=20) ¹
BackendBlockQueryRange/{resource.service.name=`loki-ingester`}_|_rate()/5-11       26.43 ± 0%   26.43 ± 0%       ~ (p=1.000 n=20) ¹
BackendBlockQueryRange/{resource.service.name=`loki-ingester`}_|_rate()/7-11       26.43 ± 0%   26.43 ± 0%       ~ (p=1.000 n=20) ¹
BackendBlockQueryRange/{status=error}_|_rate()/5-11                                27.55 ± 0%   27.55 ± 0%       ~ (p=1.000 n=20) ¹
BackendBlockQueryRange/{status=error}_|_rate()/7-11                                27.55 ± 0%   27.55 ± 0%       ~ (p=1.000 n=20) ¹
geomean                                                                            28.21        28.21       +0.00%
¹ all samples are equal

                                                                             │ before-all.txt │            after-all.txt             │
                                                                             │    spans/op    │  spans/op    vs base                 │
BackendBlockQueryRange/{}_|_rate()/5-11                                           4.916M ± 0%   4.916M ± 0%       ~ (p=1.000 n=20) ¹
BackendBlockQueryRange/{}_|_rate()/7-11                                           7.271M ± 0%   7.271M ± 0%       ~ (p=1.000 n=20) ¹
BackendBlockQueryRange/{}_|_rate()_by_(name)/5-11                                 4.916M ± 0%   4.916M ± 0%       ~ (p=1.000 n=20) ¹
BackendBlockQueryRange/{}_|_rate()_by_(name)/7-11                                 7.271M ± 0%   7.271M ± 0%       ~ (p=1.000 n=20) ¹
BackendBlockQueryRange/{}_|_rate()_by_(resource.service.name)/5-11                4.916M ± 0%   4.916M ± 0%       ~ (p=1.000 n=20) ¹
BackendBlockQueryRange/{}_|_rate()_by_(resource.service.name)/7-11                7.271M ± 0%   7.271M ± 0%       ~ (p=1.000 n=20) ¹
BackendBlockQueryRange/{}_|_rate()_by_(span.http.url)/5-11                        4.916M ± 0%   4.916M ± 0%       ~ (p=1.000 n=20) ¹
BackendBlockQueryRange/{}_|_rate()_by_(span.http.url)/7-11                        7.271M ± 0%   7.271M ± 0%       ~ (p=1.000 n=20) ¹
BackendBlockQueryRange/{resource.service.name=`loki-ingester`}_|_rate()/5-11      60.97k ± 0%   60.97k ± 0%       ~ (p=1.000 n=20) ¹
BackendBlockQueryRange/{resource.service.name=`loki-ingester`}_|_rate()/7-11      89.74k ± 0%   89.74k ± 0%       ~ (p=1.000 n=20) ¹
BackendBlockQueryRange/{status=error}_|_rate()/5-11                               16.93k ± 0%   16.93k ± 0%       ~ (p=1.000 n=20) ¹
BackendBlockQueryRange/{status=error}_|_rate()/7-11                               23.46k ± 0%   23.46k ± 0%       ~ (p=1.000 n=20) ¹
geomean                                                                           1.111M        1.111M       +0.00%
¹ all samples are equal

                                                                             │ before-all.txt │           after-all.txt            │
                                                                             │    spans/s     │   spans/s    vs base               │
BackendBlockQueryRange/{}_|_rate()/5-11                                           5.765M ± 4%   5.791M ± 1%       ~ (p=0.862 n=20)
BackendBlockQueryRange/{}_|_rate()/7-11                                           8.360M ± 1%   8.453M ± 0%       ~ (p=0.086 n=20)
BackendBlockQueryRange/{}_|_rate()_by_(name)/5-11                                 2.834M ± 3%   2.807M ± 2%       ~ (p=0.698 n=20)
BackendBlockQueryRange/{}_|_rate()_by_(name)/7-11                                 3.799M ± 1%   3.791M ± 1%       ~ (p=0.640 n=20)
BackendBlockQueryRange/{}_|_rate()_by_(resource.service.name)/5-11                4.175M ± 2%   4.180M ± 1%       ~ (p=0.461 n=20)
BackendBlockQueryRange/{}_|_rate()_by_(resource.service.name)/7-11                5.599M ± 2%   5.645M ± 1%       ~ (p=0.758 n=20)
BackendBlockQueryRange/{}_|_rate()_by_(span.http.url)/5-11                        2.552M ± 1%   2.552M ± 0%       ~ (p=0.620 n=20)
BackendBlockQueryRange/{}_|_rate()_by_(span.http.url)/7-11                        3.602M ± 1%   3.605M ± 1%       ~ (p=0.583 n=20)
BackendBlockQueryRange/{resource.service.name=`loki-ingester`}_|_rate()/5-11      970.1k ± 4%   995.1k ± 0%       ~ (p=0.478 n=20)
BackendBlockQueryRange/{resource.service.name=`loki-ingester`}_|_rate()/7-11      1.421M ± 3%   1.458M ± 0%       ~ (p=0.265 n=20)
BackendBlockQueryRange/{status=error}_|_rate()/5-11                               193.3k ± 1%   208.6k ± 0%  +7.90% (p=0.000 n=20)
BackendBlockQueryRange/{status=error}_|_rate()/7-11                               268.0k ± 0%   289.6k ± 0%  +8.08% (p=0.000 n=20)
geomean                                                                           2.114M        2.153M       +1.84%

@joe-elliott joe-elliott requested a review from stoewer February 27, 2025 18:43
@joe-elliott joe-elliott merged commit c1f6280 into grafana:main Feb 28, 2025
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@stoewer stoewer stoewer approved these changes

@mdisibio mdisibio Awaiting requested review from mdisibio mdisibio is a code owner

@mapno mapno Awaiting requested review from mapno mapno is a code owner

@yvrhdn yvrhdn Awaiting requested review from yvrhdn yvrhdn is a code owner

@zalegrala zalegrala Awaiting requested review from zalegrala zalegrala is a code owner

@electron0zero electron0zero Awaiting requested review from electron0zero electron0zero is a code owner

@ie-pham ie-pham Awaiting requested review from ie-pham ie-pham is a code owner

@javiermolinar javiermolinar Awaiting requested review from javiermolinar javiermolinar is a code owner

@carles-grafana carles-grafana Awaiting requested review from carles-grafana carles-grafana is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@joe-elliott @stoewer