Skip to content

[PERF] TSDB: Chunk encoding: shorten some write sequences #14932

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 4 commits into from
Sep 20, 2024
Merged

[PERF] TSDB: Chunk encoding: shorten some write sequences #14932

merged 4 commits into from
Sep 20, 2024

Conversation

bboreham
Copy link
Member

First, I improved the benchmark:

  • Benchmarks must do the same work N times.
  • Run 3 cases, where the values are constant, vary a bit, and vary a lot.
  • Also aim for 120 samples per chubk, same as TSDB default.

Then, simplify writeByte - rather than append a zero then set the value at that position, append the value.

Finally, combine writes in timestamps - instead of a 2-bit write followed by a 14-bit write, do two 8-bit writes, which goes much faster since it avoids looping.
In the most common case timestamp delta-of-deltas is zero, but this impacts the next-most-common case where it is nonzero and fits in 14 bits.

goos: darwin
goarch: arm64
pkg: github.com/prometheus/prometheus/tsdb/chunkenc
cpu: Apple M2
                           │ before.txt  │              after.txt              │
                           │   sec/op    │    sec/op     vs base               │
XORAppender/constant-8       737.2n ± 4%   737.5n ±  0%        ~ (p=1.000 n=6)
XORAppender/random_steps-8   3.806µ ± 9%   2.063µ ± 13%  -45.80% (p=0.002 n=6)
XORAppender/random_0-1-8     5.048µ ± 3%   3.145µ ±  8%  -37.69% (p=0.002 n=6)
geomean                      2.419µ        1.685µ        -30.35%

                           │  before.txt  │              after.txt               │
                           │     B/op     │     B/op      vs base                │
XORAppender/constant-8         320.0 ± 0%     320.0 ± 0%       ~ (p=1.000 n=6) ¹
XORAppender/random_steps-8   1.062Ki ± 0%   1.062Ki ± 0%       ~ (p=1.000 n=6) ¹
XORAppender/random_0-1-8     1.938Ki ± 0%   1.938Ki ± 0%       ~ (p=1.000 n=6) ¹
geomean                        884.0          884.0       +0.00%
¹ all samples are equal

                           │ before.txt │             after.txt              │
                           │ allocs/op  │ allocs/op   vs base                │
XORAppender/constant-8       4.000 ± 0%   4.000 ± 0%       ~ (p=1.000 n=6) ¹
XORAppender/random_steps-8   6.000 ± 0%   6.000 ± 0%       ~ (p=1.000 n=6) ¹
XORAppender/random_0-1-8     7.000 ± 0%   7.000 ± 0%       ~ (p=1.000 n=6) ¹
geomean                      5.518        5.518       +0.00%
¹ all samples are equal

@bboreham
TSDB: Improve xor-chunk benchmarks
b65f1b6 
Benchmarks must do the same work N times.
Run 3 cases, where the values are constant, vary a bit, and vary a lot.

Also aim for 120 samples same as TSDB default.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
@bboreham
[PERF] Chunk encoding: simplify writeByte
b9a9689 
Rather than append a zero then set the value at that position, append the value.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
@bboreham
[PERF] Chunk encoding: combine timestamp writes
648a668 
Instead of a 2-bit write followed by a 14-bit write, do two 8-bit
writes, which goes much faster since it avoids looping.

Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
@bboreham bboreham mentioned this pull request Sep 18, 2024
Merged
beorn7
beorn7 previously approved these changes Sep 18, 2024
View reviewed changes
Copy link
Member

@beorn7 beorn7 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you very much.

@beorn7
Copy link
Member

beorn7 commented Sep 18, 2024

Linter seems to complain about spurious whitespace at the end?

@beorn7
Copy link
Member

beorn7 commented Sep 18, 2024

I see, I guess it doesn't like the empty line 311, now that only closing braces are left.

@bboreham
lint
e8c2d91 
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
@bboreham bboreham mentioned this pull request Sep 18, 2024
Open
@bboreham bboreham merged commit d42232e into prometheus:main Sep 20, 2024
26 checks passed
@bboreham bboreham deleted the chunk-xor-combine-writebits branch September 20, 2024 16:53
julienduchesne pushed a commit to julienduchesne/prometheus that referenced this pull request Dec 13, 2024
@aknuds1
Merge pull request prometheus#14932 from bboreham/chunk-xor-combine-w…
0f05f2b 
…ritebits
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@beorn7 beorn7 beorn7 approved these changes

@jesusvazquez jesusvazquez Awaiting requested review from jesusvazquez jesusvazquez is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@bboreham @beorn7