lib/mergeset: reduce memory allocations on blockcache misses #9324
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
TODO: add benchmark |
534767c to
7892b18
Compare
makasim
reviewed
Jun 30, 2025
rtm0
reviewed
Jul 1, 2025
7892b18 to
66202f7
Compare
This commit adds tmp inmemory and data blocks buffers for index search requests. It allows to reduce memory allocations on block cache misses. Since block cache puts block into cache only on after configured number of cache misses. Signed-off-by: f41gh7 <nik@victoriametrics.com>
66202f7 to
4fdc984
Compare
Signed-off-by: f41gh7 <nik@victoriametrics.com>
hagen1778
reviewed
Jul 17, 2025
Co-authored-by: Roman Khavronenko <roman@victoriametrics.com>
hagen1778
reviewed
Jul 17, 2025
| @@ -276,7 +289,9 @@ func (ps *partSearch) nextBHS() error { | |||
| return fmt.Errorf("cannot read index block: %w", err) | |||
| } | |||
| b = idxb | |||
| idxbCache.PutBlock(idxbKey, b) | |||
| if idxbCache.TryPutBlock(idxbKey, b) { | |||
There was a problem hiding this comment.
please add a comment to explain why we stop referring the block
lib/mergeset/part_search.go
Outdated
Comment on lines
42
to
43
| tmpIB *inmemoryBlock | ||
| tmpIdB *indexBlock |
There was a problem hiding this comment.
please add a comment explaining how tmp blocks are used
Signed-off-by: f41gh7 <nik@victoriametrics.com>
makasim
reviewed
Jul 18, 2025
makasim
reviewed
Jul 18, 2025
Co-authored-by: Max Kotliar <mkotlyar@victoriametrics.com>
f41gh7
added a commit
that referenced
this pull request
Jul 18, 2025
This commit adds tmp inmemory and data blocks buffers for index search requests. It allows to reduce memory allocations on block cache misses. Since block cache puts block into cache only on after configured number of cache misses. Related PR #9324
|
The enhancement has been released in v1.122.0 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This commit adds tmp inmemory and data blocks buffers for
index search requests. It allows to reduce memory allocations on block cache misses. Since block cache puts block into cache only on after configured number of cache misses.
Describe Your Changes
Please provide a brief description of the changes you made. Be as specific as possible to help others understand the purpose and impact of your modifications.
Checklist
The following checks are mandatory: