[Bifrost] Introduce read batch limit in configuration #3392
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
e654a14 to
5517b90
Compare
muhamadazmy
reviewed
Jun 17, 2025
|
This will not be part of v1.4.0 until we test it under various load conditions to tune the default values. |
tillrohrmann
approved these changes
Jul 1, 2025
There was a problem hiding this comment.
Thanks for exposing this config option @AhmedSoliman. Curious to learn what's the impact of different read batch sizes on throughput/latency and overall memory consumption. LGTM. +1 for merging :-)
The new configuration option `bifrost.replicated-loglet.read-batch-size` will set (in bytes) the budget for every read batch request bifrost client will make to a log-server. The log-server will send at-least a single record even if it exceeds this budget. The option is set by default to 32KiB. Additionally: - Default value of `bifrost.replicated-loglet.readahead-records` is reduced from 100 to 20 to reduce the potential memory ballooning during backfilling. As consequence, backfills will be slower. Note: this doesn't address all potential memory ballooning issues, there is still a risk of ballooning during massive backfills depending on how many readers and how many loglets are being backfilled at the same time. The risk of ballooning is increased on the log-server side due to the concurrent number of read requests. ``` // intentionally empty ```
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The new configuration option
bifrost.replicated-loglet.read-batch-sizewill set (in bytes) the budget for every read batch request bifrost client will make to a log-server. The log-server will send at-least a single record even if it exceeds this budget.The option is set by default to 32KiB.
Additionally:
bifrost.replicated-loglet.readahead-recordsis reduced from 100 to 20 to reduce the potential memory ballooning during backfilling. As consequence, backfills will be slower.Note: this doesn't address all potential memory ballooning issues, there is still a risk of ballooning during massive backfills depending on how many readers and how many loglets are being backfilled at the same time. The risk of ballooning is increased on the log-server side due to the concurrent number of read requests.