IO resource usage permit #6015
Merged
IO resource usage permit #6015
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
a98195c to
9656b26
Compare
generall
commented
Feb 19, 2025
Comment on lines
459
to
493
| // 000 - acquired | ||
| // +++ - blocked on waiting | ||
| // | ||
| // Case: 1 indexation job at a time, long indexing | ||
| // | ||
| // IO limit = 1 | ||
| // CPU limit = 2 Next optimization | ||
| // │ loop | ||
| // │ | ||
| // ▼ | ||
| // IO 0 00000000000000 000000000 | ||
| // CPU 1 00000000000000000 | ||
| // 2 00000000000000000 | ||
| // | ||
| // | ||
| // IO 0 ++++++++++++++00000000000000000 | ||
| // CPU 1 ++++++++0000000000 | ||
| // 2 ++++++++0000000000 | ||
| // | ||
| // | ||
| // Case: 1 indexing j0b at a time, short indexation | ||
| // | ||
| // | ||
| // IO limit = 1 | ||
| // CPU limit = 2 | ||
| // | ||
| // | ||
| // IO 0 000000000000 ++++++++0000000000 | ||
| // CPU 1 00000 | ||
| // 2 00000 | ||
| // | ||
| // IO 0 ++++++++++++00000000000 +++++++ | ||
| // CPU 1 00000 | ||
| // 2 00000 | ||
| // At this stage workload shifts from IO to CPU, so we can release IO permit |
There was a problem hiding this comment.
Most important part is here
timvisee
reviewed
Feb 19, 2025
timvisee
approved these changes
Feb 19, 2025
|
Are there some benchmark results to share to validate this patch? |
|
Not yet. I wanted to make them before merging |
|
Sorry! 🤡 |
|
Improvement is visible on large-scale indexation. In my experiment I was indexing 400M 512d vectors, and current release take about 40 hours on 8 CPU machine, while dev with this change finishes in 28 hours. Also attaching CPU/Disk usage proviles, which clearly indicate an improvement: NEW OLD |
timvisee
added a commit
that referenced
this pull request
Mar 21, 2025
* rename cpu_budget -> resource_budget * clippy * add io budget to resources * fmt * move budget structures into a separate file * add extend permit function * dont extend existing permit * switch from IO to CPU permit * do not release resource before aquiring an extension * fmt * Review remarks * Improve resource permit number assertion * Make resource permit replace_with only acquire extra needed permits * Remove obsolete drop implementation * allocate IO budget same as CPU * review fixes --------- Co-authored-by: timvisee <tim@visee.me>
Merged
6 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Optimization is running in multiple stages. One stage is IO-heavy, another is CPU-heavy.
We want to utilize all types of resources at the same time, so we introduce a new type of resource budget, that allows other parallel processes to pick up IO-heavy phase without awaiting for CPU-heavy one