Speed up pending changes lookups using ahash #6385
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
A very simple change - replace the standard
HashMap
withAHashMap
. Or more specifically, hash usingahash
.It changes the hashmap in the mmap/rocksdb update buffer types where it lists changes pending flush. Its also applied to some surrounding types/functions. There are more places we can benefit from this, but lets take it step by step.
Ahash is much faster, at least on small numeric keys like we're using in many places. Here's a generic benchmark of sip13 (used by standard HashMap) versus ahash:
source
I've added a criterion benchmark that tests the change in the buffered mmap bitslice. I expect the effect to be similar for the other types.
Here's a bench for 100 iterations on look ups with 1 million pending changes:
Red shows the lookup timing distribution for the type before this PR. Blue shows the timings after this PR, being about 30% faster.
Note that in the case of the pending changes list, it may be empty a lot of the time. In that case there is no time difference. The improvement in lookup time will get more and more significant as the pending changes list grows.
All Submissions:
dev
branch. Did you create your branch fromdev
?New Feature Submissions:
cargo +nightly fmt --all
command prior to submission?cargo clippy --all --all-features
command?Changes to Core Features: