use id-tracker to estimate internal points range #6593
Conversation
📝 WalkthroughWalkthroughThe ✨ Finishing Touches
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
lib/segment/src/segment_constructor/segment_builder.rs (1)
359-369: Remove redundant debug assertion.The
debug_assert!on line 360 is redundant since it's already inside anifblock with the same condition. The assertion will always trigger in debug mode when the condition is true, making it unnecessary.if new_internal_range != internal_range { - debug_assert!( - new_internal_range != internal_range, - "Internal ids range mismatch between self segment vectors and other segment vectors\n\ - vector_name: {vector_name}, self range: {new_internal_range:?}, other range: {internal_range:?}" - ); return Err(OperationError::service_error(format!( "Internal ids range mismatch between self segment vectors and other segment vectors\n\ vector_name: {vector_name}, self range: {new_internal_range:?}, other range: {internal_range:?}" )));
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
lib/segment/src/segment_constructor/segment_builder.rs(2 hunks)
⏰ Context from checks skipped due to timeout of 90000ms (13)
- GitHub Check: test-snapshot-operations-s3-minio
- GitHub Check: Basic TLS/HTTPS tests
- GitHub Check: test-shard-snapshot-api-s3-minio
- GitHub Check: test-consistency
- GitHub Check: test-consensus-compose
- GitHub Check: test-low-resources
- GitHub Check: storage-compat-test
- GitHub Check: integration-tests-consensus
- GitHub Check: integration-tests
- GitHub Check: lint
- GitHub Check: rust-tests (windows-latest)
- GitHub Check: rust-tests (macos-latest)
- GitHub Check: rust-tests (ubuntu-latest)
🔇 Additional comments (2)
lib/segment/src/segment_constructor/segment_builder.rs (2)
326-330: Good simplification of internal range calculation.The direct computation of the internal range from the id tracker is cleaner and more reliable than depending on vector storage, especially for edge cases like empty vectors or multivectors.
374-440: Well-structured refactoring of the point processing loop.The refactoring improves code clarity by:
- Using
zipiterator for cleaner iteration- Better variable naming (
new_internal_idvsold_internal_id)- Consolidating payload propagation logic outside the match statement
This makes the code more maintainable while preserving the original functionality.
Previously optimizer used vector storage to estimate range of internal points
But this approach became problematic once we introduced multivectors. And in case of empty vectors it doesn't even work properly.
This PR simplified assignment of the internal range, by using id tracker instead of vectors storage