Always run cancellation handle for interrupted optimizations #6549
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
timvisee
reviewed
May 19, 2025
There was a problem hiding this comment.
There's still some places where we might error without handling cancellation, for example:
qdrant/lib/collection/src/collection_manager/optimizers/segment_optimizer.rs
Lines 713 to 728 in d36e0dc
I think it'd be best to handle those too, possibly by moving everything that works with proxies into a sub-function.
This comment was marked as resolved.
This comment was marked as resolved.
agourlay
commented
May 19, 2025
| @@ -10,7 +10,7 @@ use common::disk::dir_size; | |||
| use io::storage_version::StorageVersion; | |||
| use itertools::Itertools; | |||
| use parking_lot::{Mutex, RwLockUpgradableReadGuard}; | |||
| use segment::common::operation_error::{OperationResult, check_process_stopped}; | |||
There was a problem hiding this comment.
I have unified the code file on on the local self.check_cancellation to not have two styles.
6 tasks
timvisee
approved these changes
May 20, 2025
generall
pushed a commit
that referenced
this pull request
May 22, 2025
* Always run cancelletatiob handle for interrupted optimizations * Stick to one way of checking cancelation * Move segment optimization in sub function, cancel in single place (#6557) * Move segment optimization into separate function, cancel in single place * Don't use owned lock * Unify cancellation check --------- Co-authored-by: Tim Visée <tim+github@visee.me>
n0x29a
pushed a commit
that referenced
this pull request
May 23, 2025
* Always run cancelletatiob handle for interrupted optimizations * Stick to one way of checking cancelation * Move segment optimization in sub function, cancel in single place (#6557) * Move segment optimization into separate function, cancel in single place * Don't use owned lock * Unify cancellation check --------- Co-authored-by: Tim Visée <tim+github@visee.me>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Seems to fix Crasher, will let CI chew on it.
This PR makes sure we handle the on going proxy segments when failing with any error to build a new optimized segment.