Wait until replicas are activated, when creating custom shards #6817
Conversation
8db9214 to
c105990
Compare
- so that it creates custom shards in `Initializing` state instead of `Active` - and in doing so, also covers the check that `create_shard_key` request waits for replica activation
This comment was marked as resolved.
This comment was marked as resolved.
| &self, | ||
| check: F, | ||
| timeout: Duration, | ||
| ) -> impl Future<Output = CollectionResult<()>> + 'static |
There was a problem hiding this comment.
What was a reason for this change?
There was a problem hiding this comment.
Without this change, wait_for was borrowing/lifetime-bound to ReplicaSet, so you had to hold a lock to ReplicaSet while await-ing wait_for.
With this change, wait_for creates a future that is completely independent of ReplicaSet, so that you can call wait_for, release the lock to ReplicaSet, and then await wait_for future.
| if status { | ||
| Ok(()) | ||
| } else { | ||
| Err(CollectionError::service_error( |
There was a problem hiding this comment.
I checked usages, it should be safe to convert this error to timeout. Both Service Error and timeout are transient, also it looks like all actual places where it is used either don't care or force convert error into service error.
| for replica_set in shard_holder.all_shards() { | ||
| if replica_set.shard_key() != Some(&shard_key) { | ||
| continue; | ||
| } |
There was a problem hiding this comment.
Shall we create a getter in the shard holder to which we pass the key? That way we keep the shard key selection logic inside the shard holder.
We have some methods related to shard keys there, to get the shard IDs related to a shard key for example. But for this we'd probably require a new one.
Co-authored-by: generall <andrey@vasnetsov.com> Co-authored-by: timvisee <tim@visee.me>
Follow-up to #6778.
After #6778, we initialize custom shard replicas in
Initializingstate, and so when Qdrant returns response to the client, it does not yet guarantee that custom shard is ready to accept write requests. There will be a small delay while shards switch fromInitializingintoActivestate and fully available.This PR adds explicit
wait_for_statecheck, so that we ensure all replicas areActivebefore returning response to the client. I've also very slightly tweakedtest_shard_consistencytest, so that it should cover this check as part of existing test.All Submissions:
devbranch. Did you create your branch fromdev?New Feature Submissions:
cargo +nightly fmt --allcommand prior to submission?cargo clippy --all --all-featurescommand?Changes to Core Features: