[minor] Minor improvements to partition snapshotting task #3605
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This was referenced Jul 30, 2025
Cleans up worker tasks and adds a new feature for task center to get a TaskGuard for unmanaged tasks that should be cancelled on drop. This also waits for worker shutdown explicitly before shutting down the rest of the system to allow partitions to drain cleanly with potential improvement in a follow-up PR that would let rocksdb flush to happen once PartitionStoreManager is dropped. Bonus: This includes a minor fix to let PPM cancel processors while they are still starting up, if they are still opening partition store or mid-initialization delay.
Durability tracker shouldn't hold a strong reference for PartitionStoreManager. It's a minor step towards the worker taking full control over PSM.
Should fix the flaky three-node trim test
- _Actually_ cancel in-flight snapshots on worker shutdown. - Fix error logging that didn't show the underlying error in log messages and make it stylistically consistent with the other errors.
pcholakov
approved these changes
Jul 31, 2025
There was a problem hiding this comment.
Thanks for adding the missing shutdown handling, @AhmedSoliman! 🙏
Comment on lines
+49
to
+50
| if let Some(result) = cancellation_token() | ||
| .run_until_cancelled(self.create_snapshot_inner()) |
There was a problem hiding this comment.
Thanks for cleaning up after me :-) I will learn.
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Stack created with Sapling. Best reviewed with ReviewStack.