Remove ingress dedicated runtime #3607
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This was referenced Jul 30, 2025
Cleans up worker tasks and adds a new feature for task center to get a TaskGuard for unmanaged tasks that should be cancelled on drop. This also waits for worker shutdown explicitly before shutting down the rest of the system to allow partitions to drain cleanly with potential improvement in a follow-up PR that would let rocksdb flush to happen once PartitionStoreManager is dropped. Bonus: This includes a minor fix to let PPM cancel processors while they are still starting up, if they are still opening partition store or mid-initialization delay.
Durability tracker shouldn't hold a strong reference for PartitionStoreManager. It's a minor step towards the worker taking full control over PSM.
Should fix the flaky three-node trim test
- _Actually_ cancel in-flight snapshots on worker shutdown. - Fix error logging that didn't show the underlying error in log messages and make it stylistically consistent with the other errors.
This PR removes the dedicated ingress runtime and allows ingress to share the runtime with `default`. This is a step towards reducing the number of runtimes, threads, and metric dimensions. In my testing, the impact of this change is negligible and in all cases we'd be improving poll latencies in default runtime by avoiding sync IO operations from log-server and metadata server soon.
pcholakov
approved these changes
Jul 31, 2025
There was a problem hiding this comment.
You probably want to wait for Till's review, but FWIW, if this has marginal benefit then it looks good to go! Thanks @AhmedSoliman
tillrohrmann
approved these changes
Jul 31, 2025
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR removes the dedicated ingress runtime and allows ingress to share the runtime with
default. This is a step towards reducing the number of runtimes, threads, and metric dimensions. In my testing, the impact of this change is negligible and in all cases we'd be improving poll latencies in default runtime by avoiding sync IO operations from log-server and metadata server soon.Stack created with Sapling. Best reviewed with ReviewStack.