buffer: change `Buffer` to be generic over inner service's `Future` #654

hawkw · 2022-03-11T17:34:50Z

This commit changes the Buffer service to be generic over the inner
service's Future type, rather than over the Service type itself. The
Worker type is still generic over the inner Service (and, it must
be, as it stores an instance of that service). This should reduce type
complexity of buffers a bit, as they will erase nested service types.

Unfortunately, Buffer is still generic over the inner service's
Future, which may also be nested, but it is likely to be less complex
than the Service. It would be nice if we could erase the Future type
as well, but I don't believe this is possible without either boxing the
futures or changing them to always be spawned on a background task,
neither of which seems desirable here.

Closes #641

This commit changes the `Buffer` service to be generic over the inner service's `Future` type, rather than over the `Service` type itself. The `Worker` type is still generic over the inner `Service` (and, it must be, as it stores an instance of that service). This should reduce type complexity of buffers a bit, as they will erase nested service types. Unfortunately, `Buffer` is still generic over the inner service's `Future`, which may also be nested, but it is likely to be less complex than the `Service`. It would be nice if we could erase the `Future` type as well, but I don't believe this is possible without either boxing the futures or changing them to always be spawned on a background task, neither of which seems desirable here. Closes #641

olix0r

Generally +1 on the change; some small readability quibbles

tower/src/buffer/service.rs

Signed-off-by: Eliza Weisman <eliza@buoyant.io>

hawkw · 2022-03-15T15:56:51Z

I think this is probably ready to merge, if anyone wants to give it another look?

davidpdrsn

Noise!

in tower-rs#654, breaking changes were made to the `Buffer` type. this commit adds mention of these breaking changes to the changelog, so that users upgrading from 0.4 to 0.5 can have record of what changed, and why.

in #654, breaking changes were made to the `Buffer` type. this commit adds mention of these breaking changes to the changelog, so that users upgrading from 0.4 to 0.5 can have record of what changed, and why.

this commit updates our tower dependency from 0.4 to 0.5. note that this commit does not affect the `tower-service` and `tower-layer` crates, reëxported by `tower` itself. the `Service<T>` trait and the closely related `Layer<S>` trait have not been changed. the `tower` crate's utilities have changed in various ways, some of particular note for the linkerd2 proxy. see these items, excerpted from the tower changelog: - **retry**: **Breaking Change** `retry::Policy::retry` now accepts `&mut Req` and `&mut Res` instead of the previous mutable versions. This increases the flexibility of the retry policy. To update, update your method signature to include `mut` for both parameters. ([#584]) - **retry**: **Breaking Change** Change Policy to accept &mut self ([#681]) - **retry**: **Breaking Change** `Budget` is now a trait. This allows end-users to implement their own budget and bucket implementations. ([#703]) - **util**: **Breaking Change** `Either::A` and `Either::B` have been renamed `Either::Left` and `Either::Right`, respectively. ([#637]) - **util**: **Breaking Change** `Either` now requires its two services to have the same error type. ([#637]) - **util**: **Breaking Change** `Either` no longer implemenmts `Future`. ([#637]) - **buffer**: **Breaking Change** `Buffer<S, Request>` is now generic over `Buffer<Request, S::Future>.` ([#654]) see: * <tower-rs/tower#584> * <tower-rs/tower#681> * <tower-rs/tower#703> * <tower-rs/tower#637> * <tower-rs/tower#654> the `Either` trait bounds are particularly impactful for us. because this runs counter to how we treat errors (skewing towards boxed errors, in general), we temporarily vendor a version of `Either` from the 0.4 release, whose variants have been renamed to match the 0.5 interface. updating to box the inner `A` and `B` services' errors, so we satiate the new `A::Error = B::Error` bounds, can be addressed as a follow-on. that's intentionally left as a separate change, due to the net size of our patchset between this branch and #3504. * <tower-rs/tower@v0.4.x...master> * <https://github.com/tower-rs/tower/blob/master/tower/CHANGELOG.md> this work is based upon #3504. for more information, see: * linkerd/linkerd2#8733 * #3504 Signed-off-by: katelyn martin <kate@buoyant.io> X-Ref: tower-rs/tower#815 X-Ref: tower-rs/tower#817 X-Ref: tower-rs/tower#818 X-Ref: tower-rs/tower#819

this commit updates our tower dependency from 0.4 to 0.5. note that this commit does not affect the `tower-service` and `tower-layer` crates, reëxported by `tower` itself. the `Service<T>` trait and the closely related `Layer<S>` trait have not been changed. the `tower` crate's utilities have changed in various ways, some of particular note for the linkerd2 proxy. see these items, excerpted from the tower changelog: - **retry**: **Breaking Change** `retry::Policy::retry` now accepts `&mut Req` and `&mut Res` instead of the previous mutable versions. This increases the flexibility of the retry policy. To update, update your method signature to include `mut` for both parameters. ([tower-rs/tower#584]) - **retry**: **Breaking Change** Change Policy to accept &mut self ([tower-rs/tower#681]) - **retry**: **Breaking Change** `Budget` is now a trait. This allows end-users to implement their own budget and bucket implementations. ([tower-rs/tower#703]) - **util**: **Breaking Change** `Either::A` and `Either::B` have been renamed `Either::Left` and `Either::Right`, respectively. ([tower-rs/tower#637]) - **util**: **Breaking Change** `Either` now requires its two services to have the same error type. ([tower-rs/tower#637]) - **util**: **Breaking Change** `Either` no longer implemenmts `Future`. ([tower-rs/tower#637]) - **buffer**: **Breaking Change** `Buffer<S, Request>` is now generic over `Buffer<Request, S::Future>.` ([tower-rs/tower#654]) see: * <tower-rs/tower#584> * <tower-rs/tower#681> * <tower-rs/tower#703> * <tower-rs/tower#637> * <tower-rs/tower#654> the `Either` trait bounds are particularly impactful for us. because this runs counter to how we treat errors (skewing towards boxed errors, in general), we temporarily vendor a version of `Either` from the 0.4 release, whose variants have been renamed to match the 0.5 interface. updating to box the inner `A` and `B` services' errors, so we satiate the new `A::Error = B::Error` bounds, can be addressed as a follow-on. that's intentionally left as a separate change, due to the net size of our patchset between this branch and #3504. * <tower-rs/tower@v0.4.x...master> * <https://github.com/tower-rs/tower/blob/master/tower/CHANGELOG.md> this work is based upon #3504. for more information, see: * linkerd/linkerd2#8733 * #3504 Signed-off-by: katelyn martin <kate@buoyant.io> X-Ref: tower-rs/tower#815 X-Ref: tower-rs/tower#817 X-Ref: tower-rs/tower#818 X-Ref: tower-rs/tower#819

* chore(deps)!: upgrade to tower 0.5 this commit updates our tower dependency from 0.4 to 0.5. note that this commit does not affect the `tower-service` and `tower-layer` crates, reëxported by `tower` itself. the `Service<T>` trait and the closely related `Layer<S>` trait have not been changed. the `tower` crate's utilities have changed in various ways, some of particular note for the linkerd2 proxy. see these items, excerpted from the tower changelog: - **retry**: **Breaking Change** `retry::Policy::retry` now accepts `&mut Req` and `&mut Res` instead of the previous mutable versions. This increases the flexibility of the retry policy. To update, update your method signature to include `mut` for both parameters. ([tower-rs/tower#584]) - **retry**: **Breaking Change** Change Policy to accept &mut self ([tower-rs/tower#681]) - **retry**: **Breaking Change** `Budget` is now a trait. This allows end-users to implement their own budget and bucket implementations. ([tower-rs/tower#703]) - **util**: **Breaking Change** `Either::A` and `Either::B` have been renamed `Either::Left` and `Either::Right`, respectively. ([tower-rs/tower#637]) - **util**: **Breaking Change** `Either` now requires its two services to have the same error type. ([tower-rs/tower#637]) - **util**: **Breaking Change** `Either` no longer implemenmts `Future`. ([tower-rs/tower#637]) - **buffer**: **Breaking Change** `Buffer<S, Request>` is now generic over `Buffer<Request, S::Future>.` ([tower-rs/tower#654]) see: * <tower-rs/tower#584> * <tower-rs/tower#681> * <tower-rs/tower#703> * <tower-rs/tower#637> * <tower-rs/tower#654> the `Either` trait bounds are particularly impactful for us. because this runs counter to how we treat errors (skewing towards boxed errors, in general), we temporarily vendor a version of `Either` from the 0.4 release, whose variants have been renamed to match the 0.5 interface. updating to box the inner `A` and `B` services' errors, so we satiate the new `A::Error = B::Error` bounds, can be addressed as a follow-on. that's intentionally left as a separate change, due to the net size of our patchset between this branch and #3504. * <tower-rs/tower@v0.4.x...master> * <https://github.com/tower-rs/tower/blob/master/tower/CHANGELOG.md> this work is based upon #3504. for more information, see: * linkerd/linkerd2#8733 * #3504 Signed-off-by: katelyn martin <kate@buoyant.io> X-Ref: tower-rs/tower#815 X-Ref: tower-rs/tower#817 X-Ref: tower-rs/tower#818 X-Ref: tower-rs/tower#819 * fix(stack/loadshed): update test affected by tower-rs/tower#635 this commit updates a test that was affected by breaking changes in tower's `Buffer` middleware. see this excerpt from the description of that change: > I had to change some of the integration tests slightly as part of this > change. This is because the buffer implementation using semaphore > permits is _very subtly_ different from one using a bounded channel. In > the `Semaphore`-based implementation, a semaphore permit is stored in > the `Message` struct sent over the channel. This is so that the capacity > is used as long as the message is in flight. However, when the worker > task is processing a message that's been recieved from the channel, > the permit is still not dropped. Essentially, the one message actively > held by the worker task _also_ occupies one "slot" of capacity, so the > actual channel capacity is one less than the value passed to the > constructor, _once the first request has been sent to the worker_. The > bounded MPSC changed this behavior so that capacity is only occupied > while a request is actually in the channel, which broke some tests > that relied on the old (and technically wrong) behavior. bear particular attention to this: > The bounded MPSC changed this behavior so that capacity is only > occupied while a request is actually in the channel, which broke some > tests that relied on the old (and technically wrong) behavior. that pr adds an additional message to the channel in tests exercising the laod-shedding behavior, on account of the previous (incorrect) behavior. https://github.com/tower-rs/tower/pull/635/files#r797108274 this commit performs the same change for our corresponding test, adding an additional `ready()` call before we hit the buffer's limit. Signed-off-by: katelyn martin <kate@buoyant.io> * review: use vendored `Either` for consistency #3744 (comment) Signed-off-by: katelyn martin <kate@buoyant.io> --------- Signed-off-by: katelyn martin <kate@buoyant.io>

hawkw requested review from olix0r, davidpdrsn and LucioFranco March 11, 2022 17:34

olix0r reviewed Mar 11, 2022

View reviewed changes

tower/src/buffer/service.rs Outdated Show resolved Hide resolved

tower/src/buffer/service.rs Outdated Show resolved Hide resolved

tower/src/buffer/service.rs Outdated Show resolved Hide resolved

type parameter name cleanup

e441eb1

Signed-off-by: Eliza Weisman <eliza@buoyant.io>

hawkw mentioned this pull request Mar 11, 2022

tower: Consider generating error enums with thiserror #655

Open

hawkw requested a review from olix0r March 11, 2022 17:49

Merge branch 'master' into eliza/bufferase

c06f815

davidpdrsn approved these changes Mar 16, 2022

View reviewed changes

hawkw merged commit 9f86b8f into master Mar 29, 2022

hawkw deleted the eliza/bufferase branch March 29, 2022 17:41

markdingram mentioned this pull request Sep 24, 2024

Upgrades Tower to 0.5.1 kube-rs/kube#1589

Merged

cratelyn mentioned this pull request Mar 12, 2025

chore: add Buffer breaking changes to changelog #818

Merged

cratelyn mentioned this pull request Mar 14, 2025

chore(deps): upgrade to tower 0.5 linkerd/linkerd2-proxy#3744

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

buffer: change `Buffer` to be generic over inner service's `Future` #654

buffer: change `Buffer` to be generic over inner service's `Future` #654

Uh oh!

hawkw commented Mar 11, 2022

Uh oh!

olix0r left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hawkw commented Mar 15, 2022

Uh oh!

davidpdrsn left a comment

Uh oh!

Uh oh!

buffer: change Buffer to be generic over inner service's Future #654

buffer: change Buffer to be generic over inner service's Future #654

Uh oh!

Conversation

hawkw commented Mar 11, 2022

Uh oh!

olix0r left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hawkw commented Mar 15, 2022

Uh oh!

davidpdrsn left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

buffer: change `Buffer` to be generic over inner service's `Future` #654

buffer: change `Buffer` to be generic over inner service's `Future` #654