Skip to content

Add initialWindowSizeKB to the io.l5d.mesh interpreter #2364

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 4 commits into from

Conversation

cpretzer
Copy link
Contributor

Signed-off-by: Charles Pretzer charles@buoyant.io

Signed-off-by: Charles Pretzer <charles@buoyant.io>
Signed-off-by: Charles Pretzer <charles@buoyant.io>
Copy link
Member

@adleong adleong left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Beautiful!

@olix0r
Copy link
Member

olix0r commented Dec 18, 2019

This configures the initial stream window size? Does it also alter the connection window size? If not, we may not actually fix the issue.

@cpretzer
Copy link
Contributor Author

@olix0r you're right, this only addresses the initial window size.

It's my understanding that the connection flow-control window is set to the initial window size (surprise! 😄) and then is updated through WINDOW_UPDATE calls.

Looking through the code, I don't see any explicit management of WINDOW_UPDATE frames, so I believe that one of the underlying libraries (netty or finagle) manages the window updates, specifically the value by which the size is increased or decreased.

While testing this fix, I triggered successive events which caused Namerd to send requests to the io.l5d.mesh interpreter with payloads above and below 64kb, and didn't run see any unexpected behavior.

That being said, I'd have to investigate the algorithm which increases and decreases the window size in order to understand how this PR will affect the behavior in a long running environment.

@olix0r
Copy link
Member

olix0r commented Dec 18, 2019

@cpretzer @adleong reminds me that we automatically acknowledge data on the connection window, so I think the point is moot. We probably should run with a larger connection window, if we can, but it need not block this. We may, however, want to call the setting initialStreamWindowSizeKB to make it clear that this does not impact the connection window.

@adleong
Copy link
Member

adleong commented Dec 18, 2019

I think what's happening here is that the client doesn't release data frame until it has accumulated enough to decode the entire gRPC message. In this case, the gRPC message is too large to fit into the flow control window and therefore nothing can make progress.

So I think increasing the flow control window size is actually the right long term solution here. Since the client doesn't need to perform backpressure in any meaningful way, we should really set the initial flow control window to the maximum amount of memory that the client is willing to buffer for service discovery data. A default of 10 MB seems very reasonable to me.

Signed-off-by: Charles Pretzer <charles@buoyant.io>
Signed-off-by: Charles Pretzer <charles@buoyant.io>
@adleong
Copy link
Member

adleong commented Dec 19, 2019

it looks like some release changes made it into this branch. let's get this PR merged with just the window size config and then we can cut a release off of master.

@adleong
Copy link
Member

adleong commented Dec 19, 2019

just to be 100% clear: we should revert the changes to Base.scala and CHANGES.md in this PR before merging it, and we should merge this PR to master before doing a release.

@cpretzer cpretzer closed this Dec 19, 2019
@cpretzer
Copy link
Contributor Author

Closing to remove changes to release files BUILD.md and CHANGES.md

@zaharidichev
Copy link
Member

@adleong Just to be clear that I fully understand. The reason why the client is not sending windows update frames is because this logic is part of the release routine on the data frame, and this frame is not released until the decoding stream has been able to materialize it into a message. Correct ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants