Add new streamAfterContentLengthKB config field for HTTP/1 #2189
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When Linkerd receives a significantly large HTTP request with a set content-length, the underlying Finagle router service buffers the entire message into memory instead of streaming the HTTP message in smaller chunks. This can cause problems when a client is sending very large file uploads without using
Transfer-Encoding: Chunked
This PR pulls in a new Finagle param (#740) that forces a finagle server to receive large HTTP messages in smaller chunks when the HTTP messages content-length exceeds a certain content-length threshold. In addition, Linkerd exposes this Finagle param to allow for configurability. I was able to test this out in a docker environment where I sent a significantly large file (140MB) to Linkerd. Previously Linkerd would load buffer the entire message into memory and was easily visible in the JVM memory usage. After the change, the same request was issued again and we no longer observe the spike in memory.
fixes #2188
Signed-off-by: Dennis Adjei-Baah dennis@buoyant.io