Skip to content

Conversation

mattklein123
Copy link
Member

We need to buffer the decoded data before performing on
complete processing.

Fixes #5646

Risk Level: Medium. Scary code to change but pretty confident in
the change and I added an integration test.
Testing: New integration test.
Docs Changes: N/A
Release Notes: N/A

We need to buffer the decoded data before performing on
complete processing.

Fixes #5646

Signed-off-by: Matt Klein <mklein@lyft.com>
@mattklein123
Copy link
Member Author

cc @irelandKen

Signed-off-by: Matt Klein <mklein@lyft.com>
Signed-off-by: Matt Klein <mklein@lyft.com>
Copy link
Contributor

@alyssawilk alyssawilk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch! I'm a bit unsure of the fix, so more questions than comments from me...

// buffer limit we give up on retries and buffering. We must buffer using addDecodedData()
// so that all buffered data is available by the time we do request complete processing and
// potentially shadow.
callbacks_->addDecodedData(data, true);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is the second time in as many weeks I've seen us have to do this workaround.

Not required for this PR but any thoughts for a change we could make to avoid this gotcha?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah this has come up a few times in slightly different contexts. I will think about this. I don't think there are any easy fixes here given the different permutations of things we need to support. The only thing I can think of quickly is to somehow indicate that a filter is of a particular type, and then the HCM would "pre-buffer." With that said, even that wouldn't work for the router filter because we buffer sometimes and not others...

Signed-off-by: Matt Klein <mklein@lyft.com>
@mattklein123
Copy link
Member Author

@alyssawilk updated

Copy link
Contributor

@alyssawilk alyssawilk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, thanks for the fix and for the clarification!

FakeStreamPtr upstream_request2;
ASSERT_TRUE(fake_upstreams_[0]->waitForHttpConnection(*dispatcher_, fake_upstream_connection2));
ASSERT_TRUE(fake_upstream_connection2->waitForNewStream(*dispatcher_, upstream_request2));
ASSERT_TRUE(upstream_request2->waitForEndStream(*dispatcher_));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Boo, one of our utilities should work for this and they totally don't.
Goes on my TODO list :-)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah sorry I thought the same thing and was going to make the utility functions work for this, but I got lazy. I will circle back if you don't get to it first.

@mattklein123 mattklein123 merged commit 61c7b79 into master Jan 23, 2019
@mattklein123 mattklein123 deleted the fix_http2_body_shadow branch January 23, 2019 16:17
danzh2010 pushed a commit to danzh2010/envoy that referenced this pull request Jan 24, 2019
We need to buffer the decoded data before performing on
complete processing.

Fixes envoyproxy#5646

Signed-off-by: Matt Klein <mklein@lyft.com>

Signed-off-by: Dan Zhang <danzh@google.com>
danzh2010 pushed a commit to danzh2010/envoy that referenced this pull request Jan 31, 2019
We need to buffer the decoded data before performing on
complete processing.

Fixes envoyproxy#5646

Signed-off-by: Matt Klein <mklein@lyft.com>

Signed-off-by: Matt Klein <mattklein123@gmail.com>
fredlas pushed a commit to fredlas/envoy that referenced this pull request Mar 5, 2019
We need to buffer the decoded data before performing on
complete processing.

Fixes envoyproxy#5646

Signed-off-by: Matt Klein <mklein@lyft.com>
Signed-off-by: Fred Douglas <fredlas@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants