Skip to content

Respect maxConcurrentStreams #2327

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

zaharidichev
Copy link
Member

@zaharidichev zaharidichev commented Sep 10, 2019

This commit adds the functionality to respect maxConcurrentStreams for server dispatchers. The reason for this is that H2 settings such as maxConcurrentStreams are merely advertisement based. We as a server need to protect against misbehaving clients that do not respect the settings we have advertised. Otherwise we are prone to memory leaks and all sorts of resource depletion problems.

This PR is a limited version of #2325 which was a more opinionated version of this work.

Signed-off-by: Zahari Dichev zaharidichev@gmail.com

Copy link
Member

@adleong adleong left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought that we discovered that Netty already enforces this limit on the client side. Shouldn't we only need to enforce this for the server?

@zaharidichev zaharidichev force-pushed the zd/respect-max-concurrent-streams-limited branch from 997175d to 0732ade Compare September 12, 2019 07:46
Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
@zaharidichev zaharidichev force-pushed the zd/respect-max-concurrent-streams-limited branch from 0732ade to c628091 Compare September 12, 2019 08:01
Copy link
Member

@adleong adleong left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I went back and forth on this a bunch but I think this is the right thing.

At first I was going to suggest that we should really only count streams in the StreamOpen state and not StreamLocalReset or StreamFailed. Unfortunately, this is more complicated because we definitely don't want to have to iterate over the whole stream map each time we need to decide if a new stream can be started.

But given that the whole point of this is to protect us from memory leaks caused by badly behaved clients, perhaps it does make sense to count everything in the streams map. We may want to add a comment explaining why we count everything in the streams map and not just StreamOpen (otherwise, future code spelunkers may see this and wonder if it's a bug/oversight)

@zaharidichev
Copy link
Member Author

@adleong We can easily avoid iterration by maintaining counters or splitting things in multiple maps. I think that adds complexity to begin with. Also I think counting streams that are in StreamLocalReset is the correct thing to do. After all these streams are still occupying some sort of resources. I think it is correct to assume that in well behaved scenarios these streams will be evicted from the map once the other side closes as well, so its safe to simply do a .size. Will add a comment about that one.

On another note, had to google what a spelunkers is...

A spelunker is an explorer of caves. If you hope to one day be a spelunker, you probably have a love of dark, damp spaces and headlamps.

Spot on! Probably the best word to describe anyone who delves into the Netty4BaseDispatcher code

Signed-off-by: Zahari Dichev <zaharidichev@gmail.com>
Copy link
Member

@adleong adleong left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@adleong adleong merged commit 4e6be5d into linkerd:master Sep 19, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants