-
Notifications
You must be signed in to change notification settings - Fork 3.4k
Endpoint redirect cleanup #35350
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Endpoint redirect cleanup #35350
Conversation
/test |
8ca9f3e
to
af40d87
Compare
/test |
af40d87
to
8dbfcf2
Compare
/test |
8dbfcf2
to
c1efc56
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The overall change LGTM, I think it simplifies things. I have one comment below as these things tend to be very subtle and just want to ensure we get it right. Typically, I'd expect cleanup
label to indicate changes without a functional impact, but this PR does have functional impact, right?
c1efc56
to
620666b
Compare
/test |
fe5fc40
to
638aba4
Compare
/test |
@jrajahalme amazing work! Some stylistic suggestions but this seems pretty correct. |
Move the set of desired redirects to the EndpointPolicy, as it is stable for any computed EndpointPolicy. If some redirects are missing, e.g., due to missing named ports or Envoy listeners, the policy will be recomputed when those are added, and the new policy will then contain also those redirects. This also allows each EndpointPolicy instance to have it's own set of redirects, depending on the selectorPolicy. If we kept a single set of realized redirects for the endpoint, we'd need to combine the current and new sets for the redirects to be available when incremental updates are added both to the current and new EndpointPolicy. Move adding redirects to happen before the MapState for the new EndpointPolicy is computed. This way the redirects are available when needed for MapState generation, removing the need for the second pass to add the redirects to the MapState. Redirect creation is simplified with new SelectorPolicy.RedirectFilters() iterator. Eliminate the (first) bpf policy map sync that was done just before removing old redirect. Old redirects are now removed in a finalizer function, that is run after we get acknowledgements for the newly added redirects (if any). If there is a NACK instead, then the finalizer function is not called at all, so there is no need for reverting the removals. The first sync could have sent traffic to Envoy while the Listeners or the NetworkPolicy were not fully updated yet. Now we rely solely on the bpf policy map sync that is done in regenerateBPF() after the wait for Envoy configuration updates is completed. At that point Envoy is ready to take traffic for the new listeners. Also after this point the old redirects would not receive any traffic any more and they can safely be removed by the finalizer function called via defer at the end of regenerateBPF(). Finally, remove old redirects through FinalizeList *after* the new redirects have been acknowledged by Envoy. Since references to the redirect ports of those redirects have already been removed from the policy map, there is no need to wait for an ack for redirect removal. This allows simplification of the related code as FinalizedList nor RevertStack are no longer needed for redirect removal. Signed-off-by: Jarno Rajahalme <jarno@isovalent.com>
Now that redirects are added before taking the endpoint lock we can update DNS restore rules with the new policy rules after the policy update, rather than from the old rules before the policy rules updates. Signed-off-by: Jarno Rajahalme <jarno@isovalent.com>
638aba4
to
1813047
Compare
/test |
Approved, thanks! |
Rationalize sequence of events for policy update and applying incremental policy map updates:
ConsumeMapChanges
inapplyPolicyMapChanges
so that it is more obvious that the selector cache version for bpf map updates and proxy network policy is the same on (as updated by ConsumeMapChanges)Keep realized redirects in EndpointPolicy: