Skip to content

Conversation

viktor-kurchenko
Copy link
Contributor

@viktor-kurchenko viktor-kurchenko commented Nov 21, 2024

This PR introduces fix for egressMasqueradeInterfaces multiple interfaces support.
Multi-interface support can be helpful when cluster contains nodes with different operation systems and network interfaces might have different names
E.g.: EKS cluster upgrade to Amazon Linux 2023 that uses ens instead of eth.

Usage example via Cilium CLI:

cilium install --helm-set=egressMasqueradeInterfaces=eth+\,ens+ ...

Please see per commit for full details.

@maintainer-s-little-helper maintainer-s-little-helper bot added the dont-merge/needs-release-note-label The author needs to describe the release impact of these changes. label Nov 21, 2024
@viktor-kurchenko viktor-kurchenko force-pushed the vk/pr/egress/masq/iptables/fix branch from bae3304 to cc378d4 Compare November 21, 2024 20:12
@viktor-kurchenko viktor-kurchenko added area/datapath Impacts bpf/ or low-level forwarding details, including map management and monitor messages. release-note/minor This PR changes functionality that users may find relevant to operating Cilium. release-note/bug This PR fixes an issue in a previous release of Cilium. area/iptables Impacts how Cilium interacts with iptables. labels Nov 21, 2024
@maintainer-s-little-helper maintainer-s-little-helper bot removed dont-merge/needs-release-note-label The author needs to describe the release impact of these changes. labels Nov 21, 2024
@viktor-kurchenko viktor-kurchenko force-pushed the vk/pr/egress/masq/iptables/fix branch from cc378d4 to 407ac01 Compare November 21, 2024 20:15
`EgressMasqueradeInterfaces` config property removed as unused.

Signed-off-by: viktor-kurchenko <viktor.kurchenko@isovalent.com>
@viktor-kurchenko viktor-kurchenko force-pushed the vk/pr/egress/masq/iptables/fix branch from 407ac01 to 24cc392 Compare November 21, 2024 20:28
@viktor-kurchenko
Copy link
Contributor Author

/test

@viktor-kurchenko viktor-kurchenko changed the title Vk/pr/egress/masq/iptables/fix Egress masquerade interfaces fix Nov 21, 2024
@viktor-kurchenko viktor-kurchenko changed the title Egress masquerade interfaces fix Egress masquerade multiple interfaces fix Nov 21, 2024
MasqueradeInterfaces param passed into Cilium as quoted string and Viper
GetStringSlice method always return it as one element slice.
The commit changes MasqueradeInterfaces Viper param type and parsing logic.

Signed-off-by: viktor-kurchenko <viktor.kurchenko@isovalent.com>
When `NodeIpsetNeeded` param enabled, iptables rules might include
MasqueradeInterfaces (if configured) and pass them as output
interface (`-o`) param for `iptables` command.
Iptables output interface param accepts single interface instead
of list.

The commit fixes iptables commands construction logic (to consider
multiple egress masquerade interfaces) and extracts the logic in a
separate function.

Signed-off-by: viktor-kurchenko <viktor.kurchenko@isovalent.com>
When `EnableMasqueradeRouteSource` param disabled, iptables rules might include
MasqueradeInterfaces (if configured) and pass them as output interface (`-o`)
param for `iptables` command.
Iptables output interface param accepts single interface instead of list.

The commit fixes iptables commands construction logic (to consider multiple egress
masquerade interfaces) and extracts the logic in a separate function.

Signed-off-by: viktor-kurchenko <viktor.kurchenko@isovalent.com>
@viktor-kurchenko viktor-kurchenko force-pushed the vk/pr/egress/masq/iptables/fix branch from 24cc392 to 9c52c85 Compare November 22, 2024 10:02
@viktor-kurchenko
Copy link
Contributor Author

/test

@viktor-kurchenko viktor-kurchenko marked this pull request as ready for review November 22, 2024 13:00
@viktor-kurchenko viktor-kurchenko requested review from a team as code owners November 22, 2024 13:00
@auriaave
Copy link
Contributor

auriaave commented Nov 22, 2024

@viktor-kurchenko , what is the difference between the previous ruleset and the current ones?

IIUC, an example of a previous post rule with multiple interfaces is:

-t nat -A CILIUM_POST_nat -o eth+,ens+ -m set --match-set 1.1.1.1 dst -m comment --comment "xxxxx" -j ACCEPT

while now, it seems like we get 2 post rules with a single interface each one of them:

-t nat -A CILIUM_POST_nat -o eth+ -m set --match-set 1.1.1.1 dst -m comment --comment "xxxxx" -j ACCEPT
-t nat -A CILIUM_POST_nat -o ens+ -m set --match-set 1.1.1.1 dst -m comment --comment "xxxxx" -j ACCEPT

Is the above correct?

And same for the AllEgressMasqueradeCmds, having a single rule with comma-separated values or having multiple rules with a single interface value per rule.

If it is the case, what is the difference betwen both approaches?

@viktor-kurchenko
Copy link
Contributor Author

@viktor-kurchenko , what is the difference between the previous ruleset and the current ones?

Yes, you are right.
I don't think that -o supports more than one interface. So, the PR adds one rule per interface.
Quote from man iptables:

       [!] -o, --out-interface name
              Name of an interface via which a packet is going to be sent (for packets entering the FORWARD, OUTPUT and POSTROUTING chains).  When the "!" argument is  used  be‐
              fore the interface name, the sense is inverted.  If the interface name ends in a "+", then any interface which begins with this name will match.  If this option is
              omitted, any interface name will match.

I've successfully tested the fix with EKS AL2023.

@auriaave
Copy link
Contributor

LGTM - thank you for the clarification and for the fix!

@joestringer
Copy link
Member

@viktor-kurchenko One more small follow-up - would you also be able to review the wording in Documentation/installation/k8s-install-helm.rst around "Mixed node clusters" and update it (+mark for backport for v1.17)? I think it's describing the limitation fixed by this PR, so we should be able to remove the sentence that says this is unsupported.

@viktor-kurchenko
Copy link
Contributor Author

@viktor-kurchenko One more small follow-up - would you also be able to review the wording in Documentation/installation/k8s-install-helm.rst around "Mixed node clusters" and update it (+mark for backport for v1.17)? I think it's describing the limitation fixed by this PR, so we should be able to remove the sentence that says this is unsupported.

Hey @joestringer, I had a discussion with @auriaave about this quote.
I think technically we support mixed node clusters now but AFAIK nobody tested this yet.
So, is it fare to remove the sentence (Mixed node clusters are not supported currently.) ?

@joestringer
Copy link
Member

joestringer commented Jan 15, 2025

Is it [fair] to remove the sentence [...]

I think that's a reasonable start point yeah. If we think it should work, then we shouldn't tell people it doesn't work.

I understand if we haven't yet provisioned a mixed node cluster to try it out. We could also maybe publish the intended new configuration options in the Cilium docs there, then community members can try it out and report whether the options worked well or not.

@viktor-kurchenko
Copy link
Contributor Author

I think that's a reasonable start point yeah. If we think it should work, then we shouldn't tell people it doesn't work.

So, what if we update only OSS doc for now?
CC: @networkop @auriaave

viktor-kurchenko added a commit that referenced this pull request Jan 17, 2025
The commit removes EKS cluster mixed node support restriction since
egress masquerade supports multiple interfaces (see fix: #36103).

Signed-off-by: viktor-kurchenko <viktor.kurchenko@isovalent.com>
github-merge-queue bot pushed a commit that referenced this pull request Jan 17, 2025
The commit removes EKS cluster mixed node support restriction since
egress masquerade supports multiple interfaces (see fix: #36103).

Signed-off-by: viktor-kurchenko <viktor.kurchenko@isovalent.com>
@viktor-kurchenko viktor-kurchenko added backport/1.16 This PR represents a backport for Cilium 1.16.x of a PR that was merged to main. needs-backport/1.16 This PR / issue needs backporting to the v1.16 branch labels Jan 20, 2025
@julianwiedmann julianwiedmann removed the backport/1.16 This PR represents a backport for Cilium 1.16.x of a PR that was merged to main. label Jan 22, 2025
@rastislavs rastislavs mentioned this pull request Jan 22, 2025
19 tasks
@rastislavs rastislavs added backport-pending/1.16 The backport for Cilium 1.16.x for this PR is in progress. and removed needs-backport/1.16 This PR / issue needs backporting to the v1.16 branch labels Jan 22, 2025
@github-actions github-actions bot added backport-done/1.16 The backport for Cilium 1.16.x for this PR is done. and removed backport-pending/1.16 The backport for Cilium 1.16.x for this PR is in progress. labels Jan 24, 2025
giorio94 pushed a commit that referenced this pull request Jan 27, 2025
[ upstream commit 69c1d74 ]

The commit removes EKS cluster mixed node support restriction since
egress masquerade supports multiple interfaces (see fix: #36103).

Signed-off-by: viktor-kurchenko <viktor.kurchenko@isovalent.com>
Signed-off-by: Marco Iorio <marco.iorio@isovalent.com>
giorio94 pushed a commit that referenced this pull request Jan 27, 2025
[ upstream commit 69c1d74 ]

The commit removes EKS cluster mixed node support restriction since
egress masquerade supports multiple interfaces (see fix: #36103).

Signed-off-by: viktor-kurchenko <viktor.kurchenko@isovalent.com>
Signed-off-by: Marco Iorio <marco.iorio@isovalent.com>
@giorio94 giorio94 mentioned this pull request Jan 27, 2025
8 tasks
@giorio94 giorio94 added backport-pending/1.15 The backport for Cilium 1.15.x for this PR is in progress. and removed needs-backport/1.15 labels Jan 27, 2025
giorio94 pushed a commit that referenced this pull request Jan 27, 2025
[ upstream commit 69c1d74 ]

The commit removes EKS cluster mixed node support restriction since
egress masquerade supports multiple interfaces (see fix: #36103).

Signed-off-by: viktor-kurchenko <viktor.kurchenko@isovalent.com>
Signed-off-by: Marco Iorio <marco.iorio@isovalent.com>
aanm pushed a commit that referenced this pull request Jan 28, 2025
[ upstream commit 69c1d74 ]

The commit removes EKS cluster mixed node support restriction since
egress masquerade supports multiple interfaces (see fix: #36103).

Signed-off-by: viktor-kurchenko <viktor.kurchenko@isovalent.com>
Signed-off-by: Marco Iorio <marco.iorio@isovalent.com>
aanm pushed a commit that referenced this pull request Jan 29, 2025
[ upstream commit 69c1d74 ]

The commit removes EKS cluster mixed node support restriction since
egress masquerade supports multiple interfaces (see fix: #36103).

Signed-off-by: viktor-kurchenko <viktor.kurchenko@isovalent.com>
Signed-off-by: Marco Iorio <marco.iorio@isovalent.com>
@github-actions github-actions bot added backport-done/1.15 The backport for Cilium 1.15.x for this PR is done. and removed backport-pending/1.15 The backport for Cilium 1.15.x for this PR is in progress. labels Jan 29, 2025
github-merge-queue bot pushed a commit that referenced this pull request Jan 30, 2025
[ upstream commit 69c1d74 ]

The commit removes EKS cluster mixed node support restriction since
egress masquerade supports multiple interfaces (see fix: #36103).

Signed-off-by: viktor-kurchenko <viktor.kurchenko@isovalent.com>
Signed-off-by: Marco Iorio <marco.iorio@isovalent.com>
github-merge-queue bot pushed a commit that referenced this pull request Jan 30, 2025
[ upstream commit 69c1d74 ]

The commit removes EKS cluster mixed node support restriction since
egress masquerade supports multiple interfaces (see fix: #36103).

Signed-off-by: viktor-kurchenko <viktor.kurchenko@isovalent.com>
Signed-off-by: Marco Iorio <marco.iorio@isovalent.com>
jongj pushed a commit to jongj/cilium that referenced this pull request Feb 11, 2025
The commit removes EKS cluster mixed node support restriction since
egress masquerade supports multiple interfaces (see fix: cilium#36103).

Signed-off-by: viktor-kurchenko <viktor.kurchenko@isovalent.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/datapath Impacts bpf/ or low-level forwarding details, including map management and monitor messages. area/iptables Impacts how Cilium interacts with iptables. backport-done/1.15 The backport for Cilium 1.15.x for this PR is done. backport-done/1.16 The backport for Cilium 1.16.x for this PR is done. ready-to-merge This PR has passed all tests and received consensus from code owners to merge. release-note/bug This PR fixes an issue in a previous release of Cilium.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants