-
Notifications
You must be signed in to change notification settings - Fork 3.4k
aws: change IPv4Masquerade to false in ENI mode #38663
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
aws: change IPv4Masquerade to false in ENI mode #38663
Conversation
/test |
a3a514f
to
17ef921
Compare
/test |
8faeb33
to
045622a
Compare
/test |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is this change necessary?
If we don't need to enable masquerading, then I assume we don't need to discuss which interfaces you want to apply masquerading to. |
Sorry I was not clear, this is precisely my question: Why do we disable masquerading? You submit a PR to turn it off, but there's no explanation why. What's the objective, here? |
Ah sorry I hadn't paid attention to the linked issue, now I see. Could you please summarise the reason for the change in the commit description? So that we understand the motivation when looking at the Git history in the future. |
Sure. I thought I should put more why in the issue to discuss and put what change this is in the PR commits. I will change that and I'm waiting for more feedback to push my local commits |
It makes sense to have the full context in the issue. My recommendation would be to still provide at least a brief in summary in the PR and commit description, because that's where we'll be looking at when trying to understand the reason for the change.
Ideally, it should be easy to understand what changes by looking at the code. The part that is not present in the diff is why it changes - that's why it's also important to motivate the change in the description 🙂
Sure, thanks! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
one minor detail around defaulting, but looks good otherwise!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good, thank you! Some formatting suggestions.
Thanks and will push those in my next commit |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@liyihuang Nice work!
beeef19
to
fca7198
Compare
/test |
Please note that you have a merge conflict to address, on the upgrade notes. |
fca7198
to
aa6d0a7
Compare
In the ENI mode, all the pods IP address are routable in the VPC. By default we still have IPv4Masquerade enabled, but we have the following iptable rules to skip the SNAT for the VPC traffic. -A CILIUM_POST_nat ! -d 192.168.0.0/16 -o eth+ -m comment --comment "cilium masquerade non-cluster" -j MASQUERADE This commit will just disable the IPv4Masquerade in the ENI mode by default, we expect that AWS NAT gateway will perform SNAT when the traffic accessing the Internet. - Change the helm values of IPv4Masquerade from true to false in ENI mode - Update the upgradeCompatibility so users could have smooth upgrade - Remove the IPv4Masquerade in the doc and remove egressMasqueradeInterfaces config. - Remove the IPv4Masquerade in the cilium-cli in ENI mode and remove egressMasqueradeInterfaces config. - Update the IPv4Masquerade value type to be bool or null and use null in the helm value as the default - Update the upgrade doc Signed-off-by: Liyi Huang <liyi.huang@isovalent.com>
aa6d0a7
to
e769ea7
Compare
/test |
We met some problem on public nodes. When I upgrade to cilium 1.18, I see errors in cilium console that enableIPv4Masquerade was not set, and I see this issue, so I just removed bpf.masquerade=true, then pods on public nodes cannot access Internet anymore while pods on private nodes work well. May I ask what could be the problem? |
I dont think there is any problem and it's the default behaviour change just for enableIPv4Masquerade. Before the change I checked Cilium and AWS doc, I can see they suggest or assume that the worker nodes are in the private subnets. Before this change, I can see the users having the confusion since they have enabled the Masqueraded but found some traffic got SNAT(VPC to external) but some not(traffic within the VPC). I think it's more confusing and that's why I proposed this change. |
In the ENI mode, all the pods IP address are routable in the VPC.
By default we still have IPv4Masquerade enabled, but we have the following iptable rules
to skip the SNAT for the VPC traffic.
-A CILIUM_POST_nat ! -d 192.168.0.0/16 -o eth+ -m comment --comment "cilium masquerade non-cluster" -j MASQUERADE
This commit will just disable the IPv4Masquerade in the ENI mode by default, we expect that
AWS NAT gateway will perform SNAT when the traffic accessing the Internet.
egressMasqueradeInterfaces config.
Fixes: #38661