-
Notifications
You must be signed in to change notification settings - Fork 3.4k
Description
Is there an existing issue for this?
- I have searched the existing issues
What happened?
With cilium 1.14 we are having troubles with IPv6 routing between nodes. Cilium adds this to the interface:
7: cilium_host@cilium_net: <BROADCAST,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 9a:aa:d1:1e:5c:dc brd ff:ff:ff:ff:ff:ff
inet 10.11.22.33/32 scope global cilium_host
valid_lft forever preferred_lft forever
inet6 2a02:aa:bb:8a00::2080:2bd/64 scope global
valid_lft forever preferred_lft forever
which also - by default - adds a route
2a02:aa:bb:8a00::2080:2b8 dev lxc_health proto kernel metric 1024 pref medium
2a02:aa:bb:8a00::2080:200/120 dev cilium_host proto kernel src 2a02:aa:bb:8a00::2080:2bd metric 1024 pref medium
2a02:aa:bb:8a00::/64 dev cilium_host proto kernel metric 256 pref medium
which crosses with our ip pool:
2a02:aa:bb:8a00::2080:0/105
meaning all the pod-to-pod traffic gets improperly routed.
It seems to work fine with Cilium 1.13. I do not have any kubernetes cluster by hand to check, but our openstack cilium 1.13 cluster has assigned IPv6 with /128
mask but also --ipv6-node
flag set.
There's noprefixroute
flag in netlink which seems to add the IP but not add the route on that interface, but I am struggling to build cilium outside of CI and it is Friday ( 🙃 ) so cannot test it until Monday. My idea to try out is this patch:
diff --git a/pkg/datapath/loader/netlink.go b/pkg/datapath/loader/netlink.go
index bfecce37f0..9b1ce01ced 100644
--- a/pkg/datapath/loader/netlink.go
+++ b/pkg/datapath/loader/netlink.go
@@ -435,6 +435,7 @@ func addHostDeviceAddr(hostDev netlink.Link, ipv4, ipv6 net.IP) error {
IP: ipv6,
Mask: net.CIDRMask(64, 128), // corresponds to /64
},
+ Flags: unix.IFA_F_NOPREFIXROUTE,
}
if err := netlink.AddrReplace(hostDev, &addr); err != nil {
if this works I will be happy to sent a PR
Cilium Version
1.14.1
Kernel Version
5.15.0-69-generic
Kubernetes Version
1.27.1
Sysdump
No response
Relevant log output
No response
Anything else?
No response
Code of Conduct
- I agree to follow this project's Code of Conduct