bpf: service loopback for ipv6 #39594

saiaunghlyanhtet · 2025-05-17T12:29:30Z

Service Loopback IPv6

bpf/lib/lb.h

pkg/node/local_node_store.go

bpf/include/bpf/config/node.h

saiaunghlyanhtet · 2025-06-08T08:12:47Z

@jrife I have changed to serivce_loopback_ipv6. If you have free time, can you have a look at it?. Currently, I am stuck at the condition in which source ip is not translated back to the service cluster IP. I think that the reverse NAT for the reply path is not being applied correctly. But, I have no idea which is going wrong.

The logs show multiple SYN packets from [fd00:10:244:1::dc8e]:53060 to [fd00:10:96::cd54]:80, indicating that the client is retransmitting SYN packets, likely because it is not receiving properly formatted replies.

$ k exec -ti -n kube-system cilium-qz7tt -- cilium-dbg monitor --related-to 542

Listening for events on 8 CPUs with 64x4096 of shared memory
Press Ctrl-C to quit
time=2025-06-08T07:43:34.15455902Z level=info msg="Initializing dissection cache..."
<- endpoint 542 flow 0x70e2ab0d , identity 20084->unknown state unknown ifindex 0 orig-ip 0.0.0.0: [fd00:10:244:1::dc8e]:53060 -> [fd00:10:96::cd54]:80 tcp SYN
-> endpoint 542 flow 0x70e2ab0d , identity 20084->20084 state new ifindex lxc14f4f090a3e0 orig-ip fd00::1: [fd00::1]:53060 -> [fd00:10:244:1::dc8e]:80 tcp SYN
<- endpoint 542 flow 0xcfd94455 , identity 20084->unknown state unknown ifindex 0 orig-ip 0.0.0.0: [fd00:10:244:1::dc8e]:80 -> [fd00::1]:53060 tcp SYN, ACK
-> endpoint 542 flow 0xcfd94455 , identity 20084->20084 state reply ifindex lxc14f4f090a3e0 orig-ip fd00:10:244:1::dc8e: [fd00:10:244:1::dc8e]:80 -> [fd00::1]:53060 tcp SYN, ACK
<- endpoint 542 flow 0xc34aebbb , identity 20084->unknown state unknown ifindex 0 orig-ip 0.0.0.0: [fd00:10:244:1::dc8e]:53060 -> [fd00:10:96::cd54]:80 tcp SYN
-> endpoint 542 flow 0xc34aebbb , identity 20084->20084 state established ifindex lxc14f4f090a3e0 orig-ip fd00::1: [fd00::1]:53060 -> [fd00:10:244:1::dc8e]:80 tcp SYN
<- endpoint 542 flow 0x8c902078 , identity 20084->unknown state unknown ifindex 0 orig-ip 0.0.0.0: [fd00:10:244:1::dc8e]:80 -> [fd00::1]:53060 tcp SYN, ACK
-> endpoint 542 flow 0x8c902078 , identity 20084->20084 state reply ifindex lxc14f4f090a3e0 orig-ip fd00:10:244:1::dc8e: [fd00:10:244:1::dc8e]:80 -> [fd00::1]:53060 tcp SYN, ACK
<- endpoint 542 flow 0x15fde42f , identity 20084->unknown state unknown ifindex 0 orig-ip 0.0.0.0: [fd00:10:244:1::dc8e]:80 -> [fd00::1]:53060 tcp SYN, ACK
-> endpoint 542 flow 0x15fde42f , identity 20084->20084 state reply ifindex lxc14f4f090a3e0 orig-ip fd00:10:244:1::dc8e: [fd00:10:244:1::dc8e]:80 -> [fd00::1]:53060 tcp SYN, ACK
<- endpoint 542 flow 0xfb86fd56 , identity 20084->unknown state unknown ifindex 0 orig-ip 0.0.0.0: [fd00:10:244:1::dc8e]:53060 -> [fd00:10:96::cd54]:80 tcp SYN
-> endpoint 542 flow 0xfb86fd56 , identity 20084->20084 state established ifindex lxc14f4f090a3e0 orig-ip fd00::1: [fd00::1]:53060 -> [fd00:10:244:1::dc8e]:80 tcp SYN
<- endpoint 542 flow 0xf6a6963b , identity 20084->unknown state unknown ifindex 0 orig-ip 0.0.0.0: [fd00:10:244:1::dc8e]:80 -> [fd00::1]:53060 tcp SYN, ACK
-> endpoint 542 flow 0xf6a6963b , identity 20084->20084 state reply ifindex lxc14f4f090a3e0 orig-ip fd00:10:244:1::dc8e: [fd00:10:244:1::dc8e]:80 -> [fd00::1]:53060 tcp SYN, ACK
<- endpoint 542 flow 0x3f4fed5 , identity 20084->unknown state unknown ifindex 0 orig-ip 0.0.0.0: [fd00:10:244:1::dc8e]:53060 -> [fd00:10:96::cd54]:80 tcp SYN
-> endpoint 542 flow 0x3f4fed5 , identity 20084->20084 state established ifindex lxc14f4f090a3e0 orig-ip fd00::1: [fd00::1]:53060 -> [fd00:10:244:1::dc8e]:80 tcp SYN
<- endpoint 542 flow 0xce35e2bf , identity 20084->unknown state unknown ifindex 0 orig-ip 0.0.0.0: [fd00:10:244:1::dc8e]:80 -> [fd00::1]:53060 tcp SYN, ACK
-> endpoint 542 flow 0xce35e2bf , identity 20084->20084 state reply ifindex lxc14f4f090a3e0 orig-ip fd00:10:244:1::dc8e: [fd00:10:244:1::dc8e]:80 -> [fd00::1]:53060 tcp SYN, ACK
<- endpoint 542 flow 0x72c6a27e , identity 20084->unknown state unknown ifindex 0 orig-ip 0.0.0.0: [fd00:10:244:1::dc8e]:80 -> [fd00::1]:53060 tcp SYN, ACK
-> endpoint 542 flow 0x72c6a27e , identity 20084->20084 state reply ifindex lxc14f4f090a3e0 orig-ip fd00:10:244:1::dc8e: [fd00:10:244:1::dc8e]:80 -> [fd00::1]:53060 tcp SYN, ACK
<- endpoint 542 flow 0x0 , identity 20084->unknown state unknown ifindex 0 orig-ip 0.0.0.0: fe80::8034:dfff:fe01:1dbf -> fd00:10:244:1::6ec8 icmp NeighborSolicitation

Documentation/cmdref/cilium-agent.md

bpf/bpf_lxc.c

bpf/lib/lb.h

bpf/tests/tc_nodeport_test.c

pkg/datapath/loader/config.go

jrife · 2025-06-09T19:45:11Z

Currently, I am stuck at the condition in which source ip is not translated back to the service cluster IP. I think that the reverse NAT for the reply path is not being applied correctly. But, I have no idea which is going wrong.

Off the top of my head, I'm not sure. I may have some time later this week to look at it in a bit more detail. In the meantime, I left a few comments. I'd recommend fixing whatever unit tests and checkpatch checks are failing and try testing again. The checkpatch stuff is likely similar to some of the formatting stuff I commented on.

saiaunghlyanhtet · 2025-07-26T06:38:07Z

/test

saiaunghlyanhtet · 2025-07-29T08:20:48Z

/test

saiaunghlyanhtet · 2025-07-29T13:14:36Z

/test

saiaunghlyanhtet · 2025-07-30T23:52:09Z

/test

asauber · 2025-08-07T04:51:17Z

Are you able to see the output of the checkpatch workflow? You have some lines >100 chars which is causing it to fail.

saiaunghlyanhtet · 2025-08-07T07:14:25Z

Are you able to see the output of the checkpatch workflow? You have some lines >100 chars which is causing it to fail.

I have fixed some warnings. But, I think the warning in node.h cannot be solved.

asauber · 2025-08-08T01:10:17Z

This edit would make checkpatch happy

diff --git a/bpf/include/bpf/config/node.h b/bpf/include/bpf/config/node.h
index 8f4d647cbe..048fde6229 100644
--- a/bpf/include/bpf/config/node.h
+++ b/bpf/include/bpf/config/node.h
@@ -15,10 +15,16 @@
 /* Legacy node config rendered at agent runtime. */
 #include <node_config.h>

-NODE_CONFIG(union v4addr, service_loopback_ipv4, "IPv4 source address used for SNAT when a Pod talks to itself over a Service")
-NODE_CONFIG(union v6addr, router_ipv6, "Internal IPv6 router address assigned to the cilium_host interface")
+NODE_CONFIG(union v4addr, service_loopback_ipv4,
+           "IPv4 source address used for SNAT when a Pod talks to itself over a Service")
+NODE_CONFIG(union v6addr, service_loopback_ipv6,
+           "IPv6 source address used for SNAT when a Pod talks to itself over a Service")
+NODE_CONFIG(union v6addr, router_ipv6,
+           "Internal IPv6 router address assigned to the cilium_host interface")

-NODE_CONFIG(__u32, trace_payload_len, "Length of payload to capture when tracing native packets.")
+NODE_CONFIG(__u32, trace_payload_len,
+           "Length of payload to capture when tracing native packets.")
 #define TRACE_PAYLOAD_LEN CONFIG(trace_payload_len) /* Backwards compatibility */

-NODE_CONFIG(__u32, trace_payload_len_overlay, "Length of payload to capture when tracing overlay packets.")
+NODE_CONFIG(__u32, trace_payload_len_overlay,
+           "Length of payload to capture when tracing overlay packets.")

thorn3r

Approving for the agent-related work, it looks good 👍 I think a more detailed commit message, and even splitting the datapath and agent changes into separate commits, would be nice

jrife · 2025-08-13T20:40:59Z

bpf/include/bpf/config/node.h

-NODE_CONFIG(__u32, trace_payload_len, "Length of payload to capture when tracing native packets.")
+NODE_CONFIG(union v4addr, service_loopback_ipv4,
+	    "IPv4 source address used for SNAT when a Pod talks to itself over a Service")
+NODE_CONFIG(union v6addr, service_loopback_ipv6,


nit: checkpatch would probably be happy if you just made this adjustment to the lines you added. My preference would be to only change what you need for this PR and leave reformatting the other lines for another patch, but not a blocker.

Fixed it. I think that the current checkpatch failed because of unknown manifest issue related t checkpatch image.

jrife · 2025-08-13T20:41:39Z

/test

asauber · 2025-08-19T05:00:20Z

pkg/option/config.go

@@ -773,6 +773,9 @@ const (
 	// ServiceLoopbackIPv4 is the address to use for service loopback SNAT
 	ServiceLoopbackIPv4 = "ipv4-service-loopback-address"

+	// ServiceLoopbackIPv6 is the address to use for service loopback SNAT
+	ServiceLoopbackIPv6 = "ipv6-service-loopback-address"


Sorry for the delay on this comment. We prefer that all new config options are added to a hive cell, to begin the process of modularizing the related feature. If you take a look at the history on this file, you will see some recent example of refactors into cells. The initial refactor can be purely for the sake of config, with other modularlization coming later. Could you look into making this change?

To make it clear, I would like to confirm the cell file to which I need to move the current config. Is it ok to move to datapath/loader/cell.go? Or other file?

Logically, pkg/ipam/cell/cell.go seems like the right place, but given the breadth of changes that are required in this PR, that might not be feasible on the dependency graph. Could you give that a try and see if you run into any issues?

@joamaki is there a good way to visualize the dependency graph in order to make this type of decision, or any other advice that you would provide here?

Since we already have ServiceLoopbackIPv4 here I'd be OK with adding it here for now and then moving them both into a saner place in another PR. It'd probably make sense to do that as part of a bigger refactoring that moves daemon/cmd/ipam.go into pkg/ipam. @cilium/sig-ipam any opinion?

asauber

Approving for config. Given the above discussion, it seems like a separate refactor would be the place to move the flag.

Add agent-side support to enable IPv6 service loopback handling so that connections to a service’s own ClusterIP from a backend pod loop back correctly. Summary of changes: - Extend service / LB logic to recognize and mark IPv6 loopback scenarios - Introduce service_loopback_ipv6 runtime variable for IPv6 service loopback support Signed-off-by: saiaunghlyanhtet <saiaunghlyanhtet2003@gmail.com>

Introduce datapath logic to support IPv6 service loopback so that packets originating from a pod targeting its own service ClusterIP are correctly short-circuited and subject to the standard LB/rev-NAT path without external detours. Summary of changes: - Adjust service lookup / backend selection and reverse NAT handling for IPv6 loopback - Add BPF selftests to cover IPv6 loopback Signed-off-by: saiaunghlyanhtet <saiaunghlyanhtet2003@gmail.com>

maintainer-s-little-helper bot added the dont-merge/needs-release-note-label The author needs to describe the release impact of these changes. label May 17, 2025

saiaunghlyanhtet force-pushed the pr/ipv6-service-loopback branch 2 times, most recently from c6b7c0b to 1052f70 Compare May 17, 2025 14:18

julianwiedmann reviewed May 19, 2025

View reviewed changes

bpf/lib/lb.h Outdated Show resolved Hide resolved

bpf/lib/lb.h Outdated Show resolved Hide resolved

pkg/node/local_node_store.go Outdated Show resolved Hide resolved

saiaunghlyanhtet force-pushed the pr/ipv6-service-loopback branch 7 times, most recently from b91865f to 62fa8cf Compare May 27, 2025 14:22

julianwiedmann added release-note/minor This PR changes functionality that users may find relevant to operating Cilium. feature/ipv6 Relates to IPv6 protocol support area/loadbalancing Impacts load-balancing and Kubernetes service implementations labels May 28, 2025

maintainer-s-little-helper bot removed the dont-merge/needs-release-note-label The author needs to describe the release impact of these changes. label May 28, 2025

julianwiedmann added area/loader Impacts the loading of BPF programs into the kernel. area/datapath Impacts bpf/ or low-level forwarding details, including map management and monitor messages. labels May 28, 2025

jrife reviewed May 28, 2025

View reviewed changes

bpf/include/bpf/config/node.h Outdated Show resolved Hide resolved

saiaunghlyanhtet force-pushed the pr/ipv6-service-loopback branch 5 times, most recently from b16a408 to f056282 Compare June 1, 2025 09:10

saiaunghlyanhtet force-pushed the pr/ipv6-service-loopback branch 2 times, most recently from 31b0c2e to cb050e5 Compare June 8, 2025 01:24

saiaunghlyanhtet requested a review from jrife June 8, 2025 08:13

jrife reviewed Jun 9, 2025

View reviewed changes

saiaunghlyanhtet force-pushed the pr/ipv6-service-loopback branch from cb050e5 to 0850415 Compare June 14, 2025 09:16

saiaunghlyanhtet force-pushed the pr/ipv6-service-loopback branch from c617e76 to 79d7a22 Compare July 25, 2025 22:00

saiaunghlyanhtet requested review from jrife, Andreagit97 and julianwiedmann July 26, 2025 07:33

saiaunghlyanhtet force-pushed the pr/ipv6-service-loopback branch from 79d7a22 to 48dbbe7 Compare July 29, 2025 07:59

saiaunghlyanhtet force-pushed the pr/ipv6-service-loopback branch from 48dbbe7 to 16ee0b4 Compare July 29, 2025 12:52

ldelossa removed request for ldelossa and Andreagit97 July 30, 2025 16:03

saiaunghlyanhtet force-pushed the pr/ipv6-service-loopback branch from 16ee0b4 to f0c02ac Compare July 30, 2025 23:27

saiaunghlyanhtet force-pushed the pr/ipv6-service-loopback branch from f0c02ac to 3c2f808 Compare August 5, 2025 06:06

saiaunghlyanhtet force-pushed the pr/ipv6-service-loopback branch from 3c2f808 to 97641e7 Compare August 7, 2025 07:07

thorn3r approved these changes Aug 11, 2025

View reviewed changes

saiaunghlyanhtet force-pushed the pr/ipv6-service-loopback branch 2 times, most recently from 61c6480 to 63b343b Compare August 12, 2025 12:29

jrife reviewed Aug 13, 2025

View reviewed changes

asauber reviewed Aug 19, 2025

View reviewed changes

asauber self-requested a review August 19, 2025 06:14

asauber approved these changes Aug 21, 2025

View reviewed changes

saiaunghlyanhtet force-pushed the pr/ipv6-service-loopback branch from 63b343b to 2f3a33a Compare August 24, 2025 02:03

saiaunghlyanhtet added 2 commits August 24, 2025 11:18

saiaunghlyanhtet force-pushed the pr/ipv6-service-loopback branch from 2f3a33a to 2907909 Compare August 24, 2025 02:19

bpf: service loopback for ipv6 #39594

Are you sure you want to change the base?

bpf: service loopback for ipv6 #39594

Uh oh!

Conversation

saiaunghlyanhtet commented May 17, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

saiaunghlyanhtet commented Jun 8, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jrife commented Jun 9, 2025

Uh oh!

saiaunghlyanhtet commented Jul 26, 2025

Uh oh!

saiaunghlyanhtet commented Jul 29, 2025

Uh oh!

saiaunghlyanhtet commented Jul 29, 2025

Uh oh!

saiaunghlyanhtet commented Jul 30, 2025

Uh oh!

asauber commented Aug 7, 2025

Uh oh!

saiaunghlyanhtet commented Aug 7, 2025

Uh oh!

asauber commented Aug 8, 2025

Uh oh!

thorn3r left a comment

Choose a reason for hiding this comment

Uh oh!

jrife Aug 13, 2025

Choose a reason for hiding this comment

Uh oh!

saiaunghlyanhtet Aug 24, 2025

Choose a reason for hiding this comment

Uh oh!

jrife commented Aug 13, 2025

Uh oh!

asauber Aug 19, 2025

Choose a reason for hiding this comment

Uh oh!

saiaunghlyanhtet Aug 19, 2025

Choose a reason for hiding this comment

Uh oh!

asauber Aug 19, 2025

Choose a reason for hiding this comment

Uh oh!

joamaki Aug 19, 2025

Choose a reason for hiding this comment

Uh oh!

asauber left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!