Proposal / RFE

Is your feature request related to a problem?
Yes

Describe the solution you'd like
Today cilium requires host reachable svc to enable bpf masquerading, the restriction is added in this commit.

The reason is that when hostns pods talking to clusterIP, kernel picks node IP as source IP but not cilium_host IP. The packets are still tunneled to remote backend. Due to #12544, the return packet is masqueraded on the remote node. The real fix should be letting kernel pick cilium_host IP for such traffic so that we have symmetric data path.

The feature is useful for kernel < 4.19 where people can enable bpf nodeport and masquerading while keep host-reachable-svc off in kubeproxy partial mode.

Proposal:
Passing a --cilium-host-route-cidr flag to cilium-agent and install a route based on that flag:

[cilium-host-route-cidr] via 192.168.4.111 dev cilium_host src 192.168.4.111 mtu 1450

To make the implementation simpler, maybe we could always install this route when --cilium-host-route-cidr is passed regardless of other flags. So for whoever want to enable bpf masquerading without host-reachable-svc, he needs to pass --cilium-host-route-cidr