Cilium Feature Proposal

Describe the feature you'd like

Cilium supports several different tunnel mode: geneve and vxlan as 2 different supported tunnel, as well as disabled in the case of all the nodes are in the same L2 subnet, so the can be direct routed without tunneling.

The proposal is to support hybrid mode where cilium can detect if the source and destination node is in the same L2 subnet, and choose whether or not to route the packet through a tunnel. This will avoid additional tunneling overhead between 2 nodes in the same subnet where tunnel is not necessary, while in the meantime still support nodes across L2 subnets. It's similar to antrea's hybrid mode:

Besides the default Encap mode, in which Pod traffic across Nodes will be encapsulated and sent over tunnels, Antrea also supports NoEncap and Hybrid traffic modes. In NoEncap mode, Antrea does not encapsulate Pod traffic, but relies on the Node network to route the traffic across Nodes. In Hybrid mode, Antrea encapsulates Pod traffic when the source Node and the destination Node are in different subnets, but does not encapsulate when the source and the destination Nodes are in the same subnet. This document describes how to configure Antrea with the NoEncap and Hybrid modes.

(Optional) Describe your proposed solution

Please complete this section if you have ideas / suggestions on how to implement the feature. We strongly recommend discussing your approach with Cilium committers before spending lots of time implementing a change.

For longer proposals, you are welcome to link to an external doc (e.g. a Google doc). We have a Cilium Feature Proposal template to help you structure your proposal - if you would like to use it, please make a copy and ensure it's publicly visible, and then add the link here.