CiliumNetworkPolicy egressDeny blocking non-matching traffic #28136
Description
Is there an existing issue for this?
- I have searched the existing issues
What happened?
In a cluster with the following CiliumNetworkPolicy:
apiVersion: "cilium.io/v2"
kind: CiliumNetworkPolicy
metadata:
name: "combined"
spec:
endpointSelector:
matchLabels: {}
egressDeny:
- toPorts:
- ports:
- port: "1025"
egress:
- toPorts:
- ports:
- port: "1025"
protocol: TCP
- port: "1026"
protocol: TCP
egress traffic to port 1026 is blocked (see monitor log below) even if it's not listed in the egressDeny and it has an explicit allow policy.
My expectation would have been for egress traffic to port 1026 to be allowed, egress traffic to port 1025 to be denied (because of the egressDeny) and egress to all other ports to be denied (because of the default deny)
Cilium Version
1.14.1
Kernel Version
5.15.72
Kubernetes Version
Client Version: version.Info{Major:"1", Minor:"25", GitVersion:"v1.25.2", GitCommit:"5835544ca568b757a8ecae5c153f317e5736700e", GitTreeState:"clean", BuildDate:"2022-09-21T14:33:49Z", GoVersion:"go1.19.1", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v4.5.7
Server Version: version.Info{Major:"1", Minor:"27", GitVersion:"v1.27.3", GitCommit:"25b4e43193bcda6c7328a6d147b1fb73a33f1598", GitTreeState:"clean", BuildDate:"2023-06-14T09:47:40Z", GoVersion:"go1.20.5", Compiler:"gc", Platform:"linux/amd64"}
Sysdump
cilium-sysdump-20230912-052643.zip
Relevant log output
$ kubectl --kubeconfig ../sha-byoh-okta-test.kubeconfig -n kube-system logs cilium-j4cz2 -f
level=info msg="Policy Add Request" ciliumNetworkPolicy="[&{EndpointSelector:{\"matchLabels\":{\"k8s:io.kubernetes.pod.namespace\":\"default\"}} NodeSelector:{} Ingress:[] IngressDeny:[] Egress:[{EgressCommonRule:{ToEndpoints:[] ToRequires:[] ToCIDR: ToCIDRSet:[] ToEntities:[] ToServices:[] ToGroups:[] aggregatedSelectors:[]} ToPorts:[{Ports:[{Port:1025 Protocol:TCP} {Port:1026 Protocol:TCP}] TerminatingTLS:<nil> OriginatingTLS:<nil> ServerNames:[] Listener:<nil> Rules:<nil>}] ToFQDNs:[] ICMPs:[] Authentication:<nil>}] EgressDeny:[{EgressCommonRule:{ToEndpoints:[] ToRequires:[] ToCIDR: ToCIDRSet:[] ToEntities:[] ToServices:[] ToGroups:[] aggregatedSelectors:[]} ToPorts:[{Ports:[{Port:1025 Protocol:}]}] ICMPs:[]}] Labels:[k8s:io.cilium.k8s.policy.derived-from=CiliumNetworkPolicy k8s:io.cilium.k8s.policy.name=combined k8s:io.cilium.k8s.policy.namespace=default k8s:io.cilium.k8s.policy.uid=451b0e14-5d4d-4682-bc06-e7669565f7dc] Description:}]" policyAddRequest=14e7817c-582e-49b2-8680-a3fcfeb4ac6f subsys=daemon
level=info msg="Policy imported via API, recalculating..." policyAddRequest=14e7817c-582e-49b2-8680-a3fcfeb4ac6f policyRevision=4 subsys=daemon
level=info msg="Imported CiliumNetworkPolicy" ciliumNetworkPolicyName=combined k8sApiVersion= k8sNamespace=default subsys=k8s-watcher
level=info msg="Rewrote endpoint BPF program" containerID=c425fcdf8c datapathPolicyRevision=3 desiredPolicyRevision=4 endpointID=1243 identity=44612 ipv4=10.0.1.68 ipv6= k8sPodName=default/debian subsys=endpoint
level=warning msg="Policy map sync fixed errors, consider running with debug verbose = policy to get detailed dumps" containerID=c425fcdf8c count=1 datapathPolicyRevision=4 desiredPolicyRevision=4 endpointID=1243 identity=44612 ipv4=10.0.1.68 ipv6= k8sPodName=default/debian subsys=endpoint
level=info msg="Conntrack garbage collector interval recalculated" deleteRatio=0.0012766375331434744 newInterval=11m15s subsys=map-ct
# hubble observe --namespace default --follow
Sep 11 13:23:21.164: default/debian:57876 (ID:44612) <> 10.20.73.59:1025 (world) policy-verdict:L4-Only EGRESS DENIED (TCP Flags: SYN)
Sep 11 13:23:21.164: default/debian:57876 (ID:44612) <> 10.20.73.59:1025 (world) Policy denied by denylist DROPPED (TCP Flags: SYN)
Sep 11 13:23:22.183: default/debian:57876 (ID:44612) <> 10.20.73.59:1025 (world) policy-verdict:L4-Only EGRESS DENIED (TCP Flags: SYN)
Sep 11 13:23:22.183: default/debian:57876 (ID:44612) <> 10.20.73.59:1025 (world) Policy denied by denylist DROPPED (TCP Flags: SYN)
Sep 11 13:23:24.689: default/debian:38284 (ID:44612) <> 10.20.73.59:1026 (world) policy-verdict:L4-Only EGRESS DENIED (TCP Flags: SYN)
Sep 11 13:23:24.689: default/debian:38284 (ID:44612) <> 10.20.73.59:1026 (world) Policy denied by denylist DROPPED (TCP Flags: SYN)
Sep 11 13:23:25.703: default/debian:38284 (ID:44612) <> 10.20.73.59:1026 (world) policy-verdict:L4-Only EGRESS DENIED (TCP Flags: SYN)
Sep 11 13:23:25.703: default/debian:38284 (ID:44612) <> 10.20.73.59:1026 (world) Policy denied by denylist DROPPED (TCP Flags: SYN)
# cilium bpf policy get 1243 -n
POLICY DIRECTION IDENTITY PORT/PROTO PROXY PORT AUTH TYPE BYTES PACKETS PREFIX
Deny Egress 0 ANY NONE disabled 1628 22 0
Allow Ingress 0 ANY NONE disabled 0 0 0
Allow Ingress 1 ANY NONE disabled 0 0 0Anything else?
No response
Code of Conduct
- I agree to follow this project's Code of Conduct