Access to kube-apiserver from pod #39573
Description
Is there an existing issue for this?
- I have searched the existing issues
Version
equal or higher than v1.17.3 and lower than v1.18.0
What happened?
Condition
- Run curl image with out any network policy:
/ $ curl -k https://kubernetes.default.svc.cluster.local/
{
"kind": "Status",
"apiVersion": "v1",
"metadata": {},
"status": "Failure",
"message": "Unauthorized",
"reason": "Unauthorized",
"code": 401
}/ $
Test another servce in default namespace:
$ curl -k http://whoami.default.svc.cluster.local/
Hostname: whoami-57b9d799f-mpkrs
IP: 127.0.0.1
IP: ::1
IP: 172.16.6.61
IP: fe80::906b:9bff:fe4f:ecf7
RemoteAddr: 172.16.7.18:60032
GET / HTTP/1.1
Host: whoami.default.svc.cluster.local
User-Agent: curl/7.87.0-DEV
Accept: */*
/ $
- Create network policy with allow only kube-apiserver access:
Describe service kubernetes in default namespace
$ kubectl describe svc -n default kubernetes
Name: kubernetes
Namespace: default
Labels: component=apiserver
provider=kubernetes
Annotations: <none>
Selector: <none>
Type: ClusterIP
IP Family Policy: SingleStack
IP Families: IPv4
IP: 10.247.0.1
IPs: 10.247.0.1
Port: https 443/TCP
TargetPort: 6443/TCP
Endpoints: 172.22.37.111:6443,172.22.37.112:6443,172.22.37.113:6443
Session Affinity: None
Events: <none>
NetworkPolicy:
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: custom-np-curl
spec:
egress:
- to:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: default ### trying access kube-apiserver by namespace selector
- to:
- ipBlock:
cidr: 172.22.37.0/24 ### trying access to kube-apiserver over endpoint by wide mask
ports:
- port: 6443
protocol: TCP
- to:
- ipBlock:
cidr: 10.247.0.0/16 ### trying access to 10.247.0.1 by wide mask
ports:
- port: 443
protocol: TCP
- ports:
- port: 53
protocol: UDP
to:
- namespaceSelector: {}
- ipBlock:
cidr: 10.247.0.0/16
podSelector: {}
policyTypes:
- Egress
- Ingress
ingress:
- {}
Cilium Monitor to kube-apiserver:
Policy verdict log: flow 0xc65db137 local EP ID 2307, remote ID kube-apiserver, proto 6, egress, action deny, auth: disabled, match none, 172.16.7.18:53480 -> 172.22.37.113:6443 tcp SYN
xx drop (Policy denied) flow 0xc65db137 to endpoint 0, ifindex 81, file bpf_lxc.c:1360, , identity 4120->kube-apiserver: 172.16.7.18:53480 -> 172.22.37.113:6443 tcp SYN
Cilium Monitor to whoami-service:
Policy verdict log: flow 0x2d29dc90 local EP ID 2307, remote ID 11467, proto 6, egress, action allow, auth: disabled, match L3-Only, 172.16.7.18:46838 -> 172.16.6.61:80 tcp SYN
- But if create CiliumNetworkPolicy all work fine:
CiliumNetworkPolicy
apiVersion: "cilium.io/v2"
kind: CiliumNetworkPolicy
metadata:
name: "dev-to-kube-apiserver"
spec:
endpointSelector: {}
egress:
- toEntities:
- kube-apiserver
Ciilium Monitor
Policy verdict log: flow 0x4435df85 local EP ID 2307, remote ID kube-apiserver, proto 6, egress, action allow, auth: disabled, match L3-Only, 172.16.7.18:40198 -> 172.22.37.112:6443 tcp SYN
Question
Why dont work NetworkPolicy for kube-apiserver?
How can we reproduce the issue?
cilium install with helm values
k8sServiceHost: "localhost"
k8sServicePort: "7445"
hubble:
enabled: true
relay:
enabled: true
prometheus:
serviceMonitor:
enabled: true
ui:
enabled: true
ingress:
enabled: true
hosts:
- hubble.test.loc
operator:
prometheus:
serviceMonitor:
enabled: true
preflight:
envoy:
enabled: false
securityContext:
capabilities:
ciliumAgent:
- CHOWN
- KILL
- NET_ADMIN
- NET_RAW
- IPC_LOCK
- SYS_ADMIN
- SYS_RESOURCE
- DAC_OVERRIDE
- FOWNER
- SETGID
- SETUID
cleanCiliumState:
- NET_ADMIN
- SYS_ADMIN
- SYS_RESOURCE
l2announcements:
enabled: true
leaseDuration: 3s
leaseRenewDeadline: 1s
leaseRetryPeriod: 200ms
ipam:
mode: "kubernetes"
cgroup:
autoMount:
enabled: false
hostRoot: /sys/fs/cgroup
bpf:
hostLegacyRouting: true
devices: "eth+"
kubeProxyReplacement: "true"
externalIPs:
enabled: true
Cilium Version
cilium-1.17.1
Kernel Version
6.12.13-talos
Kubernetes Version
v1.30.4
Regression
No response
Sysdump
No response
Relevant log output
Anything else?
No response
Cilium Users Document
- Are you a user of Cilium? Please add yourself to the Users doc
Code of Conduct
- I agree to follow this project's Code of Conduct