-
Notifications
You must be signed in to change notification settings - Fork 3.4k
Closed
Labels
area/datapathImpacts bpf/ or low-level forwarding details, including map management and monitor messages.Impacts bpf/ or low-level forwarding details, including map management and monitor messages.kind/bugThis is a bug in the Cilium logic.This is a bug in the Cilium logic.kind/community-reportThis was reported by a user in the Cilium community, eg via Slack.This was reported by a user in the Cilium community, eg via Slack.kind/complexity-issueRelates to BPF complexity or program size issuesRelates to BPF complexity or program size issuesneeds/triageThis issue requires triaging to establish severity and next steps.This issue requires triaging to establish severity and next steps.
Description
Is there an existing issue for this?
- I have searched the existing issues
Version
equal or higher than v1.17.2 and lower than v1.18.0
What happened?
When I enable host-firewall in existing cluster I get "BPF program is too large" error in cilium pods. I updated Cilium to the latest (1.17.3) but still get the error. Turning off host-firewall fixes issue.
Our cilium config is bellow. Are there any known config incompatibilities with host-firewall that needs to be tweaked to get BPF program size bellow limit?
How can we reproduce the issue?
Use cilium configmap:
$ sudo kubectl get cm cilium-config -n kube-system -o yaml
apiVersion: v1
data:
agent-not-ready-taint-key: node.cilium.io/agent-not-ready
allow-localhost: policy
arping-refresh-period: 30s
auto-direct-node-routes: "false"
bgp-secrets-namespace: kube-system
bpf-events-drop-enabled: "true"
bpf-events-policy-verdict-enabled: "true"
bpf-events-trace-enabled: "true"
bpf-lb-acceleration: disabled
bpf-lb-algorithm-annotation: "false"
bpf-lb-external-clusterip: "false"
bpf-lb-map-max: "65536"
bpf-lb-mode-annotation: "false"
bpf-lb-sock: "false"
bpf-lb-source-range-all-types: "false"
bpf-map-dynamic-size-ratio: "0.0025"
bpf-policy-map-max: "16384"
bpf-root: /sys/fs/bpf
cgroup-root: /sys/fs/cgroup
cilium-endpoint-gc-interval: 5m0s
cluster-id: "0"
cluster-name: kube-gva2-lab-3-management
cluster-pool-ipv4-cidr: 172.20.104.0/22
cluster-pool-ipv4-mask-size: "26"
cluster-pool-ipv6-cidr: 2a04:c43:e00:147a:500:c::/97
cluster-pool-ipv6-mask-size: "104"
clustermesh-enable-endpoint-sync: "false"
clustermesh-enable-mcs-api: "false"
cni-exclusive: "true"
cni-log-file: /var/run/cilium/cilium-cni.log
controller-group-metrics: write-cni-file sync-host-ips sync-lb-maps-with-k8s-services
custom-cni-conf: "false"
datapath-mode: veth
debug: "false"
default-lb-service-ipam: lbipam
direct-routing-skip-unreachable: "false"
dnsproxy-enable-transparent-mode: "true"
dnsproxy-socket-linger-timeout: "10"
egress-gateway-reconciliation-trigger-interval: 1s
enable-auto-protect-node-port-range: "true"
enable-bgp-control-plane: "true"
enable-bgp-control-plane-status-report: "true"
enable-bpf-clock-probe: "false"
enable-bpf-masquerade: "true"
enable-bpf-tproxy: "true"
enable-endpoint-health-checking: "true"
enable-endpoint-lockdown-on-policy-overflow: "false"
enable-experimental-lb: "false"
enable-health-check-loadbalancer-ip: "false"
enable-health-check-nodeport: "true"
enable-health-checking: "true"
enable-host-firewall: "true"
enable-hubble: "true"
enable-internal-traffic-policy: "true"
enable-ipv4: "true"
enable-ipv4-big-tcp: "false"
enable-ipv4-masquerade: "true"
enable-ipv6: "true"
enable-ipv6-big-tcp: "false"
enable-ipv6-masquerade: "true"
enable-k8s-networkpolicy: "true"
enable-k8s-terminating-endpoint: "true"
enable-l2-neigh-discovery: "true"
enable-l7-proxy: "true"
enable-lb-ipam: "true"
enable-local-redirect-policy: "false"
enable-masquerade-to-route-source: "false"
enable-metrics: "true"
enable-node-selector-labels: "false"
enable-non-default-deny-policies: "true"
enable-pmtu-discovery: "true"
enable-policy: always
enable-policy-secrets-sync: "true"
enable-runtime-device-detection: "true"
enable-sctp: "false"
enable-source-ip-verification: "true"
enable-svc-source-range-check: "true"
enable-tcx: "true"
enable-vtep: "false"
enable-well-known-identities: "false"
enable-xt-socket-fallback: "true"
envoy-access-log-buffer-size: "4096"
envoy-base-id: "0"
envoy-keep-cap-netbindservice: "false"
external-envoy-proxy: "false"
health-check-icmp-failure-threshold: "3"
http-retry-count: "3"
hubble-disable-tls: "false"
hubble-export-file-max-backups: "5"
hubble-export-file-max-size-mb: "10"
hubble-listen-address: :4244
hubble-socket-path: /var/run/cilium/hubble.sock
hubble-tls-cert-file: /var/lib/cilium/tls/hubble/server.crt
hubble-tls-client-ca-files: /var/lib/cilium/tls/hubble/client-ca.crt
hubble-tls-key-file: /var/lib/cilium/tls/hubble/server.key
identity-allocation-mode: crd
identity-gc-interval: 15m0s
identity-heartbeat-timeout: 30m0s
install-no-conntrack-iptables-rules: "false"
ipam: cluster-pool
ipam-cilium-node-update-rate: 15s
iptables-random-fully: "false"
k8s-require-ipv4-pod-cidr: "false"
k8s-require-ipv6-pod-cidr: "false"
kube-proxy-replacement: "true"
kube-proxy-replacement-healthz-bind-address: ""
max-connected-clusters: "255"
mesh-auth-enabled: "true"
mesh-auth-gc-interval: 5m0s
mesh-auth-queue-size: "1024"
mesh-auth-rotated-identities-queue-size: "1024"
monitor-aggregation: medium
monitor-aggregation-flags: all
monitor-aggregation-interval: 5s
nat-map-stats-entries: "32"
nat-map-stats-interval: 30s
node-port-bind-protection: "true"
nodes-gc-interval: 5m0s
operator-api-serve-addr: 127.0.0.1:9234
operator-prometheus-serve-addr: :9963
policy-audit-mode: "false"
policy-cidr-match-mode: ""
policy-secrets-namespace: kube-system
policy-secrets-only-from-secrets-namespace: "true"
preallocate-bpf-maps: "false"
procfs: /host/proc
prometheus-serve-addr: :9962
proxy-connect-timeout: "2"
proxy-idle-timeout-seconds: "60"
proxy-initial-fetch-timeout: "30"
proxy-max-concurrent-retries: "128"
proxy-max-connection-duration-seconds: "0"
proxy-max-requests-per-connection: "0"
proxy-prometheus-port: "9964"
proxy-xff-num-trusted-hops-egress: "0"
proxy-xff-num-trusted-hops-ingress: "0"
remove-cilium-node-taints: "true"
routing-mode: tunnel
service-no-backend-response: reject
set-cilium-is-up-condition: "true"
set-cilium-node-taints: "true"
synchronize-k8s-nodes: "true"
tofqdns-dns-reject-response-code: refused
tofqdns-enable-dns-compression: "true"
tofqdns-endpoint-max-ip-per-hostname: "1000"
tofqdns-idle-connection-grace-period: 0s
tofqdns-max-deferred-connection-deletes: "10000"
tofqdns-proxy-response-max-delay: 100ms
tunnel-protocol: vxlan
unmanaged-pod-watcher-interval: "15"
vtep-cidr: ""
vtep-endpoint: ""
vtep-mac: ""
vtep-mask: ""
write-cni-conf-when-ready: /host/etc/cni/net.d/05-cilium.conflist
Cilium Version
1.17.3
Kernel Version
5.15.0-130-generic
Kubernetes Version
1.32.3
Regression
No response
Sysdump
No response
Relevant log output
level=error msg="Error while reloading endpoint BPF program" ciliumEndpointName=/ containerID= containerInterface= datapathPolicyRevision=0 desiredPoli
cyRevision=1535 endpointID=3799 error="attaching cilium_host: loading eBPF collection into the kernel: program tail_nodeport_nat_ingress_ipv4: load program: argument list too long: BPF prog
ram is too large. Processed 1000001 insn (1259 line(s) omitted)" identity=1 ipv4= ipv6= k8sPodName=/ subsys=endpoint
Anything else?
No response
Cilium Users Document
- Are you a user of Cilium? Please add yourself to the Users doc
Code of Conduct
- I agree to follow this project's Code of Conduct
Metadata
Metadata
Assignees
Labels
area/datapathImpacts bpf/ or low-level forwarding details, including map management and monitor messages.Impacts bpf/ or low-level forwarding details, including map management and monitor messages.kind/bugThis is a bug in the Cilium logic.This is a bug in the Cilium logic.kind/community-reportThis was reported by a user in the Cilium community, eg via Slack.This was reported by a user in the Cilium community, eg via Slack.kind/complexity-issueRelates to BPF complexity or program size issuesRelates to BPF complexity or program size issuesneeds/triageThis issue requires triaging to establish severity and next steps.This issue requires triaging to establish severity and next steps.