Deploy konnectivity agent add-on as deployment #872

bsctl · 2025-07-11T17:13:50Z

bsctl
Jul 11, 2025
Maintainer

Currently, the konnectivity agent add-on is deployed as daemoset in the tenant cluster as suggested by official kubernetes documentation here. However, it could be deployed as deployment too.

This can be useful for saving computing resources in large clusters as only few agents are deployed instead to have one agent for each worker node.

This a simple test made on a tenant cluster:

Create a Tenant Cluster and join some worker nodes
Stop Kamaji to avoid the reconciliation
Delete the konnectivity agent daemonset from the tenant cluster and turn the daemonset into a deployment like this:

apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
  labels:
    kamaji.clastix.io/component: konnectivity-agent
    kamaji.clastix.io/name: roma
    kamaji.clastix.io/project: kamaji
  name: konnectivity-agent
  namespace: kube-system
spec:
  replicas: 2  # Set to desired number of agent instances
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      k8s-app: konnectivity-agent
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 1
  template:
    metadata:
      creationTimestamp: null
      labels:
        k8s-app: konnectivity-agent
    spec:
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 100
            podAffinityTerm:
              labelSelector:
                matchLabels:
                  k8s-app: konnectivity-agent
              topologyKey: kubernetes.io/hostname
      containers:
      - args:
        - --admin-server-port=8133
        - --ca-cert=/var/run/secrets/kubernetes.io/serviceaccount/ca.crt
        - --health-server-port=8134
        - --logtostderr=true
        - --proxy-server-host=10.10.10.102
        - --proxy-server-port=8132
        - --service-account-token-path=/var/run/secrets/tokens/konnectivity-agent-token
        - -v=8
        command:
        - /proxy-agent
        image: registry.k8s.io/kas-network-proxy/proxy-agent:v0.32.0
        imagePullPolicy: IfNotPresent
        livenessProbe:
          failureThreshold: 3
          httpGet:
            path: /healthz
            port: 8134
            scheme: HTTP
          initialDelaySeconds: 15
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 15
        name: konnectivity-agent
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /var/run/secrets/tokens
          name: konnectivity-agent-token
      dnsPolicy: ClusterFirst
      nodeSelector:
        kubernetes.io/os: linux
      priorityClassName: system-cluster-critical
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      serviceAccount: konnectivity-agent
      serviceAccountName: konnectivity-agent
      terminationGracePeriodSeconds: 30
      tolerations:
      - key: CriticalAddonsOnly
        operator: Exists
      volumes:
      - name: konnectivity-agent-token
        projected:
          defaultMode: 420
          sources:
          - serviceAccountToken:
              audience: system:konnectivity-server
              expirationSeconds: 3600
              path: konnectivity-agent-token

Apply the manifest on the tenant cluster and wait agents get deployed:

clastix@kamaji-cmp:~$ kubectl -n kube-system get deploy
NAME                      READY   UP-TO-DATE   AVAILABLE   AGE
calico-kube-controllers   1/1     1            1           58m
coredns                   2/2     2            2           34m
konnectivity-agent        2/2     2            2           10m
clastix@kamaji-cmp:~$ kubectl -n kube-system get pods -o wide
NAME                                       READY   STATUS    RESTARTS   AGE     IP               NODE                          NOMINATED NODE   READINESS GATES
calico-kube-controllers-655b964589-dw49j   1/1     Running   0          58m     172.16.52.66     roma-default-md-dbcgn-cn4p9   <none>           <none>
calico-node-8rsbh                          1/1     Running   0          32m     192.168.11.100   roma-default-md-dbcgn-cn4p9   <none>           <none>
calico-node-fnvjf                          1/1     Running   0          7m54s   192.168.11.103   roma-default-md-dbcgn-hmkrf   <none>           <none>
calico-node-gpxn6                          1/1     Running   0          7m51s   192.168.11.104   roma-default-md-dbcgn-vtchk   <none>           <none>
calico-node-jvnll                          1/1     Running   0          7m55s   192.168.11.102   roma-default-md-dbcgn-tgwhs   <none>           <none>
calico-node-nrzhn                          1/1     Running   0          32m     192.168.11.101   roma-default-md-dbcgn-gctnr   <none>           <none>
coredns-796d84c46b-7gngg                   1/1     Running   0          34m     172.16.52.64     roma-default-md-dbcgn-cn4p9   <none>           <none>
coredns-796d84c46b-8bpht                   1/1     Running   0          34m     172.16.52.67     roma-default-md-dbcgn-cn4p9   <none>           <none>
konnectivity-agent-6b856c7557-rd2zn        1/1     Running   0          10m     172.16.52.68     roma-default-md-dbcgn-cn4p9   <none>           <none>
konnectivity-agent-6b856c7557-zdgl2        1/1     Running   0          10m     172.16.147.129   roma-default-md-dbcgn-gctnr   <none>           <none>
kube-proxy-7k9jf                           1/1     Running   0          7m54s   192.168.11.103   roma-default-md-dbcgn-hmkrf   <none>           <none>
kube-proxy-f2ngm                           1/1     Running   0          32m     192.168.11.100   roma-default-md-dbcgn-cn4p9   <none>           <none>
kube-proxy-nzfzh                           1/1     Running   0          32m     192.168.11.101   roma-default-md-dbcgn-gctnr   <none>           <none>
kube-proxy-sm5cc                           1/1     Running   0          7m55s   192.168.11.102   roma-default-md-dbcgn-tgwhs   <none>           <none>
kube-proxy-xzf5v                           1/1     Running   0          7m51s   192.168.11.104   roma-default-md-dbcgn-vtchk   <none>           <none>

Test the konnectivity agent by logging a pod that is running on a worker node where the agent is nor running, for example:

clastix@kamaji-cmp:~$ kubectl -n kube-system logs kube-proxy-7k9jf
I0711 16:47:12.936010       1 server_linux.go:66] "Using iptables proxy"
I0711 16:47:13.007741       1 server.go:698] "Successfully retrieved node IP(s)" IPs=["192.168.11.103"]
I0711 16:47:13.009455       1 conntrack.go:121] "Set sysctl" entry="net/netfilter/nf_conntrack_max" value=131072
I0711 16:47:13.009486       1 conntrack.go:60] "Setting nf_conntrack_max" nfConntrackMax=131072
I0711 16:47:13.009534       1 conntrack.go:121] "Set sysctl" entry="net/netfilter/nf_conntrack_tcp_timeout_established" value=86400
I0711 16:47:13.009561       1 conntrack.go:121] "Set sysctl" entry="net/netfilter/nf_conntrack_tcp_timeout_close_wait" value=3600
E0711 16:47:13.009581       1 server.go:234] "Kube-proxy configuration may be incomplete or incorrect" err="nodePortAddresses is unset; NodePort connections will be accepted on all local IPs. Consider using `--nodeport-addresses primary`"
I0711 16:47:13.021779       1 server.go:243] "kube-proxy running in dual-stack mode" primary ipFamily="IPv4"
I0711 16:47:13.021809       1 server_linux.go:482] "Detect-local-mode set to ClusterCIDR, but no cluster CIDR specified for primary IP family" ipFamily="IPv4" clusterCIDRs=null
I0711 16:47:13.021815       1 server_linux.go:170] "Using iptables Proxier"
I0711 16:47:13.023012       1 proxier.go:255] "Setting route_localnet=1 to allow node-ports on localhost; to change this either disable iptables.localhostNodePorts (--iptables-localhost-nodeports) or set nodePortAddresses (--nodeport-addresses) to filter loopback addresses" ipFamily="IPv4"
I0711 16:47:13.025612       1 server.go:497] "Version info" version="v1.32.3"
I0711 16:47:13.025629       1 server.go:499] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
I0711 16:47:13.026785       1 config.go:199] "Starting service config controller"
I0711 16:47:13.026850       1 shared_informer.go:313] Waiting for caches to sync for service config
I0711 16:47:13.026913       1 config.go:105] "Starting endpoint slice config controller"
I0711 16:47:13.026945       1 shared_informer.go:313] Waiting for caches to sync for endpoint slice config
I0711 16:47:13.026859       1 config.go:329] "Starting node config controller"
I0711 16:47:13.027159       1 shared_informer.go:313] Waiting for caches to sync for node config
I0711 16:47:13.127418       1 shared_informer.go:320] Caches are synced for node config
I0711 16:47:13.127433       1 shared_informer.go:320] Caches are synced for endpoint slice config
I0711 16:47:13.127419       1 shared_informer.go:320] Caches are synced for service config

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Deploy konnectivity agent add-on as deployment #872

Uh oh!

{{title}}

Uh oh!

Replies: 0 comments

Select a reply

Uh oh!

Deploy konnectivity agent add-on as deployment #872

Uh oh!

bsctl Jul 11, 2025 Maintainer

Replies: 0 comments

bsctl
Jul 11, 2025
Maintainer