-
Notifications
You must be signed in to change notification settings - Fork 151
Description
In Kamaji, the default preferred NodeAddressTypes
for the API server to connect to Kubelets is currently configured as:
--kubelet-preferred-address-types=Hostname,InternalIP,ExternalIP
See: github.com/clastix/kamaji/api/v1alpha1/tenantcontrolplane_types.go#L71
This makes Hostname
the first preference for the API server when connecting to nodes for Kubelet(Hostname:KubeletPort
) for operations like kubectl logs
, exec
, port-forward
, and more.
Why this causes issues
If the node hostname is not resolvable with the DNS server, the operations fail with errors like:
dial tcp: lookup node1 on 10.96.0.10:53: no such host
This happens because:
- The API server tries to resolve the node’s hostname via CoreDNS (typically
10.96.0.10:53
). - CoreDNS forwards the query to the external DNS configured in
/etc/resolv.conf
(since CoreDNS itself does not have entries for node names by default). - If the external DNS does not have records for these node hostnames, the lookup fails and API server → kubelet connection is broken.
In some environments, such as certain cloud providers or corporate networks where hostnames are registered in external DNS, this may work. However, relying on external DNS having entries for your nodes introduces fragility and environment-specific dependency.
Proposal
Kubeadm defaults to:
--kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname
This prioritizes InternalIP
, which is routable inside the cluster or VPC, avoiding DNS resolution issues for node hostnames.
Therefore update Kamaji’s default preferredAddressTypes
to:
--kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname
This change reduces dependency on external DNS or custom CoreDNS configuration and provides better out-of-the-box reliability for most environments, especially on-prem or private clouds where nodes have valid InternalIPs but hostnames may not be resolvable via CoreDNS or external DNS.
Furthermore, if nodes are in separate private networks and the API server cannot directly reach their InternalIP
, Konnectivity can be deployed to establish reliable connectivity (Konnectivity is enabled by default in Kamaji TCP pods).
Changes
- Update the default value of
TenantControlPlane.Spec.Kubernetes.Kubelet.PreferredAddressTypes
to:
"InternalIP,ExternalIP,Hostname"
. - Update the default value of
KamajiControlPlane.Spec.Kubelet.PreferredAddressTypes
to:
"InternalIP,ExternalIP,Hostname"
.