Skip to content

Change default kubelet-preferred-address-types to prioritize InternalIP for more reliable node connectivity #858

@parthyadav3105

Description

@parthyadav3105

In Kamaji, the default preferred NodeAddressTypes for the API server to connect to Kubelets is currently configured as:

--kubelet-preferred-address-types=Hostname,InternalIP,ExternalIP

See: github.com/clastix/kamaji/api/v1alpha1/tenantcontrolplane_types.go#L71

This makes Hostname the first preference for the API server when connecting to nodes for Kubelet(Hostname:KubeletPort) for operations like kubectl logs, exec, port-forward, and more.

Why this causes issues

If the node hostname is not resolvable with the DNS server, the operations fail with errors like:

dial tcp: lookup node1 on 10.96.0.10:53: no such host

This happens because:

  • The API server tries to resolve the node’s hostname via CoreDNS (typically 10.96.0.10:53).
  • CoreDNS forwards the query to the external DNS configured in /etc/resolv.conf (since CoreDNS itself does not have entries for node names by default).
  • If the external DNS does not have records for these node hostnames, the lookup fails and API server → kubelet connection is broken.

In some environments, such as certain cloud providers or corporate networks where hostnames are registered in external DNS, this may work. However, relying on external DNS having entries for your nodes introduces fragility and environment-specific dependency.

Proposal

Kubeadm defaults to:

--kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname

This prioritizes InternalIP, which is routable inside the cluster or VPC, avoiding DNS resolution issues for node hostnames.

Therefore update Kamaji’s default preferredAddressTypes to:

--kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname

This change reduces dependency on external DNS or custom CoreDNS configuration and provides better out-of-the-box reliability for most environments, especially on-prem or private clouds where nodes have valid InternalIPs but hostnames may not be resolvable via CoreDNS or external DNS.

Furthermore, if nodes are in separate private networks and the API server cannot directly reach their InternalIP, Konnectivity can be deployed to establish reliable connectivity (Konnectivity is enabled by default in Kamaji TCP pods).

Changes

  • Update the default value of TenantControlPlane.Spec.Kubernetes.Kubelet.PreferredAddressTypes to:
    "InternalIP,ExternalIP,Hostname".
  • Update the default value of KamajiControlPlane.Spec.Kubelet.PreferredAddressTypes to:
    "InternalIP,ExternalIP,Hostname".

Metadata

Metadata

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions