Skip to content

TOB-K8S-024: kubelet liveness probes can be used to enumerate host network #81129

@cji

Description

@cji

This issue was reported in the Kubernetes Security Audit Report

Description
Kubernetes supports both readiness and liveness probes to detect when a Pod is operating correctly, and when to begin or stop directing traffic to a Pod. Three methods are available to facilitate these probes: command execution, HTTP, and TCP.

Using the HTTP and TCP probes, it is possible for an operator with limited access to the cluster (purely kubernetes-service related access) to enumerate the underlying host network. This is possible due to the scope in which these probes execute. Unlike the command execution probe, which will execute a command within the container, the TCP and HTTP probes execute from the context of the kubelet process. Thus, host networking interfaces are used, and the operator is now able to specify hosts which may not be available to Pods kubelet is managing.

The enumeration of the host network uses the container’s health and readiness to determine the status of the remote host. If the pod is killed and restarted due to a failed liveness probe, this indicates that the host is inaccessible. If the Pod successfully passes the liveness check and is presented as ready, the host is accessible. These two states create boolean states of accessible and inaccessible hosts to the underlying host running kubelet.

Additionally, an attacker can append headers through the Pod specification, which are interpreted by the Go HTTP library as authentication or additional request headers. This can allow an attacker to abuse liveness probes to access a wider-range of cluster resources.

An example Pod file that can enumerate the host network is available in Appendix E.

Exploit Scenario
Alice configures a cluster which restricts communications between services on the cluster. Eve gains access to Alice’s cluster, and subsequently submits many Pods enumerating the host network in an attempt to gain information about Alice’s underlying host network.

Recommendations
Short term, restrict the kubelet in a way that prevents the kubelet from probing hosts it does not manage directly.

Long term, consider restricting probes to the container runtime, allowing liveness to be determined within the scope of the container-networking interface.

Anything else we need to know?:

See #81146 for current status of all issues created from these findings.

The vendor gave this issue an ID of TOB-K8S-024 and it was finding 21 of the report.

The vendor considers this issue Medium Severity.

To view the original finding, begin on page 60 of the Kubernetes Security Review Report

Environment:

  • Kubernetes version: 1.13.4

Metadata

Metadata

Labels

area/securitykind/bugCategorizes issue or PR as related to a bug.lifecycle/frozenIndicates that an issue or PR should not be auto-closed due to staleness.priority/backlogHigher priority than priority/awaiting-more-evidence.priority/important-longtermImportant over the long term, but may not be staffed and/or may need multiple releases to complete.sig/nodeCategorizes an issue or PR as relevant to SIG Node.triage/acceptedIndicates an issue or PR is ready to be actively worked on.wg/security-auditCategorizes an issue or PR as relevant to WG Security Audit.

Type

No type

Projects

Status

Triaged

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions