Skip to content

TOB-K8S-014: Kubeadm performs potentially-dangerous reset operations #81120

@cji

Description

@cji

This issue was reported in the Kubernetes Security Audit Report

Description
Within the kubeadm CLI, there is a command that will search for mounted directories within the static kubeadmconstants.kubeletRunDirectory. If any are found, they will be subsequently unmounted by umount. Changes to this command in the future could also be prone to command injection, due to the encapsulation of this command within the sh -c context.

umountDirsCmd := fmt.Sprintf("awk '$2 ~ path {print $2}' path=%s /proc/mounts | xargs -r umount", kubeadmconstants.kubeletRunDirectory)
...
umountOutputBytes, err := exec.Command("sh", "-c", umountDirsCmd).Output()
...

Figure 12.1: Potentially dangerous unmounting directories within the kubelet run directory.

Additionally, mounts are not checked to ensure they are not in use before unmounting. See Figure 2, where even if kubelet isn’t stopped, it will still continue execution without explicitly stopping, only logging that kubelet wasn’t stopped.

...
if err := initSystem.ServiceStop("kubelet"); err != nil {
	klog.Warningf("[reset] the kubelet service could not be stopped by kubeadm: [%v]\n", err)
	klog.Warningln("[reset] please ensure kubelet is stopped manually")
}
...

Figure 12.2: An example of lack of error handling, leading to continuation after an error with only warning logs.

The error handling shown in Figure 2 is systemic in this particular command, where other operations will occur after a logged error.

klog.V(1).Info("[reset] removing Kubernetes-managed containers")
if err := removeContainers(utilsexec.New(), r.criSocketPath); err != nil {
	klog.Errorf("[reset] failed to remove containers: %+v", err)
}

Exploit Scenario
Eve gains access to one of Alice’s Kubernetes cluster node hosts and creates a file with a filename to exploit Linux wildcard expansion. Because kubeadm uses sh to encapsulate the commands, a future implementation change in kubeadm leads Alice to fall victim to Eve’s wildcard expansion exploit.

Recommendation
Short term, ensure errors at each step are raised explicitly, and require operator continuation to prevent further errors and state modification.

Long term, avoid using compound shell commands which affect system state without appropriate validation. Errors when interacting with state should require operator intervention before continuation.

Anything else we need to know?:

See #81146 for current status of all issues created from these findings.

The vendor gave this issue an ID of TOB-K8S-014 and it was finding 12 of the report.

The vendor considers this issue Medium Severity.

To view the original finding, begin on page 40 of the Kubernetes Security Review Report

Environment:

  • Kubernetes version: 1.13.4

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/securitykind/bugCategorizes issue or PR as related to a bug.sig/cluster-lifecycleCategorizes an issue or PR as relevant to SIG Cluster Lifecycle.wg/security-auditCategorizes an issue or PR as relevant to WG Security Audit.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions