Skip to content

Garden Cluster Access For Extensions In Seed Clusters #8001

@timebertt

Description

@timebertt

How to categorize this issue?

/area security usability
/kind enhancement

Motivation

Currently, gardener extensions running in seed clusters have no access to the garden cluster. I.e., they are not able to read/watch/update API resources exposed by gardener-apiserver and related resources.
In some scenarios, such access is required to cover certain use-cases, e.g., if teams want to add additional controllers per seed that act upon Shoot objects (controllers similar to gardenlet's controllers but not fitting into gardenlet).
There are existing extension implementations that work around this limitation by re-using gardenlet's kubeconfig for talking to the garden cluster. This is obviously bad since it prevents auditability.

The Cluster resource offers extensions read-only access to a subset of information from the garden cluster. This could be considered a workaround for the missing extensions access to the garden cluster.
Based on this enhancement, the design decision behind the Cluster resource could be challenged later on.

Goals

  • offer extensions running in seed clusters a secure way to access the garden cluster
  • use a dedicated API user per extension for audit-logging purposes
  • make it easy for extensions to leverage the new mechanism
  • preferably, use short-lived credentials, i.e., support credentials rotation out-of-the-box
  • restrict access of extensions similarly to gardenlet's access to the garden cluster (see doc) and prevent privilege escalation

Non-Goals

These goals are out of scope for this issue right now, but might be implemented later on based on this enhancement.

  • change gardenlet's own access to the garden cluster
  • leverage access to the garden cluster in the extensions library
  • drop the Cluster resource

Proposal

  • the tokenrequestor controller of gardener-resource-manager is re-used and added to gardenlet
    • the controller instance in gardenlet filters for access secrets labelled with resources.gardener.cloud/class=garden
    • the controller manages ServiceAccounts and tokens in the seed-* namespace of its own Seed in the garden cluster (i.e., it ignores the serviceaccount.resources.gardener.cloud/namespace annotation)
  • the ControllerInstallation controller in gardenlet deploys additional secrets for accessing the garden cluster to every extension namespace in the seed:
    • it creates a "garden access secret" for requesting a token for a ServiceAccount dedicated to the extension in the garden cluster
    • it manages a "generic garden kubeconfig" secret that contains the garden cluster's API address and current CA bundle and a reference to a well-known location where the garden access token should be mounted
    • it injects volumes and volume mounts to all containers in apps and batch API objects installed by the ControllerInstallation's helm chart
    • it also injects the location of the injected kubeconfig into the GARDEN_KUBECONFIG environment variable
    • it also injects the seed name into the SEED_NAME environment variable
  • Seed{Autorizer,Restriction} are adapted to handle extension ServiceAccounts similarly to gardenlet's client certificates
    • managing RBAC resources is denied for preventing privilege escalation
    • additional permissions are added as needed

Steps

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions