[GEP-28] gardenadm bootstrap: Deploy gardener-resource-manager and provider extension
#12052
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
We want to reuse the botanist code for `gardenadm bootstrap` as well. This botanist instance also manages autonomous shoots but it doesn't run the control plane. Hence, the deployment mode (e.g., for gardener-resource-manager) is different than in the resulting autonomous shoot cluster itself. That's why we distinguish between `Shoot.IsAutonomous()` and `Shoot.RunsControlPlane()`.
With this, we only deploy the provider extension (`Infrastructure` and `Worker`) in `gardenadm bootstrap`. The TODO about distinguishing medium-touch and high-touch is basically moved from the generic botanist package (`shoot.go`) to the autonomous botanist package (`extensions.go`).
fca137d to
21d56b8
Compare
|
/assign |
ScheererJ
reviewed
May 12, 2025
|
/lgtm |
|
LGTM label has been added. Git tree hash: df25678d32e3df793e9d8c2ab74372ab1afd9d21
|
rfranzke
reviewed
May 13, 2025
|
LGTM label has been added. Git tree hash: 2bd90abf0c384c4ebaf9c9c71710bcb6b7129e2e
|
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: rfranzke The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
rfranzke
added a commit
to rfranzke/gardener
that referenced
this pull request
May 14, 2025
After gardener#12052, `provider-local` is deployed into the kind cluster. It runs the `dnsconfig` webhook which reacts on `app=machine` pods: https://github.com/gardener/gardener/blob/f3efe83358ef0a6182d63c759b290bd392d7260f/pkg/provider-local/webhook/dnsconfig/add.go#L63-L64 This `provider-local` is responsible for the medium-touch scenario of `gardenadm`. Hence, we actually don't want it to react on the pods related to the `machine` `StatefulSet` since these are relevant for the high-touch scenario of `gardenadm`. Let's simply change the labels here to prevent the webhook from interfering. Example flake: https://prow.gardener.cloud/view/gs/gardener-prow/logs/ci-gardener-e2e-kind-gardenadm/1922295612275953664 Here, the `kube-controller-manager` of the kind cluster fails to rollout the new `machine` pods. From https://gcsweb.prow.gardener.cloud/gcs/gardener-prow/logs/ci-gardener-e2e-kind-gardenadm/1922295612275953664/artifacts/gardener-local/gardener-local-control-plane/pods/kube-system_kube-controller-manager-gardener-local-control-plane_e4d0c8f9c71fba7c8d1b9da5e2da23b7/kube-controller-manager/0.log ``` 2025-05-13T14:26:54.433492015Z stderr F E0513 14:26:54.433359 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:54.440101864Z stderr F E0513 14:26:54.439984 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:54.444946334Z stderr F E0513 14:26:54.444823 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:54.455228214Z stderr F E0513 14:26:54.455144 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:54.501962554Z stderr F E0513 14:26:54.501844 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:54.587370813Z stderr F E0513 14:26:54.587262 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:54.753085172Z stderr F E0513 14:26:54.752971 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:55.079471078Z stderr F E0513 14:26:55.079345 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:55.726386972Z stderr F E0513 14:26:55.726251 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:57.01175209Z stderr F E0513 14:26:57.011625 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" ```
LucaBernstein
pushed a commit
to rfranzke/gardener
that referenced
this pull request
May 14, 2025
After gardener#12052, `provider-local` is deployed into the kind cluster. It runs the `dnsconfig` webhook which reacts on `app=machine` pods: https://github.com/gardener/gardener/blob/f3efe83358ef0a6182d63c759b290bd392d7260f/pkg/provider-local/webhook/dnsconfig/add.go#L63-L64 This `provider-local` is responsible for the medium-touch scenario of `gardenadm`. Hence, we actually don't want it to react on the pods related to the `machine` `StatefulSet` since these are relevant for the high-touch scenario of `gardenadm`. Let's simply change the labels here to prevent the webhook from interfering. Example flake: https://prow.gardener.cloud/view/gs/gardener-prow/logs/ci-gardener-e2e-kind-gardenadm/1922295612275953664 Here, the `kube-controller-manager` of the kind cluster fails to rollout the new `machine` pods. From https://gcsweb.prow.gardener.cloud/gcs/gardener-prow/logs/ci-gardener-e2e-kind-gardenadm/1922295612275953664/artifacts/gardener-local/gardener-local-control-plane/pods/kube-system_kube-controller-manager-gardener-local-control-plane_e4d0c8f9c71fba7c8d1b9da5e2da23b7/kube-controller-manager/0.log ``` 2025-05-13T14:26:54.433492015Z stderr F E0513 14:26:54.433359 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:54.440101864Z stderr F E0513 14:26:54.439984 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:54.444946334Z stderr F E0513 14:26:54.444823 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:54.455228214Z stderr F E0513 14:26:54.455144 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:54.501962554Z stderr F E0513 14:26:54.501844 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:54.587370813Z stderr F E0513 14:26:54.587262 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:54.753085172Z stderr F E0513 14:26:54.752971 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:55.079471078Z stderr F E0513 14:26:55.079345 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:55.726386972Z stderr F E0513 14:26:55.726251 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:57.01175209Z stderr F E0513 14:26:57.011625 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" ```
rfranzke
added a commit
to rfranzke/gardener
that referenced
this pull request
May 14, 2025
After gardener#12052, `provider-local` is deployed into the kind cluster. It runs the `dnsconfig` webhook which reacts on `app=machine` pods: https://github.com/gardener/gardener/blob/f3efe83358ef0a6182d63c759b290bd392d7260f/pkg/provider-local/webhook/dnsconfig/add.go#L63-L64 This `provider-local` is responsible for the medium-touch scenario of `gardenadm`. Hence, we actually don't want it to react on the pods related to the `machine` `StatefulSet` since these are relevant for the high-touch scenario of `gardenadm`. Let's simply change the labels here to prevent the webhook from interfering. Example flake: https://prow.gardener.cloud/view/gs/gardener-prow/logs/ci-gardener-e2e-kind-gardenadm/1922295612275953664 Here, the `kube-controller-manager` of the kind cluster fails to rollout the new `machine` pods. From https://gcsweb.prow.gardener.cloud/gcs/gardener-prow/logs/ci-gardener-e2e-kind-gardenadm/1922295612275953664/artifacts/gardener-local/gardener-local-control-plane/pods/kube-system_kube-controller-manager-gardener-local-control-plane_e4d0c8f9c71fba7c8d1b9da5e2da23b7/kube-controller-manager/0.log ``` 2025-05-13T14:26:54.433492015Z stderr F E0513 14:26:54.433359 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:54.440101864Z stderr F E0513 14:26:54.439984 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:54.444946334Z stderr F E0513 14:26:54.444823 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:54.455228214Z stderr F E0513 14:26:54.455144 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:54.501962554Z stderr F E0513 14:26:54.501844 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:54.587370813Z stderr F E0513 14:26:54.587262 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:54.753085172Z stderr F E0513 14:26:54.752971 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:55.079471078Z stderr F E0513 14:26:55.079345 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:55.726386972Z stderr F E0513 14:26:55.726251 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:57.01175209Z stderr F E0513 14:26:57.011625 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" ```
gardener-prow bot
pushed a commit
that referenced
this pull request
May 14, 2025
* Do not label `machine` `StatefulSet` with `app=machine` After #12052, `provider-local` is deployed into the kind cluster. It runs the `dnsconfig` webhook which reacts on `app=machine` pods: https://github.com/gardener/gardener/blob/f3efe83358ef0a6182d63c759b290bd392d7260f/pkg/provider-local/webhook/dnsconfig/add.go#L63-L64 This `provider-local` is responsible for the medium-touch scenario of `gardenadm`. Hence, we actually don't want it to react on the pods related to the `machine` `StatefulSet` since these are relevant for the high-touch scenario of `gardenadm`. Let's simply change the labels here to prevent the webhook from interfering. Example flake: https://prow.gardener.cloud/view/gs/gardener-prow/logs/ci-gardener-e2e-kind-gardenadm/1922295612275953664 Here, the `kube-controller-manager` of the kind cluster fails to rollout the new `machine` pods. From https://gcsweb.prow.gardener.cloud/gcs/gardener-prow/logs/ci-gardener-e2e-kind-gardenadm/1922295612275953664/artifacts/gardener-local/gardener-local-control-plane/pods/kube-system_kube-controller-manager-gardener-local-control-plane_e4d0c8f9c71fba7c8d1b9da5e2da23b7/kube-controller-manager/0.log ``` 2025-05-13T14:26:54.433492015Z stderr F E0513 14:26:54.433359 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:54.440101864Z stderr F E0513 14:26:54.439984 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:54.444946334Z stderr F E0513 14:26:54.444823 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:54.455228214Z stderr F E0513 14:26:54.455144 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:54.501962554Z stderr F E0513 14:26:54.501844 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:54.587370813Z stderr F E0513 14:26:54.587262 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:54.753085172Z stderr F E0513 14:26:54.752971 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:55.079471078Z stderr F E0513 14:26:55.079345 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:55.726386972Z stderr F E0513 14:26:55.726251 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:57.01175209Z stderr F E0513 14:26:57.011625 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" ``` * Increase timeout for building `gardenadm` binary Since the complexity of the binary has increased in the past weeks, and since running many e2e in the Prow cluster can lead to CPU shortage, let's better increase the timeout a bit to prevent running into flakes. Example flake: https://prow.gardener.cloud/view/gs/gardener-prow/logs/ci-gardener-e2e-kind-gardenadm/1922537540246573056 * Remove `gardener.cloud/role=shoot` label from `gardenadm-high-touch` namespace to prevent NetworkPolicies to be applied --------- Co-authored-by: Luca Bernstein <luca.bernstein@sap.com>
gardener-ci-robot
pushed a commit
to gardener-ci-robot/gardener
that referenced
this pull request
May 14, 2025
After gardener#12052, `provider-local` is deployed into the kind cluster. It runs the `dnsconfig` webhook which reacts on `app=machine` pods: https://github.com/gardener/gardener/blob/f3efe83358ef0a6182d63c759b290bd392d7260f/pkg/provider-local/webhook/dnsconfig/add.go#L63-L64 This `provider-local` is responsible for the medium-touch scenario of `gardenadm`. Hence, we actually don't want it to react on the pods related to the `machine` `StatefulSet` since these are relevant for the high-touch scenario of `gardenadm`. Let's simply change the labels here to prevent the webhook from interfering. Example flake: https://prow.gardener.cloud/view/gs/gardener-prow/logs/ci-gardener-e2e-kind-gardenadm/1922295612275953664 Here, the `kube-controller-manager` of the kind cluster fails to rollout the new `machine` pods. From https://gcsweb.prow.gardener.cloud/gcs/gardener-prow/logs/ci-gardener-e2e-kind-gardenadm/1922295612275953664/artifacts/gardener-local/gardener-local-control-plane/pods/kube-system_kube-controller-manager-gardener-local-control-plane_e4d0c8f9c71fba7c8d1b9da5e2da23b7/kube-controller-manager/0.log ``` 2025-05-13T14:26:54.433492015Z stderr F E0513 14:26:54.433359 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:54.440101864Z stderr F E0513 14:26:54.439984 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:54.444946334Z stderr F E0513 14:26:54.444823 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:54.455228214Z stderr F E0513 14:26:54.455144 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:54.501962554Z stderr F E0513 14:26:54.501844 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:54.587370813Z stderr F E0513 14:26:54.587262 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:54.753085172Z stderr F E0513 14:26:54.752971 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:55.079471078Z stderr F E0513 14:26:55.079345 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:55.726386972Z stderr F E0513 14:26:55.726251 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:57.01175209Z stderr F E0513 14:26:57.011625 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" ```
gardener-prow bot
pushed a commit
that referenced
this pull request
May 15, 2025
* Do not label `machine` `StatefulSet` with `app=machine` After #12052, `provider-local` is deployed into the kind cluster. It runs the `dnsconfig` webhook which reacts on `app=machine` pods: https://github.com/gardener/gardener/blob/f3efe83358ef0a6182d63c759b290bd392d7260f/pkg/provider-local/webhook/dnsconfig/add.go#L63-L64 This `provider-local` is responsible for the medium-touch scenario of `gardenadm`. Hence, we actually don't want it to react on the pods related to the `machine` `StatefulSet` since these are relevant for the high-touch scenario of `gardenadm`. Let's simply change the labels here to prevent the webhook from interfering. Example flake: https://prow.gardener.cloud/view/gs/gardener-prow/logs/ci-gardener-e2e-kind-gardenadm/1922295612275953664 Here, the `kube-controller-manager` of the kind cluster fails to rollout the new `machine` pods. From https://gcsweb.prow.gardener.cloud/gcs/gardener-prow/logs/ci-gardener-e2e-kind-gardenadm/1922295612275953664/artifacts/gardener-local/gardener-local-control-plane/pods/kube-system_kube-controller-manager-gardener-local-control-plane_e4d0c8f9c71fba7c8d1b9da5e2da23b7/kube-controller-manager/0.log ``` 2025-05-13T14:26:54.433492015Z stderr F E0513 14:26:54.433359 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:54.440101864Z stderr F E0513 14:26:54.439984 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:54.444946334Z stderr F E0513 14:26:54.444823 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:54.455228214Z stderr F E0513 14:26:54.455144 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:54.501962554Z stderr F E0513 14:26:54.501844 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:54.587370813Z stderr F E0513 14:26:54.587262 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:54.753085172Z stderr F E0513 14:26:54.752971 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:55.079471078Z stderr F E0513 14:26:55.079345 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:55.726386972Z stderr F E0513 14:26:55.726251 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" 2025-05-13T14:26:57.01175209Z stderr F E0513 14:26:57.011625 1 stateful_set.go:438] "Unhandled Error" err="error syncing StatefulSet gardenadm-high-touch/machine, requeuing: admission webhook \"dnsconfig.local.extensions.gardener.cloud\" denied the request: Service \"coredns\" not found" logger="UnhandledError" ``` * Increase timeout for building `gardenadm` binary Since the complexity of the binary has increased in the past weeks, and since running many e2e in the Prow cluster can lead to CPU shortage, let's better increase the timeout a bit to prevent running into flakes. Example flake: https://prow.gardener.cloud/view/gs/gardener-prow/logs/ci-gardener-e2e-kind-gardenadm/1922537540246573056 * Remove `gardener.cloud/role=shoot` label from `gardenadm-high-touch` namespace to prevent NetworkPolicies to be applied --------- Co-authored-by: rfranzke <rafael.franzke@sap.com> Co-authored-by: Luca Bernstein <luca.bernstein@sap.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
How to categorize this PR?
/area ipcei
/kind enhancement
What this PR does / why we need it:
This PR continues #12041 and deploys both gardener-resource-manager and the provider extension in
gardenadm bootstrap.Which issue(s) this PR fixes:
Part of #2906
Special notes for your reviewer:
/cc @rfranzke @ScheererJ
Although gardener-resource-manager runs in the control plane namespace (
shoot--project--name), it is configured like the runtime/seed resource manager in the garden namespace.Furthermore, the
ManagedResourcesof extensions are deployed into the control plane namespace.This deployment mode makes cleanup very simple. Even though we probably don't want to automate this in
gardenadm, it helps with development. Instead of recreating the kind cluster, you can reset the state using (orphans global resources like CRDs though):Release note: