[node-agent] Integrate gardener-node-agent into gardenlet's Shoot controller
#8847
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
55 tasks
5ad59a3 to
5d4bff5
Compare
|
/assign |
|
LGTM label has been added. Git tree hash: 2f7a7b4cc8eb2befb2f56020ea04ceddff358662
|
1 task
cmd/gardener-node-agent/app/bootstrappers/kubelet_bootstrap_kubeconfig_test.go
Outdated
Show resolved
Hide resolved
This value is needed later when generating the `gardener-node-agent`'s component configuration, see: https://github.com/gardener/gardener/blob/764df0ee5ebc13b2634eba98169b409244f19bfe/pkg/component/extensions/operatingsystemconfig/original/components/nodeagent/component.go#L127
when feature gate is enabled
Previously (with `cloud-config-downloader`), this was done as part of the `executor.Script` function (see https://github.com/gardener/gardener/blob/67a049a3f66ce489002a67dd59ac7b95e8d2573b/pkg/operation/botanist/operatingsystemconfig.go#L181-L189). Now (with `gardener-node-agent`), the `hyperkube` image is added as a file with type `imageRef` to the OSC's `.spec.files`, see https://github.com/gardener/gardener/blob/67a049a3f66ce489002a67dd59ac7b95e8d2573b/pkg/component/extensions/operatingsystemconfig/original/components/kubelet/component.go#L192-L218. Hence, the original components context must have the correct `hyperkube` image for the Kubernetes version of the worker pool.
They will be deployed as part of a `ManagedResource` in a subsequent commit.
- For backwards-compatibility, we use the same annotation keys like before (`checksum/cloud-config-data` and `checksum/data-script`), even if these names are not fully accurate. Maybe we can change them later - `gardenlet` will compute the checksum of the OSC secret and adds the result as annotation - `gardener-node-agent` reads this checksum from the OSC secret and adds it as annotation to the `Node` after successful reconciliation - This allows `gardenlet` to check whether GNA applied the most recent OSC on the nodes (health checks, adapted in a subsequent commit)
These `Secret`s get reconciled later by `gardener-node-agent`'s `OperatingSystemConfig` controller. They will be deployed as part of a `ManagedResource` in a subsequent commit.
Earlier, this function deployed a `ManagedResource` containing the RBAC rules of `cloud-config-downloader` as well as the secrets containing the bash scripts that get executed by `cloud-config-downloader`. In the next commit, we introduce a functon that deploys a `ManagedResource` containing the RBAC rules for `gardener-node-agent` as well as the secrets containing the OSC that get reconciled by `gardener-node-agent`
- For backwards-compatibility, we use the same annotation keys like before (`checksum/cloud-config-data` and `checksum/data-script`), even if these names are not fully accurate. Maybe we can change them later - Only when a secret with label `gardener.cloud/role=operating-system-config` is found in the shoot, those secrets are considered for the health checks. This is to make the checks compatible with both (a) shoots that weren't reconciled yet, i.e. don't have gardener-node-agent yet, and (b) shoots that were reconciled and have gardener-node-agent
- `gardener-node-agent` uses `cloud-config-downloader` token to download its own access token when deployed on an existing node - It deletes the directory and systemd files on the node after start-up - After the OSC on all nodes was updated, `gardenlet` deletes the `cloud-config-downloader` access secret from both seed and shoot, and the no longer needed `ManagedResource` and `Secret`s for the cloud config execution bash scripts
5d4bff5 to
3a2f552
Compare
|
/approve |
|
LGTM label has been added. Git tree hash: 3933c3207ce45c7cf57d7d61e583942ea634bd14
|
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: oliver-goetz The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
rfranzke
added a commit
to rfranzke/gardener
that referenced
this pull request
May 31, 2024
(from gardener#8847, released with `v1.85.0`)
Merged
rfranzke
added a commit
to rfranzke/gardener
that referenced
this pull request
May 31, 2024
(from gardener#8847, released with `v1.85.0`)
gardener-prow bot
pushed a commit
that referenced
this pull request
Jun 4, 2024
* Remove deprecated fields from `OperatingSystemConfig` (from #9477, released with `v1.92.0`) * Remove cleanup of old `kube-apiserver` `Ingress` resource (from #9300, released with `v1.91.0`) * Remove Istio zone migration code (from #9304 and #9457, released with `v1.91.0` and `v1.92.0`) * Increase removal period of `<name>.ca-cluster` `Secret` To give users more time to adapt * Remove PVC migration for `garden` Prometheus (from #9543, released with `v1.93.0`) * Remove PVC migration for `longterm` Prometheus (from #9606, released with `v1.94.0`) * Drop migration code in `skaffold.yaml` for `core.gardener.cloud/v1` API (from #9771, released with `v1.96.0`) * Remove migration code for e2e upgrade tests after `provider-local` VPN fix (from #9752, released with `v1.96.0`) * Remove cleanup of old `vali` `VerticalPodAutoscaler`s (from #9681, released with `v1.94.0`) * Remove cleanuop code after making `Secret`s of `ManagedResource`s immutable (from #8116, released with `v1.77.0`) * Remove cleanup code of resources of legacy `cloud-config-downloader` (from #8847, released with `v1.85.0`) * Revert "Remove Istio zone migration code" This reverts commit 8850346. * Increase removal period of Istio zone migration code
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
How to categorize this PR?
/area ops-productivity dev-productivity
/kind enhancement
What this PR does / why we need it:
This PR integrates
gardener-node-agentintogardenlet'sShootcontroller. Health checks in itsshoot-carereconciler are adapted as well.The overall flow is similar to before with
cloud-config-downloader:gardenletgenerates aManagedResourcewhich has multipleSecretreferences:gardener-node-agentOperatingSystemConfigsecret that is later reconciled bygardener-node-agentWhen
gardener-node-agentwas successfully rolled out, all leftover files ofcloud-config-downloaderare cleaned up. For existing nodes,cloud-config-downloaderfetches the updatedOperatingSystemConfig(which now containsgardener-node-initandgardener-node-agentunits), and it deletes its ownsystemdunit file (gardener-node-agentmust just clean a few leftovers):The PR also adds handling for the
kubelet's bootstrap kubeconfig generation/cleanup intogardener-node-agent.Which issue(s) this PR fixes:
Part of #8023
Special notes for your reviewer:
/cc @oliver-goetz @ScheererJ
Release note: