[provider-local] Harmonize local VPN setup with real-world scenario #9752

rfranzke · 2024-05-14T16:09:16Z

How to categorize this PR?

/area dev-productivity
/kind enhancement

What this PR does / why we need it:
Currently, in the local scenario, some pods talk to the machine pods directly (instead of using the VPN tunnel). See the referenced issue for a more detailed description.

This PR harmonizes the local VPN setup by specifying the node network for Shoots and creating a dedicated IP pool.
With this, the VPN components correctly configure IP routes for talking to the shoot node network.
As a consequence, all traffic correctly traverses the VPN tunnel and gardenlet's tunnel health check reliably detects a broken tunnel.

Which issue(s) this PR fixes:
Part of #9604
Fixes #9020
See also: https://github.com/gardener-community/hackathon/blob/main/2024-05_Schelklingen/README.md#-harmonize-local-vpn-setup-with-real-world-scenario

Special notes for your reviewer:
/cc @timebertt

We need to do some workarounds for making the e2e upgrade tests pass for this specific version. The workarounds are only active until the next minor release.

Release note:

The `allow-shoot-networks` `NetworkPolicy` has been dropped entirely, hence, the `networking.gardener.cloud/to-shoot-networks=allowed` label has no effect anymore and should be removed.

gardener-prow · 2024-05-14T16:09:19Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

timebertt · 2024-05-14T16:37:50Z

/test pull-gardener-e2e-kind pull-gardener-e2e-kind-ipv6

timebertt · 2024-05-14T16:59:17Z

/test pull-gardener-e2e-kind pull-gardener-e2e-kind-ipv6

timebertt · 2024-05-14T18:25:15Z

/test pull-gardener-e2e-kind pull-gardener-e2e-kind-ipv6

timebertt · 2024-05-15T07:12:34Z

We figured that we must perform the NetworkPolicy-related changes (dropping the allow-shoot-networks NetworkPolicy) in the next release. Otherwise, the upgrade tests will fail because the VPN will be broken after the upgrade.

EDIT: we added workarounds to make the upgrade tests past even we perform the NetworkPolicy-related changes immediately.

ScheererJ · 2024-05-22T13:39:59Z

/assign

ScheererJ

Thanks for bringing the local setup closer to Gardener in the real world.

It looks like there are still some end-to-end tests failing, though.

pkg/provider-local/controller/infrastructure/actuator.go

pkg/provider-local/controller/worker/machines.go

Co-Authored-By: Rafael Franzke <rafael.franzke@sap.com> Co-Authored-By: Marcel Boehm <marcel.boehm@inovex.de>

Co-Authored-By: Johannes Scheerer <johannes.scheerer@sap.com>

ScheererJ · 2024-05-28T14:54:47Z

/lgtm

gardener-prow · 2024-05-28T14:54:52Z

LGTM label has been added.

Git tree hash: 3126d936a08bf1c4238dcf512c1b1fd428a9d7e4

rfranzke · 2024-05-29T06:31:04Z

Kaum macht man's richtig, schon geht's 😉

…N fix (from gardener#9752, released with `v1.96.0`)

Ref: gardener/gardener#9752

* Bump github.com/gardener/gardener from 1.95.1 to 1.96.0 Bumps [github.com/gardener/gardener](https://github.com/gardener/gardener) from 1.95.1 to 1.96.0. - [Release notes](https://github.com/gardener/gardener/releases) - [Commits](gardener/gardener@v1.95.1...v1.96.0) --- updated-dependencies: - dependency-name: github.com/gardener/gardener dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * [dependabot skip] make tidy * Adapt to change in the monitoring API * Run make generate * Fix script to work with already cloned repo and v1.ControllerDeployment * Nodes CIDR becomes mandatory since gardener/gardener#9752 --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: gardener-robot-ci-1 <gardener.ci.user@gmail.com> Co-authored-by: vpnachev <vladimir.nachev@sap.com>

* Remove deprecated fields from `OperatingSystemConfig` (from #9477, released with `v1.92.0`) * Remove cleanup of old `kube-apiserver` `Ingress` resource (from #9300, released with `v1.91.0`) * Remove Istio zone migration code (from #9304 and #9457, released with `v1.91.0` and `v1.92.0`) * Increase removal period of `<name>.ca-cluster` `Secret` To give users more time to adapt * Remove PVC migration for `garden` Prometheus (from #9543, released with `v1.93.0`) * Remove PVC migration for `longterm` Prometheus (from #9606, released with `v1.94.0`) * Drop migration code in `skaffold.yaml` for `core.gardener.cloud/v1` API (from #9771, released with `v1.96.0`) * Remove migration code for e2e upgrade tests after `provider-local` VPN fix (from #9752, released with `v1.96.0`) * Remove cleanup of old `vali` `VerticalPodAutoscaler`s (from #9681, released with `v1.94.0`) * Remove cleanuop code after making `Secret`s of `ManagedResource`s immutable (from #8116, released with `v1.77.0`) * Remove cleanup code of resources of legacy `cloud-config-downloader` (from #8847, released with `v1.85.0`) * Revert "Remove Istio zone migration code" This reverts commit 8850346. * Increase removal period of Istio zone migration code

Ref: gardener/gardener#9752

* Bump github.com/gardener/gardener from 1.95.2 to 1.96.1 Bumps [github.com/gardener/gardener](https://github.com/gardener/gardener) from 1.95.2 to 1.96.1. - [Release notes](https://github.com/gardener/gardener/releases) - [Commits](gardener/gardener@v1.95.2...v1.96.1) --- updated-dependencies: - dependency-name: github.com/gardener/gardener dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * [GEP-19] Adapt monitoring configuration * Use `core.gardener.cloud/v1.ControllerDeployment` - ref: gardener/gardener#9771. Exclude local-setup from the REUSE compliance check. Fix the skaffold dependencies check - ref: gardener/gardener#9778 & gardener/gardener#8766. * Fix e2e tests Ref: gardener/gardener#9752 --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Dimitar Kostadinov <dimitar.kostadinov@sap.com>

(part of gardener#9752, released with `v1.96.0`)

* Drop deletion of deprecated `allow-to-shoot-networks` `NetworkPolicy` (part of #9752, released with `v1.96.0`) * Drop fetching extension observability configs with deprecated/legacy method (part of #9695, released with `v1.95.0`) * Drop Prometheus/Alertmanager migration coding (part of #9695, released with `v1.95.0`) * Drop deprecated `.spec.pools[].userData` from `extensions.gardener.cloud/v1alpha1.Worker` API (part of #9722, released with `v1.95.0`) * Drop OSC hash migration `Secret` creation (part of #9846, released with `v1.97.0`) * Drop OSC hash assertion from upgrade tests (part of #9865, released with `v1.98.0`) * Drop removal code of `HVPA` resources (part of #9698, released with `v1.95.0`) * Address PR review feedback

gardener-prow bot requested a review from timebertt May 14, 2024 16:09

rfranzke force-pushed the hackathon-provider-local-vpn branch 2 times, most recently from 928e29f to a483ac7 Compare May 14, 2024 16:58

rfranzke force-pushed the hackathon-provider-local-vpn branch from a483ac7 to d8873a1 Compare May 14, 2024 18:24

timebertt force-pushed the hackathon-provider-local-vpn branch 2 times, most recently from 305949a to 6259b95 Compare May 14, 2024 20:14

timebertt marked this pull request as ready for review May 14, 2024 21:22

gardener-prow bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 14, 2024

timebertt force-pushed the hackathon-provider-local-vpn branch 6 times, most recently from 90822ed to 1caa265 Compare May 16, 2024 18:38

gardener-prow bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 22, 2024

gardener-prow bot assigned ScheererJ May 22, 2024

ScheererJ reviewed May 22, 2024

View reviewed changes

timebertt self-assigned this May 23, 2024

timebertt force-pushed the hackathon-provider-local-vpn branch from 1caa265 to 993a8bd Compare May 23, 2024 06:33

timebertt and others added 6 commits May 28, 2024 16:13

Distinguish between IPPool for IPv4 and IPv6

1716d36

Workaround for upgrade tests

3c74e8e

Co-Authored-By: Rafael Franzke <rafael.franzke@sap.com> Co-Authored-By: Marcel Boehm <marcel.boehm@inovex.de>

Ignore pods using non-default IPPools during startup

ab30145

Drop default values from IPPool spec

ad51ff7

Simplify IP families handling

ca9a063

Set nodeSelector to !all() for local IPPools

e4f54b4

Co-Authored-By: Johannes Scheerer <johannes.scheerer@sap.com>

rfranzke force-pushed the hackathon-provider-local-vpn branch from 7218483 to e4f54b4 Compare May 28, 2024 14:13

gardener-prow bot added the lgtm Indicates that a PR is ready to be merged. label May 28, 2024

gardener-prow bot merged commit 3b61cb9 into gardener:master May 29, 2024

rfranzke deleted the hackathon-provider-local-vpn branch May 29, 2024 06:30

rfranzke added a commit to rfranzke/gardener that referenced this pull request May 31, 2024

Remove migration code for e2e upgrade tests after provider-local VP…

8496207

…N fix (from gardener#9752, released with `v1.96.0`)

rfranzke mentioned this pull request May 31, 2024

🧹 Cleanup some TODOs #9885

Merged

vpnachev added a commit to gardener/gardener-extension-shoot-oidc-service that referenced this pull request May 31, 2024

Nodes CIDR becomes mandatory since gardener/gardener#9752

fe49eec

rfranzke added a commit to rfranzke/gardener that referenced this pull request May 31, 2024

Remove migration code for e2e upgrade tests after provider-local VP…

5b9ebce

…N fix (from gardener#9752, released with `v1.96.0`)

dimitar-kostadinov added a commit to gardener/gardener-extension-registry-cache that referenced this pull request Jun 3, 2024

Fix e2e tests.

3811ba3

Ref: gardener/gardener#9752

dimitar-kostadinov added a commit to gardener/gardener-extension-registry-cache that referenced this pull request Jun 4, 2024

Fix e2e tests.

972ea11

Ref: gardener/gardener#9752

dimitar-kostadinov added a commit to gardener/gardener-extension-registry-cache that referenced this pull request Jun 4, 2024

Fix e2e tests.

074ad49

Ref: gardener/gardener#9752

dimitar-kostadinov added a commit to gardener/gardener-extension-registry-cache that referenced this pull request Jun 4, 2024

Fix e2e tests

fcce861

Ref: gardener/gardener#9752

rfranzke added a commit to rfranzke/gardener that referenced this pull request Jul 29, 2024

Drop deletion of deprecated allow-to-shoot-networks NetworkPolicy

c0132f3

(part of gardener#9752, released with `v1.96.0`)

rfranzke mentioned this pull request Jul 29, 2024

🧹 Cleanup some TODOs #10220

Merged

rfranzke added a commit to rfranzke/gardener that referenced this pull request Jul 31, 2024

Drop deletion of deprecated allow-to-shoot-networks NetworkPolicy

fd4966c

(part of gardener#9752, released with `v1.96.0`)

This was referenced Sep 1, 2025

[GEP-28] gardenadm bootstrap: enable provider-local DNS config #12836

Merged

provider-local: move shoot node CIDR into kind pod CIDR #12873

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[provider-local] Harmonize local VPN setup with real-world scenario #9752

[provider-local] Harmonize local VPN setup with real-world scenario #9752

Uh oh!

rfranzke commented May 14, 2024 •

edited

Loading

Uh oh!

gardener-prow bot commented May 14, 2024

Uh oh!

timebertt commented May 14, 2024

Uh oh!

timebertt commented May 14, 2024

Uh oh!

timebertt commented May 14, 2024

Uh oh!

timebertt commented May 15, 2024 •

edited

Loading

Uh oh!

ScheererJ commented May 22, 2024

Uh oh!

ScheererJ left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ScheererJ commented May 28, 2024

Uh oh!

gardener-prow bot commented May 28, 2024

Uh oh!

rfranzke commented May 29, 2024

Uh oh!

Uh oh!

[provider-local] Harmonize local VPN setup with real-world scenario #9752

[provider-local] Harmonize local VPN setup with real-world scenario #9752

Uh oh!

Conversation

rfranzke commented May 14, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gardener-prow bot commented May 14, 2024

Uh oh!

timebertt commented May 14, 2024

Uh oh!

timebertt commented May 14, 2024

Uh oh!

timebertt commented May 14, 2024

Uh oh!

timebertt commented May 15, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ScheererJ commented May 22, 2024

Uh oh!

ScheererJ left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ScheererJ commented May 28, 2024

Uh oh!

gardener-prow bot commented May 28, 2024

Uh oh!

rfranzke commented May 29, 2024

Uh oh!

Uh oh!

rfranzke commented May 14, 2024 •

edited

Loading

timebertt commented May 15, 2024 •

edited

Loading