Skip to content

Conversation

oliver-goetz
Copy link
Member

How to categorize this PR?

/area auto-scaling control-plane
/kind enhancement

What this PR does / why we need it:
kube-apiservers create a single HTTP2 connection to extension API-servers as gardener-apiserver.
This means that if there are more gardener-apiservers than kube-apiservers in the garden control plane, the "excess" gardener-apiservers do not get any requests.

The consequences of this behavior are:

  • Waste of resources since the GAPIs occupy CPU and memory but do not do anything meaningful
  • VPA resource calculation is inaccurate. The idle GAPIs reduce the CPU and memory usage so the working GAPIs most likely consume more resources than requested by VPA.

There is a high change that this scenario happens because there is usually more load on GAPIs than on the KAPIs in the gardener-controlplane.

Thus, this PR removes HPA from gardener-apiserver and sets its replicas to 3 in HA case and 2 in the non-HA case. This way we ensure that there are not more GAPIs than KAPIs.
Additionally, the max allowed requests are increased since this is now the only autoscaling mechanism for GAPI.

There could be other options to solve this problem e.g. disabling HTTP2 in GAPIs to force KAPIs to create more connections.
However, it is hard to estimate the consequences of this change and it would need further investigations.

Which issue(s) this PR fixes:
Fixes #

Special notes for your reviewer:

Release note:

`gardener-apiserver` is now deployed with a fixed number of replicas (HA configuration: 3, non-HA configuration: 2). It does not use `HPA` anymore.

@gardener-prow gardener-prow bot added the area/auto-scaling Auto-scaling (CA/HPA/VPA/HVPA, predominantly control plane, but also otherwise) related label Mar 17, 2025
@gardener-prow gardener-prow bot requested review from Kostov6 and timuthy March 17, 2025 13:31
@gardener-prow gardener-prow bot added area/control-plane Control plane related kind/enhancement Enhancement, improvement, extension cla: yes Indicates the PR's author has signed the cla-assistant.io CLA. labels Mar 17, 2025
@oliver-goetz
Copy link
Member Author

/cc @voelzmo

@gardener-prow gardener-prow bot requested a review from voelzmo March 17, 2025 13:31
@gardener-prow gardener-prow bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Mar 17, 2025
Copy link
Member

@LucaBernstein LucaBernstein left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@gardener-prow gardener-prow bot added the lgtm Indicates that a PR is ready to be merged. label Mar 17, 2025
@gardener-prow gardener-prow bot removed the lgtm Indicates that a PR is ready to be merged. label Mar 17, 2025
@gardener-prow gardener-prow bot requested a review from LucaBernstein March 17, 2025 13:58
Copy link
Member

@rfranzke rfranzke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@gardener-prow gardener-prow bot added the lgtm Indicates that a PR is ready to be merged. label Mar 18, 2025
Copy link
Contributor

gardener-prow bot commented Mar 18, 2025

LGTM label has been added.

Git tree hash: ab79abd0cb2104cadc5f2ef8a5c5328b95144a82

Copy link
Member

@ialidzhikov ialidzhikov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

Copy link
Contributor

gardener-prow bot commented Mar 19, 2025

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ialidzhikov

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@gardener-prow gardener-prow bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 19, 2025
@gardener-prow gardener-prow bot merged commit 90d1a82 into gardener:master Mar 19, 2025
19 checks passed
@oliver-goetz oliver-goetz deleted the enh/remove-hpa-gardener-apiserver branch March 19, 2025 14:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/auto-scaling Auto-scaling (CA/HPA/VPA/HVPA, predominantly control plane, but also otherwise) related area/control-plane Control plane related cla: yes Indicates the PR's author has signed the cla-assistant.io CLA. kind/enhancement Enhancement, improvement, extension lgtm Indicates that a PR is ready to be merged. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants