Skip to content

Conversation

vicwicker
Copy link
Member

@vicwicker vicwicker commented Aug 13, 2025

How to categorize this PR?

/area monitoring
/kind enhancement

What this PR does / why we need it:

We recently identified that certain Prometheus volumes still contain the prometheus- folder with outdated Prometheus data. The following PR, with further details, addressed this issue by implementing a cleanup process:

The cleanup code introduced a new init container in the Prometheus resource. For the garden, longterm, aggregate, cache, and seed Prometheus, this init container was added directly to the Prometheus resource. In the case of shoot Prometheus, where successful shoot reconciliation can't be guaranteed, the init container was temporarily added to each shoot Prometheus resource as a one-time migration during the gardenlet startup. Successfully cleaned up shoot Prometheus were annotated to prevent repeated execution of this cleanup process during gardenlet startup.

Now, after three Gardener releases, it is time to drop this init container and annotation from the Prometheus resources and eliminate as much of the migration code as possible.

Special notes for your reviewer:

/cc @istvanballok @rfranzke

Release note:

Remove the init container and annotation used for the Prometheus volume cleanup migration from Prometheus resources

@gardener-prow gardener-prow bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Aug 13, 2025
Copy link
Contributor

gardener-prow bot commented Aug 13, 2025

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@gardener-prow gardener-prow bot added area/monitoring Monitoring (including availability monitoring and alerting) related kind/enhancement Enhancement, improvement, extension needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. cla: yes Indicates the PR's author has signed the cla-assistant.io CLA. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Aug 13, 2025
@vicwicker vicwicker changed the title Remove Init Container and Annotation for Prometheus Volume Cleanup Migration Remove init container and annotation used for Prometheus volume cleanup migration Aug 13, 2025
@vicwicker vicwicker force-pushed the remove-promehetus-disk-annotation branch from 674d8c9 to 64b506e Compare August 13, 2025 10:18
@gardener-prow gardener-prow bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 13, 2025
@vicwicker vicwicker force-pushed the remove-promehetus-disk-annotation branch from 64b506e to 7018c08 Compare August 13, 2025 11:31
@gardener-prow gardener-prow bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 13, 2025
@vicwicker vicwicker force-pushed the remove-promehetus-disk-annotation branch from 7018c08 to 4b16e43 Compare August 13, 2025 15:28
@gardener-prow gardener-prow bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 13, 2025
@vicwicker vicwicker marked this pull request as ready for review August 14, 2025 15:04
@gardener-prow gardener-prow bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Aug 14, 2025
Copy link
Member

@istvanballok istvanballok left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are considering to prepare the commits differently,

  • first revert #12219 entirely, and
  • then readd the pieces that need to be retained.

That approach will show in the diff the things that "stay", i.e. things that could be reviewed, and not the things that are deleted.
This will allow the reviewer to find things that might be deleted in addition - an aspect that is not obvious in the current representation.

vicwicker and others added 2 commits August 18, 2025 17:06
gardener#12219)"

This reverts commit 3b6c17c. The
Prometheus annotations added in the original commit are not
automatically removed by this revert. There will be a follow up commit
to remove them.

git fetch origin pull/12219/head # d1e7cd5
git fetch origin pull/12728/head # 4b16e43
gitk HEAD 4b16e43 d1e7cd5 ^3b6c17c808^ -- $(git log --oneline -1 --numstat 3b6c17c | tail -n +2 | awk '{print $3}')

Conflicts:
  cmd/gardenlet/app/migration.go
    Resolve conflicts manually by removing the function definition and
    function call that was added in the original PR.
  pkg/apis/resources/v1alpha1/types.go
    Manual conflict resolution.
  pkg/component/observability/monitoring/prometheus/component.go
    Manual conflict resolution.
  pkg/component/observability/monitoring/prometheus/prometheus.go
    Manual conflict resolution.

Co-authored-by: Istvan Zoltan Ballok <istvan.zoltan.ballok@sap.com>
This commit introduces a new migration step during the gardenlet startup
to remove the Prometheus annotation used in the previous migration step
to mark it as cleaned up.

Co-authored-by: Istvan Zoltan Ballok <istvan.zoltan.ballok@sap.com>
@vicwicker vicwicker force-pushed the remove-promehetus-disk-annotation branch from 4b16e43 to 19997df Compare August 19, 2025 07:10
Copy link
Member

@istvanballok istvanballok left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@gardener-prow gardener-prow bot added the lgtm Indicates that a PR is ready to be merged. label Aug 19, 2025
Copy link
Contributor

gardener-prow bot commented Aug 19, 2025

LGTM label has been added.

Git tree hash: a9b8c02c7382fd9375e03f4aa223df187c12c1f6

Copy link
Member

@rfranzke rfranzke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for following up :)
/approve

Copy link
Contributor

gardener-prow bot commented Aug 20, 2025

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: istvanballok, rfranzke

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@gardener-prow gardener-prow bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 20, 2025
@gardener-prow gardener-prow bot merged commit b9f9d87 into gardener:master Aug 20, 2025
19 checks passed
@vicwicker vicwicker deleted the remove-promehetus-disk-annotation branch August 25, 2025 09:28
Duciwuci pushed a commit to stackitcloud/gardener that referenced this pull request Sep 1, 2025
…up migration (gardener#12728)

* Revert "Clean up obsolete `prometheus-` folder from Prometheus volumes (gardener#12219)"

This reverts commit 3b6c17c. The
Prometheus annotations added in the original commit are not
automatically removed by this revert. There will be a follow up commit
to remove them.

git fetch origin pull/12219/head # d1e7cd5
git fetch origin pull/12728/head # 4b16e43
gitk HEAD 4b16e43 d1e7cd5 ^3b6c17c808^ -- $(git log --oneline -1 --numstat 3b6c17c | tail -n +2 | awk '{print $3}')

Conflicts:
  cmd/gardenlet/app/migration.go
    Resolve conflicts manually by removing the function definition and
    function call that was added in the original PR.
  pkg/apis/resources/v1alpha1/types.go
    Manual conflict resolution.
  pkg/component/observability/monitoring/prometheus/component.go
    Manual conflict resolution.
  pkg/component/observability/monitoring/prometheus/prometheus.go
    Manual conflict resolution.

Co-authored-by: Istvan Zoltan Ballok <istvan.zoltan.ballok@sap.com>

* Remove Prometheus annotation from gardenlet startup

This commit introduces a new migration step during the gardenlet startup
to remove the Prometheus annotation used in the previous migration step
to mark it as cleaned up.

Co-authored-by: Istvan Zoltan Ballok <istvan.zoltan.ballok@sap.com>

---------

Co-authored-by: Istvan Zoltan Ballok <istvan.zoltan.ballok@sap.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/monitoring Monitoring (including availability monitoring and alerting) related cla: yes Indicates the PR's author has signed the cla-assistant.io CLA. kind/enhancement Enhancement, improvement, extension lgtm Indicates that a PR is ready to be merged. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants