[v1.12] Author Backport of 28382 (Metrics associated with a deleted node should not be reported) #28977
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
/test-backport-1.12 |
cced45a to
1b0dbe1
Compare
christarazi
approved these changes
Nov 6, 2023
|
/test-backport-1.12 |
1 similar comment
|
/test-backport-1.12 |
…er reported. [ upstream commit e9f97cd ] When a node is deleted from a cluster, metrics associated with that node are still being exported to prometheus. Short of restarting the agent, we want to dynamically delete these metrics when a node is removed from the cluster. This PR ensures node_connectivity_status and node_connectivity_latency no longer report metrics for nodes that are no longer present on the cluster. [ Backporter's notes: Original PR was adapted! ] The original PR depends (mainly!) on 2 other PRs that haven't been backported and are fairly substential. Given this, I've opted to adapt the original implementation to surface the fix while minimizing impact with these updates: 1. pkg/metrics/interfaces did not introduce pkg/metrics/metric wrappers as of this release. Hence adapted deletableVec to use the current implementation. (Referring to commit: 84ea383) 2. pkg/node/manager/manager was adapted to provide for metrics deletion when a node is deleted. Subsequent PR refactored the manager metrics structure which the original PR used. (Referring to commit: c49ef45) 3. In order to pickup prom metrics vec delete feature github.com/prometheus/client_golang dep was bumped to v1.14.0 Signed-off-by: Fernand Galiana <fernand.galiana@isovalent.com>
1b0dbe1 to
5d05417
Compare
|
/test-backport-1.12 Edit: Both 4.9 jobs failed with flakes similar to
Job 'Cilium-PR-K8s-1.16-kernel-4.9' failed: Click to show.Test NameFailure OutputJenkins URL: https://jenkins.cilium.io/job/Cilium-PR-K8s-1.16-kernel-4.9/225/ If it is a flake and a GitHub issue doesn't already exist to track it, comment Then please upload the Jenkins artifacts to that issue. |
|
The runtime tests all failed which might either suggest a VM provisioning failure or something in the privileged unit test suite is broken from changes in this PR. Edit: Looks like it was a VM provisioning failure. |
squeed
approved these changes
Nov 8, 2023
|
/test-1.16-4.9 Edit: #24840 |
|
/test-1.19-4.9 |
|
/test-runtime |
|
/test-1.16-4.9 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Once this PR is merged, you can update the PR labels via: