Add cilium operator go runtime sched latency metrics #29245
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
98845d7 to
99b774a
Compare
|
/test |
99b774a to
83fd95f
Compare
|
/test |
|
Nit: checkpatch is complaining about a typo in the commit message: |
|
/test |
0c3eab3 to
ad6017b
Compare
|
/test |
ad6017b to
36729e5
Compare
|
/test |
36729e5 to
c90943a
Compare
c90943a to
fa71a0f
Compare
|
/test |
fa71a0f to
346bca4
Compare
|
/test |
346bca4 to
4d890c4
Compare
|
/test |
4d890c4 to
743d118
Compare
|
/test |
The operator depends on various goroutines being scheduled one time to perform critical tasks. Significant scheduling lags could indicate potential issues with the operator functions. - Added GO scheduler latency metrics tracking to the cilium operator. Signed-off-by: Fernand Galiana <fernand.galiana@isovalent.com>
743d118 to
f006d91
Compare
|
/test |
mhofstetter
added a commit
to mhofstetter/cilium
that referenced
this pull request
Nov 28, 2024
Currently, the Cilium Operator only uses the metrics registry of the `controller-runtime` as base registry if Gateway API functionality is enabled. This has historic reasons as Gateway API functionality was the first module that used `controller-runtime` for its controllers. In the meantime, other modules (Ingress Controller, Secret Sync, ...) are using the `controller-runtime` too - but the metrics registry of the library is still only used if Gateway API is enabled. In these cases, the metrics of the `controller-runtime` library (https://book.kubebuilder.io/reference/metrics-reference) aren't exposed. Therefore, this commit changes the behaviour, that the metrics registry of the `controller-runtime` library is always used as the base registry. Note: Unfortunately, the `controller-runtime` library configures the base registry to have Go runtime metrics included by default (https://github.com/cilium/cilium/blob/main/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/metrics/metrics.go#L91-L92). Therefore, this Go metrics collector has to be removed from the metrics registry before registering a new Go metrics collector with custom Go runtime metrics enabled. Otherwise the registration fails because some Go runtime metrics are registered twice. Related PRs: - cilium#28931 (extract `controller-runtime` integration into its own Hive Cell) - cilium#29245 (export go runtime sched latency metrics) Signed-off-by: Marco Hofstetter <marco.hofstetter@isovalent.com>
github-merge-queue bot
pushed a commit
that referenced
this pull request
Nov 29, 2024
Currently, the Cilium Operator only uses the metrics registry of the `controller-runtime` as base registry if Gateway API functionality is enabled. This has historic reasons as Gateway API functionality was the first module that used `controller-runtime` for its controllers. In the meantime, other modules (Ingress Controller, Secret Sync, ...) are using the `controller-runtime` too - but the metrics registry of the library is still only used if Gateway API is enabled. In these cases, the metrics of the `controller-runtime` library (https://book.kubebuilder.io/reference/metrics-reference) aren't exposed. Therefore, this commit changes the behaviour, that the metrics registry of the `controller-runtime` library is always used as the base registry. Note: Unfortunately, the `controller-runtime` library configures the base registry to have Go runtime metrics included by default (https://github.com/cilium/cilium/blob/main/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/metrics/metrics.go#L91-L92). Therefore, this Go metrics collector has to be removed from the metrics registry before registering a new Go metrics collector with custom Go runtime metrics enabled. Otherwise the registration fails because some Go runtime metrics are registered twice. Related PRs: - #28931 (extract `controller-runtime` integration into its own Hive Cell) - #29245 (export go runtime sched latency metrics) Signed-off-by: Marco Hofstetter <marco.hofstetter@isovalent.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Please ensure your pull request adheres to the following guidelines:
description and a
Fixes: #XXXline if the commit addresses a particularGitHub issue.
Fixes: <commit-id>tag, thenplease add the commit author[s] as reviewer[s] to this issue.
The operator depends on various goroutines being scheduled one time to perform critical tasks. Significatn scheduling lags could indicate potential issues with the operator functions.
Signed-off-by: Fernand Galiana fernand.galiana@gmail.com