Skip to content

Avoid mutating the client-go informer cache #7743

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

wallrj
Copy link
Member

@wallrj wallrj commented May 9, 2025

See #7723 (comment) for context.

/kind bug

NONE

Signed-off-by: Richard Wall <richard.wall@venafi.com>
@cert-manager-prow cert-manager-prow bot added kind/bug Categorizes issue or PR as related to a bug. release-note-none Denotes a PR that doesn't merit a release note. dco-signoff: yes Indicates that all commits in the pull request have the valid DCO sign-off message. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels May 9, 2025
@wallrj wallrj requested a review from Copilot May 9, 2025 13:36
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR aims to prevent accidental mutation of the client-go informer cache by deep copying the Certificate object before applying defaults.

  • A deep copy is now performed in ProcessItem to safeguard the informer cache.
  • Documentation in the defaulting functions is updated to clearly warn against mutating cached objects.

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
pkg/controller/certificates/keymanager/keymanager_controller.go Deep copy of Certificate objects to avoid cache mutation
internal/apis/certmanager/v1/defaults.go Updated comments to caution against direct mutation of cached objects

@maelvls
Copy link
Member

maelvls commented May 9, 2025

I imagine that you can now revert the "update status" logic (https://github.com/cert-manager/cert-manager/pull/7723/files#r2078072530)?

-_, err := c.client.CertmanagerV1().Certificates(crt.Namespace).UpdateStatus(ctx, &cmapi.Certificate{
-	ObjectMeta: crt.ObjectMeta,
-	Status:     crt.Status,
-}, metav1.UpdateOptions{})
+_, err := c.client.CertmanagerV1().Certificates(crt.Namespace).UpdateStatus(ctx, crt, metav1.UpdateOptions{})

@wallrj
Copy link
Member Author

wallrj commented May 9, 2025

I imagine that you can now revert the "update status" logic (https://github.com/cert-manager/cert-manager/pull/7723/files#r2078072530)?

-_, err := c.client.CertmanagerV1().Certificates(crt.Namespace).UpdateStatus(ctx, &cmapi.Certificate{
-	ObjectMeta: crt.ObjectMeta,
-	Status:     crt.Status,
-}, metav1.UpdateOptions{})
+_, err := c.client.CertmanagerV1().Certificates(crt.Namespace).UpdateStatus(ctx, crt, metav1.UpdateOptions{})

No, because the defaulting function will still set the Certificate.Spec.PrivateKey which will be picked up by the fake updatestatus client in the ProcessItem tests.
The real UpdateStatus function ignores the supplied spec fields but the fake version records the spec....it's a bug in the fake client really....in that the fake does not behave the same as the real client.

@maelvls
Copy link
Member

maelvls commented May 9, 2025

Ah, makes sense.

/lgtm
/hold if Tim wants to look at the change

@cert-manager-prow cert-manager-prow bot added do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. lgtm Indicates that a PR is ready to be merged. labels May 9, 2025
@@ -157,6 +157,8 @@ func (c *controller) ProcessItem(ctx context.Context, key types.NamespacedName)

// Apply runtime defaults to apply default values that are governed by
// controller feature gates, such as DefaultPrivateKeyRotationPolicyAlways.
// We deep copy the object to avoid mutating the client-go cache.
crt = crt.DeepCopy()
cminternal.SetRuntimeDefaults_Certificate(crt)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively, SetRuntimeDefaults_Certificate could return a copy instead of modifying the value that was passed in.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I considered that, because I think that would be more intuitive and less error prone.

But I also thought it worth keeping the semantics the same as standard defaulting functions and imagined there might be cases where the deep copy had already been made by a parent controller in which case it would be wasteful to do an extra deep copy.

@inteon
Copy link
Member

inteon commented May 12, 2025

/approve

@cert-manager-prow
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: inteon

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@cert-manager-prow cert-manager-prow bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 12, 2025
@inteon
Copy link
Member

inteon commented May 12, 2025

/unhold

@cert-manager-prow cert-manager-prow bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label May 12, 2025
@cert-manager-prow cert-manager-prow bot merged commit 0b21f63 into cert-manager:master May 12, 2025
6 checks passed
@wallrj wallrj deleted the rotation-policy-default-always-2 branch May 13, 2025 08:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. dco-signoff: yes Indicates that all commits in the pull request have the valid DCO sign-off message. kind/bug Categorizes issue or PR as related to a bug. lgtm Indicates that a PR is ready to be merged. release-note-none Denotes a PR that doesn't merit a release note. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants