Report

we have recently upgraded keda version to 2.12.1.
after successful upgrade , we have modified scaledObject parameters from Application chart
e.g minReplica value . threshould etc

So During helm upgrade to application we're getting below error

Error: UPGRADE FAILED: cannot patch "security-services" with kind ScaledObject: Timeout: request did not complete within requested timeout - context deadline exceeded && cannot patch "security-services" with kind ScaledObject: Timeout: request did not complete within requested timeout - context deadline exceeded && cannot patch "config-services" with kind ScaledObject: Timeout: request did not complete within requested timeout - context deadline exceeded && cannot patch "security-services" with kind ScaledObject: Timeout: request did not complete within requested timeout - context deadline exceeded

so Helm upgrade failed and we could see error in keda operator pod logs

2024-02-06T11:34:09Z INFO Initializing Scaling logic according to ScaledObject Specification {"controller": "scaledobject", "controllerGroup": "keda.sh", "controllerKind": "ScaledObject", "ScaledObject": {"name":"security-services","namespace":"ffs"}, "namespace": "ffs", "name": "security-services", "reconcileID": "6d14dcf1-ac2c-458e-ae91-07b67d6509eb"} 2024-02-06T11:34:09Z ERROR scaleexecutor failed to patch Objects {"scaledobject.Name": "security-services", "scaledObject.Namespace": "ffs", "scaleTarget.Name": "security-services", "error": "client rate limiter Wait returned an error: context canceled"} github.com/kedacore/keda/v2/pkg/status.TransformObject /workspace/pkg/status/status.go:186 github.com/kedacore/keda/v2/pkg/scaling/executor.(*scaleExecutor).updateLastActiveTime /workspace/pkg/scaling/executor/scale_executor.go:81 github.com/kedacore/keda/v2/pkg/scaling/executor.(*scaleExecutor).RequestScale /workspace/pkg/scaling/executor/scale_scaledobjects.go:153 github.com/kedacore/keda/v2/pkg/scaling.(*scaleHandler).checkScalers /workspace/pkg/scaling/scale_handler.go:247 github.com/kedacore/keda/v2/pkg/scaling.(*scaleHandler).startScaleLoop /workspace/pkg/scaling/scale_handler.go:180 2024-02-06T11:34:09Z ERROR scaleexecutor Error updating last active time {"scaledobject.Name": "security-services", "scaledObject.Namespace": "ffs", "scaleTarget.Name": "security-services", "error": "client rate limiter Wait returned an error: context canceled"} github.com/kedacore/keda/v2/pkg/scaling/executor.(*scaleExecutor).RequestScale /workspace/pkg/scaling/executor/scale_scaledobjects.go:155 github.com/kedacore/keda/v2/pkg/scaling.(*scaleHandler).checkScalers /workspace/pkg/scaling/scale_handler.go:247 github.com/kedacore/keda/v2/pkg/scaling.(*scaleHandler).startScaleLoop /workspace/pkg/scaling/scale_handler.go:180

Expected Behavior

Applicatiion should have upgraded with specified ScaledObject configuration which were going to applied with helm upgrade

Actual Behavior

Helm upgrade failed with below error

Error: UPGRADE FAILED: cannot patch "app-gateway" with kind ScaledObject: Timeout: request did not complete within requested timeout - context deadline exceeded && cannot patch "security-services" with kind ScaledObject: Timeout: request did not complete within requested timeout - context deadline exceeded && cannot patch "config-services" with kind ScaledObject: Timeout: request did not complete within requested timeout - context deadline exceeded && cannot patch "security-services" with kind ScaledObject: Timeout: request did not complete within requested timeout - context deadline exceeded

Steps to Reproduce the Problem

1.modify scaledObject parameter
2.try to upgrade application chart with modified values
3.Helm upgrade would fail with error given in description

Logs from KEDA operator

2024-02-06T11:34:09Z	INFO	Initializing Scaling logic according to ScaledObject Specification	{"controller": "scaledobject", "controllerGroup": "keda.sh", "controllerKind": "ScaledObject", "ScaledObject": {"name":"security-services","namespace":"ffs"}, "namespace": "ffs", "name": "security-services", "reconcileID": "6d14dcf1-ac2c-458e-ae91-07b67d6509eb"}
2024-02-06T11:34:09Z	ERROR	scaleexecutor	failed to patch Objects	{"scaledobject.Name": "security-services", "scaledObject.Namespace": "ffs", "scaleTarget.Name": "security-services", "error": "client rate limiter Wait returned an error: context canceled"}
github.com/kedacore/keda/v2/pkg/status.TransformObject
	/workspace/pkg/status/status.go:186
github.com/kedacore/keda/v2/pkg/scaling/executor.(*scaleExecutor).updateLastActiveTime
	/workspace/pkg/scaling/executor/scale_executor.go:81
github.com/kedacore/keda/v2/pkg/scaling/executor.(*scaleExecutor).RequestScale
	/workspace/pkg/scaling/executor/scale_scaledobjects.go:153
github.com/kedacore/keda/v2/pkg/scaling.(*scaleHandler).checkScalers
	/workspace/pkg/scaling/scale_handler.go:247
github.com/kedacore/keda/v2/pkg/scaling.(*scaleHandler).startScaleLoop
	/workspace/pkg/scaling/scale_handler.go:180
2024-02-06T11:34:09Z	ERROR	scaleexecutor	Error updating last active time	{"scaledobject.Name": "security-services", "scaledObject.Namespace": "ffs", "scaleTarget.Name": "security-services", "error": "client rate limiter Wait returned an error: context canceled"}
github.com/kedacore/keda/v2/pkg/scaling/executor.(*scaleExecutor).RequestScale
	/workspace/pkg/scaling/executor/scale_scaledobjects.go:155
github.com/kedacore/keda/v2/pkg/scaling.(*scaleHandler).checkScalers
	/workspace/pkg/scaling/scale_handler.go:247
github.com/kedacore/keda/v2/pkg/scaling.(*scaleHandler).startScaleLoop
	/workspace/pkg/scaling/scale_handler.go:180

KEDA Version

2.12.1

Kubernetes Version

1.27

Platform

Google Cloud

Scaler Details

Confluent Kafka

Anything else?

No response