A slow ScaledObject's SQL query results in keda-operator in CrashLoopBackOff #6725
Description
Report
A slow ScaledObject's SQL query results in a keda-operator restart and eventually ends up in CrashLoopBackOff.
In our case the SQL query resulted in a full table scan, and the keda-operator restarted before the database returned the result.
Expected Behavior
Keda-operator logs an error, but does not crash.
The ScaledObject's status field is updated with the error message explaining why it has failed.
Actual Behavior
PostgreSQLScaler fails.
Keda-operator crashes and eventually ends in CrashLoopBackOff.
Steps to Reproduce the Problem
- Create a ScaledObject with a trigger based on a long running SQL statement
- Keda-operator crashes and restarts
Logs from KEDA operator
The error in the keda-operator pod is
"error inspecting postgreSQL: could not query postgreSQL: context canceled"
followed by
"panic: runtime error: index out of range [0] with length 0"
2025-04-23T11:51:14Z ERROR postgresql_scaler could not query postgreSQL: context canceled {"type": "ScaledObject", "namespace": "x", "name": "y", "error": "context canceled"}
github.com/kedacore/keda/v2/pkg/scalers.(*postgreSQLScaler).getActiveNumber
/workspace/pkg/scalers/postgresql_scaler.go:218
github.com/kedacore/keda/v2/pkg/scalers.(*postgreSQLScaler).GetMetricsAndActivity
/workspace/pkg/scalers/postgresql_scaler.go:240
github.com/kedacore/keda/v2/pkg/scaling/cache.(*ScalersScaler).GetMetricsAndActivityForScaler
/workspace/pkg/scaling/cahce/scalers_cache.go:161
github.com/kedacore/keda/v2/pkg/scaling.(*scaleHandler).GetSCaledObjectMetrics.func1
/workspace/pkg/scaling/scale_handler.go:524
2025-04-23T11:51:14Z ERROR scale_handler error getting metric for trigger {"scaledObject.Namespace": "x", "scaledObject.Name": "y", "trigger": "postgreSQLScaler", "error": "error inspecting postgreSQL: could not query postgreSQL: context canceled"}
github.com/kedacore/keda/v2/pkg/scaling.(*scaleHandler).GetSCaledObjectMetrics
/workspace/pkg/scaling/scale_handler.go:553
github.com/kedacore/keda/v2/pkg/metricsservice.(*GrpcServer).GetMetrics
/workspace/pkg/metricsservice/server.go:48
github.com/kedacore/keda/v2/pkg/metricsservice/api._MetricsService_GetMetrics_Handler
/workspace/pkg/metricsservice/api/metrics_grpc.pb.go:111
google.golang.org/grpc.(*Server).processUnaryRPC
/workspace/vendor/google.golang.org/grpc/server.go:1394
google.golang.org/grpc.(*Server).handleStream
/workspace/vendor/google.golang.org/grpc/server.go:1805
google.golang.org/grpc.(*Server).sandleStreams.func2.1
/workspace/vendor/google.golang.org/grpc/server.go:1029
panic: runtime error: index out of range [0] with length 0
KEDA Version
2.16.1
Kubernetes Version
1.31
Platform
Microsoft Azure
Scaler Details
postgreSQLScaler
Anything else?
No response