What steps did you take and what happened:
[A clear and concise description of what the bug is.]

1. Install contour by Helm, the contour.replicaCount is 2, and the envoy service config is this:

envoy:
  service:
    type: LoadBalancer

2. One of the contour controller pod memory keep increasing until OOM.

What did you expect to happen:

All contour controller pod works well.

Anything else you would like to add:

The pod memory heap (generated by https://github.com/cloudwego/goref):

The pod goroutine:

Checked the goroutine of the pod, the service events consumer is hangs on ServiceStatusLoadBalancerWatcher.notify.

On non-leader contour controller pod, loadBalancerStatusWriter will not start, it will cause ServiceStatusLoadBalancerWatcher.notify blocking writing to channel.

And then, all service events are cached in pendingNotifications.ringGrowing and will not be consumed. And as new events are generated, more events are cached in pendingNotifications.ringGrowing, and the memory continues to increase, final OOM.

Environment:

• Contour version: 1.27.0
• Kubernetes version: (use kubectl version): 1.26.15
• Kubernetes installer & version:
• Cloud provider or hardware configuration:
• OS (e.g. from /etc/os-release):