CI: ci-integration: pkg/identity/cache: level=fatal msg="Unable to connect to kvstore" error="timed out while starting the heartbeat watcher. #37689
Description
CI failure
PR is not touching cilium-agent code, only cilium-cli: #37687
Failure: https://github.com/cilium/cilium/actions/runs/13379380399/job/37365114728#step:14:112
CHECK running integration tests...
INTEGRATION_TESTS=true CGO_ENABLED=0 go test -mod=vendor -vet=all -tags=osusergo ./... -timeout 720s -json -covermode=count -coverprofile=coverage.out | tee >(go run ./tools/testowners) >(tparse ) >/dev/null
make: *** [Makefile:133: integration-tests] Error 1
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ FAIL package: github.com/cilium/cilium/pkg/identity/cache ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
time="2025-02-17T22:27:17.563258101Z" level=info msg="Creating etcd client" ConfigPath= KeepAliveHeartbeat=15s KeepAliveTimeout=25s ListLimit=256 MaxInflight=20 RateLimit=20 subsys=kvstore
time="2025-02-17T22:27:17.563526426Z" level=info msg="Connecting to etcd server..." config= endpoints="[http://127.0.0.1:4002]" subsys=kvstore
{"level":"warn","ts":"2025-02-17T22:27:17.563656Z","logger":"etcd-client","caller":"v3/retry_interceptor.go:63","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc0002592c0/127.0.0.1:4002","attempt":90,"error":"rpc error: code = Canceled desc = grpc: the client connection is closing"}
{"level":"warn","ts":"2025-02-17T22:27:17.566064Z","logger":"etcd-client","caller":"v3/retry_interceptor.go:63","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc0002592c0/127.0.0.1:4002","attempt":90,"error":"rpc error: code = Canceled desc = grpc: the client connection is closing"}
time="2025-02-17T22:27:17.566849725Z" level=info msg="New lease successfully acquired" LeaseID=53267[96](https://github.com/cilium/cilium/actions/runs/13379380399/job/37365114728#step:14:97)380962447813 TTL=25s config= endpoints="[http://127.0.0.1:4002]" subsys=kvstore
time="2025-02-17T22:27:17.567362159Z" level=info msg="Starting watcher" config= endpoints="[http://127.0.0.1:4002]" prefix=cilium/.heartbeat subsys=kvstore
time="2025-02-17T22:27:17.569920881Z" level=info msg="New lease successfully acquired" LeaseID=5326796380962447818 TTL=15m0s config= endpoints="[http://127.0.0.1:4002]" subsys=kvstore
time="2025-02-17T22:27:17.571619626Z" level=fatal msg="Unable to connect to kvstore" error="timed out while starting the heartbeat watcher. Ensure that etcd is running on [http://127.0.0.1:4002]" module=etcd subsys=kvstore
@cilium/sig-policy how can we make this test more reliable?