-
Notifications
You must be signed in to change notification settings - Fork 8.1k
Closed
Description
Hey,
Istio 0.7.1, GKE 1.9.7
Noticed a few pilot reboots on one of our clusters, and these logs in discovery:
9 195 [kubeops.core-system.svc.cluster.local|http] version_info:"195" node:<id:"sidecar~10.198.0.9~ingress-nginx-external-controller-6f8946bcc7-wrwtm.ingress-nginx~ingress-nginx.svc.cluster.local" cluster:"ingress-nginx-external" build_version:"0/1.6.0//RELEASE" > resource_names:"kubeops.core-system.svc.cluster.local|http" type_url:"type.googleapis.com/envoy.api.v2.ClusterLoadAssignment" response_nonce:"195-2018-05-21 16:07:32.54788636 +0000 UTC m=+16773.986512881"
2018-05-21T16:07:33.626301Z info Handle pod calico-node-l4n8f in namespace kube-system -> 10.193.32.28
2018-05-21T16:07:33.632832Z info Handle pod calico-node-8p8fz in namespace kube-system -> 10.193.32.30
2018-05-21T16:07:33.636289Z info Handle pod calico-node-g5nkx in namespace kube-system -> 10.193.32.29
2018-05-21T16:07:33.641238Z info Handle pod filebeat-m8vzh in namespace kube-system -> 10.198.5.2
2018-05-21T16:07:33.646326Z info Handle pod calico-typha-67c95cc844-wr6fg in namespace kube-system -> 10.193.32.19
2018-05-21T16:07:33.651476Z info Handle pod cert-manager-7bcdd6768-qtr79 in namespace kube-system -> 10.198.0.19
2018-05-21T16:07:33.656456Z info Handle pod calico-node-k7dm6 in namespace kube-system -> 10.193.32.3
2018-05-21T16:07:33.661301Z info Handle pod calico-node-zcgg8 in namespace kube-system -> 10.193.32.19
2018-05-21T16:07:33.666365Z info Handle pod calico-node-jzz4z in namespace kube-system -> 10.193.32.2
panic: interface conversion: interface {} is cache.DeletedFinalStateUnknown, not *v1.Pod
goroutine 48 [running]:
istio.io/istio/pilot/pkg/serviceregistry/kube.newPodCache.func1(0x162b8a0, 0xc420de4360, 0x2, 0x0, 0x0)
/workspace/go/src/istio.io/istio/pilot/pkg/serviceregistry/kube/cache.go:47 +0x5f0
istio.io/istio/pilot/pkg/serviceregistry/kube.(*ChainHandler).Apply(0xc42000a080, 0x162b8a0, 0xc420de4360, 0x2, 0xc42096a000, 0x1)
/workspace/go/src/istio.io/istio/pilot/pkg/serviceregistry/kube/queue.go:144 +0x81
istio.io/istio/pilot/pkg/serviceregistry/kube.(*ChainHandler).Apply-fm(0x162b8a0, 0xc420de4360, 0x2, 0x0, 0x0)
/workspace/go/src/istio.io/istio/pilot/pkg/config/kube/crd/controller.go:127 +0x48
istio.io/istio/pilot/pkg/serviceregistry/kube.(*queueImpl).Run(0xc4203fda70, 0xc42007a5a0)
/workspace/go/src/istio.io/istio/pilot/pkg/serviceregistry/kube/queue.go:124 +0x18c
created by istio.io/istio/pilot/pkg/serviceregistry/kube.(*Controller).Run
/workspace/go/src/istio.io/istio/pilot/pkg/serviceregistry/kube/controller.go:183 +0x54