XDS listener moving to NOT_SERVING #5354
Description
What version of gRPC are you using?
master
What version of Go are you using (go version)?
1.15
What operating system (Linux, Windows, …) and version?
Linux, Debian (gDebian)
What did you do?
Proxyless gRPC, using Istio echo client/server
What did you expect to see?
First request works - server starts, client connects and returns expected result.
I am then connecting to an invalid/not working destination - it fails as expected.
After that - the good destination fails.
It appears the listener gets stopped because somehow the LDS response lacks the inbound listener - however on Istiod
we log the number of listeners sent, and it is correct.
I'm still debugging
Enabled verbose logging:
(normal XDS config for listener)
{"level":"error","ts":1652467715.5276887,"caller":"xds/server.go:156","msg":"[xds][xds-server 0xc0000a45a0] Listener \"[::]:17070\" entering mode: \"SERVING\"","system":"grpc","grpc_log":true,"stacktrace":"google.golang.org/grpc/xds.(*GRPCServer).loggingServerModeChangeCallback\n\t/ws/istio.io/grpc/xds/server.go:156\ngoogle.golang.org/grpc/xds.(*GRPCServer).handleServingModeChanges\n\t/ws/istio.io/grpc/xds/server.go:321\ngoogle.golang.org/grpc/xds.(*GRPCServer).Serve.func1\n\t/ws/istio.io/grpc/xds/server.go:247"}
( normal CDS/EDS for the good service )
( CDS for the bad service )
{"level":"info","ts":1652467826.8737702,"caller":"xdsresource/unmarshal_lds.go:63","msg":"Resource with name: echo-grpc-v1.echo-grpc.svc.cluster.local:7070, type: *listenerv3.Listener, contains: {\n \"na
me\": \"echo-grpc-v1.echo-grpc.svc.cluster.local:7070\",\n \"address\": {\n \"socketAddress\": {\n \"address\": \"10.0.4.239\",\n \"portValue\": 7070\n }\n },\n \"apiListener\": {\n
\"apiListener\": {\n \"@type\": \"type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager\",\n \"rds\": {\n \"configSource\": {\n \"
ads\": {\n\n }\n },\n \"routeConfigName\": \"outbound|7070||echo-grpc-v1.echo-grpc.svc.cluster.local\"\n },\n \"httpFilters\": [\n {\n \"name\": \"envoy.fi
lters.http.fault\",\n \"typedConfig\": {\n \"@type\": \"type.googleapis.com/envoy.extensions.filters.http.fault.v3.HTTPFault\"\n }\n },\n {\n \"name\":
\"envoy.filters.http.router\",\n \"typedConfig\": {\n \"@type\": \"type.googleapis.com/envoy.extensions.filters.http.router.v3.Router\"\n }\n }\n ]\n }\n }\n}"
,"system":"grpc","grpc_log":true}
{"level":"error","ts":1652467826.8744009,"caller":"xds/server.go:158","msg":"[xds][xds-server 0xc0000a45a0] Listener \"[::]:17070\" entering mode: \"NOT_SERVING\" due to error: xds: ListenerResource target xds.istio.io/grpc/lds/inbound/[::]:17070 not found in received response, xDS client nodeID: id:\"sidecar~10.48.1.174~echo-grpc-v1-94c9f5bd-wlp9z.echo-grpc~echo-grpc.svc.cluster.local\" "stacktrace":"google.golang.org/grpc/xds.(*GRPCServer).loggingServerModeChangeCallback\n\t/ws/istio.io/grpc/xds/server.go:158\ngoogle.golang.org/grpc/xds.(*GRPCServer).handleServingModeChanges\n\t/ws/istio.io/grpc/xds/server.go:321\ngoogle.golang.org/grpc/xds.(*GRPCServer).Serve.func1\n\t/ws/istio.io/grpc/xds/server.go:247"}
...