includeIPRanges takes precendence over excludeIPRanges #6027
Description
As touched on in #3533
We make use of both includeIPRanges and excludeIPRanges. This is because we have specific IPRanges allocated on our cross-cloud/dc setup for istio. we use includeIPRanges to ensure that istio only cares about stuff that sits within the cluster, and things can freely talk back to services on premise without additional rules.
We also use excludeIPRanges to ensure that communications to the kubernetes master don't go through envoy.
However, the problem we have is that the IPTables rules appear to be applied in this order:
-i <all our kubernetes ip alias ranges>-x kubernetes master
Which means that the kubernetes master IP, is included in the -i statement, thus always routed through envoy, the -x becomes useless.
If this were the other way around, we wouldn't have this problem :-)
FYI this manifests as ingress-nginx unable to connect to the kubernetes master until envoy has correctly configured:
I0605 09:53:17.836832 1 main.go:189] unexpected error discovering Kubernetes version (attempt Get https://10.192.64.1:443/version: dial tcp 10.192.64.1:443: connect: connection refused): 0
I0605 09:53:18.864980 1 main.go:189] unexpected error discovering Kubernetes version (attempt Get https://10.192.64.1:443/version: dial tcp 10.192.64.1:443: connect: connection refused): 1
I0605 09:53:20.457244 1 main.go:189] unexpected error discovering Kubernetes version (attempt Get https://10.192.64.1:443/version: dial tcp 10.192.64.1:443: connect: connection refused): 2
I0605 09:53:22.889220 1 main.go:189] unexpected error discovering Kubernetes version (attempt Get https://10.192.64.1:443/version: dial tcp 10.192.64.1:443: connect: connection refused): 3
I0605 09:53:26.391689 1 main.go:189] unexpected error discovering Kubernetes version (attempt Get https://10.192.64.1:443/version: dial tcp 10.192.64.1:443: connect: connection refused): 4
I0605 09:53:31.565879 1 main.go:189] unexpected error discovering Kubernetes version (attempt Get https://10.192.64.1:443/version: dial tcp 10.192.64.1:443: connect: connection refused): 5
W0605 09:53:39.675114 1 main.go:201] it was required to retry 6 times before reaching the API server
As ingress-nginx gives up after 10 retries, sometimes depending on the speed envoy configures, it will crash loop.
The reason we added -x was to totally bypass this.