-
Notifications
You must be signed in to change notification settings - Fork 697
Closed
Labels
kind/bugCategorizes issue or PR as related to a bug.Categorizes issue or PR as related to a bug.
Description
Contour occasionally gets stuck in a non-ready state and fails to start the XDS server. This issue is uncommon and occurs only under the following conditions:
--watch-namespaces
flag is set to monitor a single namespace.- That namespace has only "unrelated" secrets that Contour does not process.
- The stuck Contour instance is the follower (not leader).
To reproduce this issue, follow these steps:
- Deploy contour:
$ kubectl apply -f https://projectcontour.io/quickstart/contour.yaml
- Create a nearly empty namespace with one generic secret:
$ kubectl create ns almost-empty
$ kubectl -n almost-empty create secret generic my-secret --from-literal=secret=value
- Configure Contour to watch only the almost-empty namespace:
$ kubectl patch deployment contour -n projectcontour --type='json' -p='[{"op": "add", "path": "/spec/template/spec/containers/0/args/-", "value": "--watch-namespaces=almost-empty"}]'
- Restart both Contour instances by scaling down and back up:
$ kubectl -n projectcontour scale deployment --replicas=0 contour
$ kubectl -n projectcontour scale deployment --replicas=2 contour
- Verify that one Contour instance remains stuck in a non-ready state:
$ kubectl -n projectcontour get pod -l app=contour
NAME READY STATUS RESTARTS AGE
contour-db59b775d-9k4wl 1/1 Running 0 4m17s
contour-db59b775d-tnqxm 0/1 Running 0 4m17s
The problem is a race condition, and its likelihood decreases if more resources (or no resources) are created in the watched namespace.
Metadata
Metadata
Assignees
Labels
kind/bugCategorizes issue or PR as related to a bug.Categorizes issue or PR as related to a bug.