argocd app repo sync seems to pick up first argocd-server pod from list (with direct access auth) #10610
Description
Seems that argocd app repo sync tries to send requests to first pod from the list:
k get pod -l app.kubernetes.io/name=argocd-repo-server
NAME READY STATUS RESTARTS AGE
argocd-repo-server-6bdf887f59-j4ch4 0/1 Init:ContainerStatusUnknown 1 135m
argocd-repo-server-6bdf887f59-p9nh5 0/1 Running 0 67s
argocd-repo-server-6bdf887f59-qt2sr 1/1 Running 0 20h
argocd-repo-server-6bdf887f59-swv7h 0/1 Init:ContainerStatusUnknown 1 2m2s
argocd app sync app
FATA[0002] rpc error: code = FailedPrecondition desc = error getting repo server client: error upgrading connection: unable to upgrade connection: pod not found ("argocd-repo-server-6bdf887f59-j4ch4_argocd")
In my case, these pods where evicted due to node condition. Once ghost pod gets removed, it works again:
k delete pod argocd-repo-server-6bdf887f59-j4ch4
pod "argocd-repo-server-6bdf887f59-j4ch4" deleted
k get pod -l app.kubernetes.io/name=argocd-repo-server
NAME READY STATUS RESTARTS AGE
argocd-repo-server-6bdf887f59-p9nh5 0/1 Running 0 67s
argocd-repo-server-6bdf887f59-qt2sr 1/1 Running 0 20h
argocd-repo-server-6bdf887f59-swv7h 0/1 Init:ContainerStatusUnknown 1 2m2s
argocd app sync app
TIMESTAMP GROUP KIND NAMESPACE NAME STATUS HEALTH HOOK MESSAGE
...
omitted
...
Would be good if there is some sort of failover, or even check pod status before trying to setup connection.