Skip to content

Istio CNI Install not using current ServiceAccount tokens #38077

@tommyp1ckles

Description

@tommyp1ckles

Bug Description

On k8s clusters running 1.21+ mounted service account tokens can be created with a expiry. I've started noticing nodes getting into states with errors like failed to set up sandbox container ... network: Unauthorized.

This appears to be due to expired or invalid tokens being used in /etc/cni/net.d/ZZZ-istio-cni-kubeconfig. Initially we thought this was an aws-vpc-cni issue since restarting that pod would fix the issue on that node.

However this appears to only fix the problem since this triggers a filesystem notify event here: https://github.com/istio/istio/blob/1.11.4/cni/pkg/install/install.go#L78 which causes a full configuration reload, including the kubeconfig file.

Should this loop also not be watching for updates to the mounted service account token as well?

Version

Istio Version: 1.11.1
Kubernetes Version: v1.21.5-eks-bc4871b

(edit: that was k8s 1.21, not 1.12).

Additional Information

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/networkinglifecycle/automatically-closedIndicates a PR or issue that has been closed automatically.lifecycle/staleIndicates a PR or issue hasn't been manipulated by an Istio team member for a while

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions