Is there an existing issue for this?

• I have searched the existing issues

What happened?

When creating an external workload, the tunnel and host reachable services work for somewhere between 5-10 minutes on the external node, then stop working. A 5 minutes nodes-gc-interval is the default, and depending on where in this cycle the external workload is brought up, it will be garbage collected betweeen 5 and 10 minutes - first gc pass flags the external node for GC, second pass actually deletes the node.

Cilium Version

1.11.5

Kernel Version

4.18.0-372.9.1.el8.x86_64

Kubernetes Version

Client Version: version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.0", GitCommit:"4ce5a8954017644c5420bae81d72b09b735c21f0", GitTreeState:"clean", BuildDate:"2022-05-03T13:46:05Z", GoVersion:"go1.18.1", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v4.5.4

Sysdump

I can add this if you absolutely need it, but I think that will be more info than necessary

Relevant log output

2022-05-20T13:35:17.717318391-04:00 level=info msg="Perform GC for invalid CiliumNode" nodeName=plainolddocker01 subsys=watchers
2022-05-20T13:35:17.736547246-04:00 level=info msg="CiliumNode is garbage collected successfully" nodeName=plainolddocker01 subsys=watchers

Anything else?

https://github.com/cilium/cilium/pull/17329/commits

I believe this is the PR that introduced this issue. The long and short is that external workload nodes aren't associated with k8s nodes, and they don't have ownership info. So performCiliumNodeGC will 100% of the time delete nodes associated with CEW's. So I think it would be fairly easy to address in one of the following ways:

1. Add ownership information to the cilium nodes, possibly associated with the cew that allowed it to be created.
2. update performCiliumNodeGC and co to check to see if a node is associated with a CEW, and if so, don't mark for GC.

I'd be willing to submit a PR if I could get some guidance on the most appropriate path forward.

For what it's worth, setting nodes-gc-interval to 0 solves the issue

Code of Conduct

• I agree to follow this project's Code of Conduct