Skip to content

Kvstoremesh heartbeat with external etcd #39356

@balous

Description

@balous

Is there an existing issue for this?

  • I have searched the existing issues

Version

equal or higher than v1.17.3 and lower than v1.18.0

What happened?

I am running kvstoremesh with external etcd without the apiserver container. My problem is that the heartbeat is not updated in this setup (I was warned about this here).

I would like to start a discussion about how this problem should be solved. I've got some ideas:

  • apiserver contains --enable-k8s option, which, when set to false, causes apiserver to fail. OK but it seems to be named exactly like I need to - disable k8s features and retain heartbeat updating. So my idea is to implement it this way. This approach would later allow us to add kvstore identities sync (cilium etcd -> external or even internal etcd) which would allow us to implement uniform clustermesh interconnection not depending on identity allocation mode ( crd or kvstore, both would be synced into the kvstoremesh etcd).
  • add/move heartbeat to kvstoremesh. apiserver seems to be the primary container but it is disabled in some scenarios so may be we could reconsider this.
  • add extraContainers options which would allow the user to add a while true ; do date | etcdctl put cilium.heartbeat' ; done container (ugly)

So, @giorio94 or others, what do you think abou this?

How can we reproduce the issue?

  1. install cilium with helm
  2. set clustermesh.apiserver.kvstoremesh.kvstoreMode to external
  3. direct kvstoremesh to an dedicated empty etcd cluster which is not used for identity allocation

Cilium Version

N/A

Kernel Version

N/A

Kubernetes Version

N/A

Regression

No response

Sysdump

No response

Relevant log output

Anything else?

No response

Cilium Users Document

  • Are you a user of Cilium? Please add yourself to the Users doc

Code of Conduct

  • I agree to follow this project's Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/agentCilium agent related.area/clustermeshRelates to multi-cluster routing functionality in Cilium.area/operatorImpacts the cilium-operator componentkind/community-reportThis was reported by a user in the Cilium community, eg via Slack.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions