Skip to content

dynamicconfig feature does not detect removal of config options #38621

@gandro

Description

@gandro

Is there an existing issue for this?

  • I have searched the existing issues

Version

equal or higher than v1.17.2 and lower than v1.18.0

What happened?

The reflectors updating the cilium-configs StateDB table do not delete entries from the table when they are removed from the corresponding upstream K8s resource (see reproduction steps below).

This seems to be because the implementation of TransformMany doesn't issue any deletions on the table for the entries which have been removed from the upstream K8s resource. Basically removing an entry from the cilium-config ConfigMap is treated as an update and wrongly only invokes table.Modify for all still existing entries. It does not invoke table.Delete for the config map fields which have been removed in the update (note that a K8s resource update does not have entry.deleted=true), meaning these StateDB entries are leaked.

cilium/pkg/k8s/statedb.go

Lines 362 to 366 in 0e64331

if _, _, err := table.Modify(txn, obj, merge); err != nil {
r.log.Error("BUG: Modify failed", logfields.Error, err)
} else {
numUpserted++
}

How can we reproduce the issue?

  1. Install Cilium with dynamic config enabled
  2. Print the contents of the cilium-configs table, e.g. via db/show cilium-configs
  3. Remove one of the config entries in the cilium-config ConfigMap. I would expect this removes the entry from the StateDB table too.
  4. Print the contents of the cilium-configs table again, e.g. via db/show cilium-configs. Notice that the entry removed from the K8s ConfigMap is still present (and therefore leaked) in the StateDB table

Cilium Version

main as of March 31, but probably also affects v1.17

Kernel Version

N/A

Kubernetes Version

Client Version: v1.31.0
Server Version: v1.31.2

Regression

No response

Sysdump

No response

Relevant log output

Anything else?

cc @ovidiutirla

Cilium Users Document

  • Are you a user of Cilium? Please add yourself to the Users doc

Code of Conduct

  • I agree to follow this project's Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/agentCilium agent related.area/k8sImpacts the kubernetes API, or kubernetes -> cilium internals translation layers.kind/bugThis is a bug in the Cilium logic.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions