Skip to content

Conversation

pippolo84
Copy link
Member

@pippolo84 pippolo84 commented Mar 1, 2024

Add an independent cell to manage IP sets and reconcile them dynamically through the stateDB generic reconciler.

The cell manages only two different sets, one related to the Node IPv4 addresses and the other one related to the Node IPv6 addresses. Other sets are not touched in any way.

When IP sets are disabled, the IP sets manager tries to clean both Cilium managed IP sets at startup, to avoid leaving stale entries from previous runs.

When IP sets are enabled, the IP sets manager exports two methods to add and remove IPs from a named set, respectively. The sets are updated in the relative stateDB table and the kernel state reconciled later using the ipset utility.

This is a required step in preparation for the iptables manager refactoring that allows to react to devices and local node info changes. A PoC has already been developed and can be seen here.

Note to the reviewers: please review each commit individually.

@pippolo84 pippolo84 added area/datapath Impacts bpf/ or low-level forwarding details, including map management and monitor messages. release-note/misc This PR makes changes that have no direct user impact. area/iptables Impacts how Cilium interacts with iptables. labels Mar 1, 2024
@pippolo84
Copy link
Member Author

/test

@pippolo84 pippolo84 marked this pull request as ready for review March 4, 2024 10:09
@pippolo84 pippolo84 requested review from a team as code owners March 4, 2024 10:09
@pippolo84 pippolo84 requested review from meyskens and markpash March 4, 2024 10:09
@joestringer joestringer requested review from a team March 4, 2024 18:58
joestringer

This comment was marked as resolved.

@joestringer joestringer requested review from a team March 4, 2024 19:02
@pippolo84 pippolo84 force-pushed the pr/pippolo84/ipset-reconciler branch from 3196dd5 to 1da753e Compare March 12, 2024 17:37
@pippolo84 pippolo84 requested a review from a team as a code owner March 12, 2024 17:37
@pippolo84 pippolo84 requested a review from aditighag March 12, 2024 17:37
@pippolo84 pippolo84 force-pushed the pr/pippolo84/ipset-reconciler branch from 1da753e to e8c781a Compare March 12, 2024 17:56
Copy link
Member

@gandro gandro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did a review because I was curious. Some nits, otherwise looks good to me

@pippolo84 pippolo84 force-pushed the pr/pippolo84/ipset-reconciler branch 2 times, most recently from c55c6a6 to 75daa3d Compare March 13, 2024 11:21
@pippolo84 pippolo84 requested a review from tklauser March 14, 2024 09:39
Copy link
Member

@tklauser tklauser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, LGTM! Two non-blocking comments only.

Add an immutable set data structure for use in objects stored in StateDB
tables. The ImmSet[T] is useful when storing a relatively small set of
items (1-1000). Implemented as a sorted slice.  Mutations of the set
clone the slice and lookups are implemented as binary searches.

The benefit of this is a packed presentation, especially for value types
(e.g. netip.Addr) which will cause less overhead for GC.  Downside is
that inserts and deletes become expensive when there are many items.

Signed-off-by: Jussi Maki <jussi@isovalent.com>
Signed-off-by: Fabio Falzoi <fabio.falzoi@isovalent.com>
Add a stateDB table to store IP sets.

This will be used in later commits to build an IP sets manager able to
dynamically reconcile sets with v4 and v6 node IPs.

Signed-off-by: Fabio Falzoi <fabio.falzoi@isovalent.com>
Add an independent cell to manage IP sets and reconcile them dynamically
through the stateDB generic reconciler.

The cell manages only two different sets, one related to the Node IPv4
addresses and the other one related to the Node IPv6 addresses. Other
sets are not touched in any way.

When IP sets are disabled, the IP sets manager tries to clean both
Cilium managed IP sets at startup, to avoid leaving stale entries from
previous runs.

When IP sets are enabled, the IP sets manager exports two methods to add
and remove IPs from a named set, respectively. The sets are updated in
the relative stateDB table and the kernel state reconciled later using
the `ipset` utility.

The cell will be plumbed in the iptables cell in a later commit.

Signed-off-by: Fabio Falzoi <fabio.falzoi@isovalent.com>
Remove IP sets management from iptables manager and use the new cell
based on the stateDB reconciler.

The node manager has been changed to use the new ipset manager methods.

Also, a fake version of the cell has been added to mock the ipset
manager in controlplane testing.

Signed-off-by: Fabio Falzoi <fabio.falzoi@isovalent.com>
ifName is not needed to install masquerading rules, hence remove it from
the function signature.

Signed-off-by: Fabio Falzoi <fabio.falzoi@isovalent.com>
@pippolo84 pippolo84 force-pushed the pr/pippolo84/ipset-reconciler branch from 75daa3d to 250a17f Compare March 15, 2024 12:11
@pippolo84 pippolo84 requested review from joestringer and removed request for meyskens, markpash and aditighag March 18, 2024 16:43
@pippolo84
Copy link
Member Author

Removing some reviewers whose codeowners have already been covered.
@joestringer I've fixed the CODEOWNERS for the new package, do you mind taking another look? Thanks! 🙏

@pippolo84
Copy link
Member Author

/test

@pippolo84
Copy link
Member Author

CI triage:

ci-e2e: timeout while querying quay.io
ci-ginkgo: similar issue with quay.io
ci-integration: tracked here
ci-ipsec-upgrade: tracked here

Copy link
Member

@joestringer joestringer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CODEOWNERS LGTM

@maintainer-s-little-helper maintainer-s-little-helper bot added the ready-to-merge This PR has passed all tests and received consensus from code owners to merge. label Mar 19, 2024
@julianwiedmann
Copy link
Member

This PR needs approval from current @cilium/tophat, as Dylan dropped from the team.

@julianwiedmann julianwiedmann requested a review from a team March 20, 2024 07:21
@julianwiedmann julianwiedmann added this pull request to the merge queue Mar 20, 2024
Merged via the queue into cilium:main with commit bd99b26 Mar 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/datapath Impacts bpf/ or low-level forwarding details, including map management and monitor messages. area/iptables Impacts how Cilium interacts with iptables. ready-to-merge This PR has passed all tests and received consensus from code owners to merge. release-note/misc This PR makes changes that have no direct user impact.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants