Skip to content

Conversation

lmb
Copy link
Contributor

@lmb lmb commented Jun 7, 2024

loader: evict object cache when datapath config changes

The object cache currently does no invalidation, which means that we
accumulate cachedObject in memory and template ELF on disk. Use update of
the base datapath hash as an opportunity to evict some of that cache.

In practice this is probably not a big issue: datapath config changes rarely
if every, and we only have templates for endpoints and host endpoint.

Signed-off-by: Lorenz Bauer <lmb@isovalent.com>

loader: cache parsed CollectionSpec

The object cache parses an ELF from disk any time it is asked for a 
template. This is wasteful since parsing the ELF is quite resource 
intensive.

Cache the parsed CollectionSpec instead.

Signed-off-by: Lorenz Bauer <lmb@isovalent.com>

@lmb lmb added area/loader Impacts the loading of BPF programs into the kernel. release-note/misc This PR makes changes that have no direct user impact. labels Jun 7, 2024
@lmb
Copy link
Contributor Author

lmb commented Jun 7, 2024

/test

@lmb lmb force-pushed the pr/lmb/fix-cache-growth branch from b7a14e0 to 14787f4 Compare June 7, 2024 14:06
@lmb
Copy link
Contributor Author

lmb commented Jun 7, 2024

/test

@lmb lmb changed the title loader: evict object cache when datapath config changes loader: cache parsed CollectionSpec Jun 7, 2024
@lmb lmb changed the title loader: cache parsed CollectionSpec loader: cache parsed CollectionSpec Jun 7, 2024
@lmb
Copy link
Contributor Author

lmb commented Jun 7, 2024

/test

@lmb lmb force-pushed the pr/lmb/fix-cache-growth branch from e9245d3 to 09dc6b9 Compare June 7, 2024 15:34
@lmb
Copy link
Contributor Author

lmb commented Jun 7, 2024

/test

@lmb lmb force-pushed the pr/lmb/fix-cache-growth branch from 09dc6b9 to 6a58849 Compare June 10, 2024 07:56
@lmb
Copy link
Contributor Author

lmb commented Jun 10, 2024

/test

@lmb lmb marked this pull request as ready for review June 10, 2024 09:00
@lmb lmb requested review from a team as code owners June 10, 2024 09:00
@lmb lmb requested review from rgo3, gentoo-root and bimmlerd June 10, 2024 09:00
Copy link
Member

@bimmlerd bimmlerd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice, one question and a few nits

@lmb lmb force-pushed the pr/lmb/fix-cache-growth branch from 6a58849 to dd7269d Compare June 10, 2024 14:39
@lmb
Copy link
Contributor Author

lmb commented Jun 11, 2024

@rgo3 @gentoo-root can you help me get this in before the freeze? 🙏

Copy link
Contributor

@rgo3 rgo3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@lmb lmb enabled auto-merge June 13, 2024 07:54
@lmb
Copy link
Contributor Author

lmb commented Jun 13, 2024

/test

lmb added 2 commits June 13, 2024 10:01
The object cache currently does no invalidation, which means that
we accumulate cachedObject in memory and template ELF on disk.
Use update of the base datapath hash as an opportunity to evict
some of that cache.

In practice this is probably not a big issue: datapath config changes
rarely if every, and we only have templates for endpoints and host endpoint.

Signed-off-by: Lorenz Bauer <lmb@isovalent.com>
The object cache parses an ELF from disk any time it is asked for a
template. This is wasteful since parsing the ELF is quite resource
intensive.

Cache the parsed CollectionSpec instead.

Signed-off-by: Lorenz Bauer <lmb@isovalent.com>
@lmb lmb force-pushed the pr/lmb/fix-cache-growth branch from dd7269d to c459aa8 Compare June 13, 2024 08:01
@lmb
Copy link
Contributor Author

lmb commented Jun 13, 2024

/test

1 similar comment
@lmb
Copy link
Contributor Author

lmb commented Jun 13, 2024

/test

@lmb lmb added this pull request to the merge queue Jun 13, 2024
Merged via the queue into main with commit 55a9d2e Jun 13, 2024
@lmb lmb deleted the pr/lmb/fix-cache-growth branch June 13, 2024 12:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/loader Impacts the loading of BPF programs into the kernel. release-note/misc This PR makes changes that have no direct user impact.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants