Skip to content

Conversation

michaelsproul
Copy link
Member

@michaelsproul michaelsproul commented Mar 3, 2025

Use a new Milhouse method called intra_rebase to reuse identical subtrees in inactivity_scores. This reduces the memory used by each inactivity_scores update from ~70MB+ to ~4-5MB.

To update to milhouse's main branch I needed to steal Paul's changes from this PR, which I squashed into a single commit to make it easier to remove unwanted changes from unstable:

I tested this change using an even hackier branch here, which includes the Milhouse mem-usage changes: https://github.com/michaelsproul/lighthouse/tree/milhouse-mem-size-holesky-rescue

Command for testing:

RUST_LOG=debug lcli skip-slots --pre-state-path state_3714912.ssz --slots 32 --network holesky

Results:

Inactivity score intra-rebase time: 47ms
Post-state balances size (total/diff): 84594016-84594016 B/79744688 B
Post-state validators size: 522924016-522924016 B/9416 B
Post-state inactivity_scores size: 84596096-84596096 B/4696384 B

@michaelsproul michaelsproul requested a review from jxs as a code owner March 3, 2025 04:57
@michaelsproul michaelsproul changed the title Intra rebase Use milhouse intra_rebase to shrink inactivity_scores in memory Mar 3, 2025
@michaelsproul michaelsproul changed the title Use milhouse intra_rebase to shrink inactivity_scores in memory Reuse milhouse subtrees to shrink inactivity_scores in memory Mar 3, 2025
@michaelsproul michaelsproul added v7.0.0-beta.2 Second hacky Holesky rescue release optimization Something to make Lighthouse run more efficiently. labels Mar 3, 2025
@michaelsproul michaelsproul merged commit 6399ad4 into holesky-rescue Mar 3, 2025
22 of 28 checks passed
@michaelsproul michaelsproul deleted the intra-rebase branch March 3, 2025 05:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
optimization Something to make Lighthouse run more efficiently. v7.0.0-beta.2 Second hacky Holesky rescue release
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant