Skip to content

Tree states to support per-slot state diffs  #4475

New issue
New issue
Closed
Closed
Tree states to support per-slot state diffs #4475
Assignees
jimmygchenmichaelsproul
Labels
databasetree-statesOngoing state and database overhaul
@jimmygchen

Description

@jimmygchen
jimmygchen
opened on Jul 7, 2023

Description

Currently the HDiff logic does not support per-slot state diffs, and it would be nice to support this.

This requires the following (from @michaelsproul):

  • Re-writing the epoch-centric code in HDiff to work with slots instead.
  • Changing the on-disk DB format as the above change will break it.
  • Add some config for the "hierarchy exponents" to the OnDiskStoreConfig

Relevant code:

lighthouse/beacon_node/store/src/hdiff.rs

Lines 206 to 255 in 079cd67

impl HierarchyConfig {
pub fn to_moduli(&self) -> Result<HierarchyModuli, Error> {
self.validate()?;
let moduli = self.exponents.iter().map(|n| 1 << n).collect();
Ok(HierarchyModuli { moduli })
}
pub fn validate(&self) -> Result<(), Error> {
if self.exponents.len() > 2
&& self
.exponents
.iter()
.tuple_windows()
.all(|(small, big)| small < big && *big < u64::BITS as u8)
{
Ok(())
} else {
Err(Error::InvalidHierarchy)
}
}
}
impl HierarchyModuli {
pub fn storage_strategy(&self, epoch: Epoch) -> Result<StorageStrategy, Error> {
let last = self.moduli.last().copied().ok_or(Error::InvalidHierarchy)?;
if epoch % last == 0 {
return Ok(StorageStrategy::Snapshot);
}
let diff_from = self.moduli.iter().rev().find_map(|&n| {
(epoch % n == 0).then(|| {
// Diff from the previous state.
(epoch - 1) / n * n
})
});
Ok(diff_from.map_or(StorageStrategy::Nothing, StorageStrategy::DiffFrom))
}
/// Return the smallest epoch greater than or equal to `epoch` at which a full snapshot should
/// be stored.
pub fn next_snapshot_epoch(&self, epoch: Epoch) -> Result<Epoch, Error> {
let last = self.moduli.last().copied().ok_or(Error::InvalidHierarchy)?;
if epoch % last == 0 {
Ok(epoch)
} else {
Ok((epoch / last + 1) * last)
}
}
}

@michaelsproul I'll likely need some help understanding this, so adding you if you have some time to pair or chat!

Metadata

Metadata

Assignees

Labels

databasetree-statesOngoing state and database overhaul

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions