Skip to content

Conversation

danielabrozzoni
Copy link
Contributor

@danielabrozzoni danielabrozzoni commented Jun 12, 2025

This is a partial* revival of #25968

It contains a list of most-unrelated simplifications and optimizations to the code merged in #25717:

  • Avoid an IsAncestorOfBestHeaderOrTip call: Just don't call this function when it won't have any effect.
  • Compute work from headers without CBlockIndex: Avoid the need to construct a CBlockIndex object just to compute work for a header, when its nBits value suffices for that. Also use some Spans where possible.
  • Remove useless CBlock::GetBlockHeader: There is no need for a function to convert a CBlock to a CBlockHeader, as it's a child class of it.

It also contains the following code cleanups, which were suggested by reviewers in #25968:

  • Remove redundant parameter from CheckHeadersPoW: No need to pass consensusParams, as CheckHeadersPow already has access to m_chainparams.GetConsensus()
  • Remove unused parameter in ReportHeadersPresync
  • Use initializer list in CompressedHeader, also make GetFullHeader const
  • Use reference for chain_start in HeadersSyncState: chain_start can never be null, so it's better to pass it as a reference rather than a raw pointer

*I decided to leave out three commits that were in #25968 (4e7ac7b, ab52fb4, 7f1cf44), since they're a bit more involved, and I'm a new contributor. If this PR gets merged, I'll comment under #25968 to note that these three commits are still up for grabs :)

@DrahtBot
Copy link
Contributor

DrahtBot commented Jun 12, 2025

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/32740.

Reviews

See the guideline for information on the review process.

Type Reviewers
ACK hodlinator, l0rinc

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #32579 (headerssync: Preempt unrealistic unit test behavior by hodlinator)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@ajtowns
Copy link
Contributor

ajtowns commented Jun 13, 2025

Maybe change the title to describe what's actually going on for those of us who don't memorise every pr number? :) ("Header sync optimisations/simplifications" ?)

@danielabrozzoni danielabrozzoni changed the title refactor: Optimizations & simplifications following #25717 refactor: Header sync optimisations & simplifications #25717 Jun 13, 2025
@danielabrozzoni danielabrozzoni changed the title refactor: Header sync optimisations & simplifications #25717 refactor: Header sync optimisations & simplifications Jun 13, 2025
Copy link
Contributor

@hodlinator hodlinator left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 13e123e

Thanks for picking this up!

Conflicts somewhat with #32579 where I was aiming to change the headers-parameter to a span. But if we follow through and also take my enum suggestion I would still see it as an improvement on balance (see inline comment).

@@ -140,33 +140,30 @@ class HeadersSyncState {

/** Result data structure for ProcessNextHeaders. */
struct ProcessingResult {
std::vector<CBlockHeader> pow_validated_headers;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not thrilled by increasing the number of in/out parameters - it pushes readers towards having to lean on comments more, rather than having the code somewhat self-documenting. It also makes the unit tests more cumbersome. What makes the change somewhat acceptable is that it conforms to the style of net_processing.cpp.

If we keep this, I would also suggest further simplification of ProcessingResult through converting it to an enum: f2a6179

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for dropping the commit introducing the in/out parameter!

@danielabrozzoni
Copy link
Contributor Author

@hodlinator thank you for your review!

I like the idea of converting ProcessingResult to an enum, I think it makes the code easier to read.

As for pow_validated_headers, I don’t have a strong preference between using a span or an argument for I/O just yet. I’ll take a closer look at #32579 over the next few days and should have a more informed opinion afterward :) I’d also be interested to hear what other reviewers think!

Latest push:

  • style improvements based on review comments

Copy link
Contributor

@l0rinc l0rinc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to review this - can you tell me how to measure the effect of the "optimizations" using either the microbenchmarks or a reindex or anything that shows the magnitude of change?
Would be helpful if you could publish your measurements if you have any.

@danielabrozzoni
Copy link
Contributor Author

danielabrozzoni commented Jul 7, 2025

Last push:

I'd like to review this - can you tell me how to measure the effect of the "optimizations" using either the microbenchmarks or a reindex or anything that shows the magnitude of change?
Would be helpful if you could publish your measurements if you have any.

I don't have any, working on it!

Copy link
Contributor

@l0rinc l0rinc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concept ACK, I'm not sure about the first change but the rest make sense.

reviewed f4b33a1, left some comments for most commits, my main concern is the commit where we're reusing the headers vector, making the code significantly more contrived.
It would also be great if we could extract the common parts of this and @hodlinator's change and merge that (or his change) before we do this.

@@ -4365,7 +4365,7 @@ void PeerManagerImpl::ProcessMessage(CNode& pfrom, const std::string& msg_type,
MaybeSendGetHeaders(pfrom, GetLocator(m_chainman.m_best_header), *peer);
}
return;
} else if (prev_block->nChainWork + CalculateClaimedHeadersWork({{cmpctblock.header}}) < GetAntiDoSWorkThreshold()) {
} else if (prev_block->nChainWork + GetBlockProof(cmpctblock.header) < GetAntiDoSWorkThreshold()) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice!

@danielabrozzoni
Copy link
Contributor Author

After some thinking, I decided to drop the first commit, and let ProcessNextHeaders take the headers argument and return it, instead of using it as a I/O argument. I tried to write a benchmark to measure how much using the vector as I/O argument would optimize the code, but it convinced me that the new interface would be too hard to work with, as reviewers pointed out. While it might still be a performance improvement, I'm not sure if it's really worth it.

For anyone interested, here's my attempt at writing a benchmark: 852aca5 and my branch
(Unfortunately, since ProcessNextHeaders changes the content of the headers vector passed in, I had to copy the vector every single time, which I suppose could make the benchmark a bit unreliable? There are also a few assertions in the bench.run() function, which I used to make sure that my code wasn't bugged, but I'm not sure if they should be there in any serious attempt at benchmarking)

Will update the PR in the next few days :)

@danielabrozzoni danielabrozzoni force-pushed the upforgrabs/25968 branch 2 times, most recently from 4d95119 to f33bd2f Compare August 8, 2025 12:46
Copy link
Contributor Author

@danielabrozzoni danielabrozzoni left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my last push (f33bd2f):

  • Removed first commit as per my last comment (#32740 (comment))
  • Addressed review comments

src/validation.h Outdated
@@ -394,7 +394,7 @@ bool TestBlockValidity(BlockValidationState& state,
bool fCheckMerkleRoot = true) EXCLUSIVE_LOCKS_REQUIRED(cs_main);

/** Check with the proof of work on each blockheader matches the value in nBits */
bool HasValidProofOfWork(const std::vector<CBlockHeader>& headers, const Consensus::Params& consensusParams);
bool HasValidProofOfWork(const std::span<const CBlockHeader> headers, const Consensus::Params& consensusParams);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes a lot of sense, thank you!
I only removed the const, I prefer to keep the argument names

@@ -4159,9 +4159,9 @@ std::vector<unsigned char> ChainstateManager::GenerateCoinbaseCommitment(CBlock&
return commitment;
}

bool HasValidProofOfWork(const std::vector<CBlockHeader>& headers, const Consensus::Params& consensusParams)
bool HasValidProofOfWork(const std::span<const CBlockHeader> headers, const Consensus::Params& consensusParams)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I removed the const and put this change in a separate commit

Copy link
Contributor

@l0rinc l0rinc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

code review ACK f33bd2f

nit: the PR isn't really an optimization anymore, consider updating the title

@@ -4149,8 +4149,8 @@ std::vector<unsigned char> ChainstateManager::GenerateCoinbaseCommitment(CBlock&

bool HasValidProofOfWork(const std::vector<CBlockHeader>& headers, const Consensus::Params& consensusParams)
{
return std::all_of(headers.cbegin(), headers.cend(),
[&](const auto& header) { return CheckProofOfWork(header.GetHash(), header.nBits, consensusParams);});
return std::ranges::all_of(headers,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: this change could either be merged with the vector-to-std::span or moved over to that commit, it's not needed in 80c63c9

Copy link
Contributor

@hodlinator hodlinator Aug 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can see it belonging in either commit, as one modifies the container and the other modifies what operators we apply to it.

Edit after #32740 (comment) below: Now realize I was mixing up CheckProofOfWork with GetBlockProof - we are still applying the same thing.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it fits slightly better with the span change... since in c++20 span doesn't have a cbegin method, we needed to change the initial code, and update to ranges::all_of

@@ -101,18 +101,6 @@ class CBlock : public CBlockHeader
m_checked_merkle_root = false;
}

CBlockHeader GetBlockHeader() const
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice!

m_current_chain_work(chain_start->nChainWork),
m_last_header_received(m_chain_start->GetBlockHeader()),
m_current_height(chain_start->nHeight)
const CBlockIndex& chain_start, const arith_uint256& minimum_required_work) : m_commit_offset(FastRandomContext().randrange<unsigned>(HEADER_COMMITMENT_PERIOD)),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: if you touch again, please format this differently

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree using style much closer to the original would be very welcome. Listing the member initializers at the indentation level where the function arguments end makes potential comments to the right of those arguments disappear out of sight etc.

Would also be nice to use more common style for CompressedHeader but at least there the initializer list didn't exist before.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I didn't like the initial formatting either, but I thought it was common (looking at other constructors in the repository, no, it's not...)

I updated the formatting both in HeadersSyncState and CompressedHeaders!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Still have a slight preference for using the original way of formatting the HeadersSyncState initializer-list for optimal git blame integrity, but this is tolerable compared to the previous push, I see it matches PeerManagerImpl).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would still keep the reformatting as clang-format doesn't like the one on master, and reformats it as I previously did (with the member initializers pushed to the right). At least this reformat is nice and clang-format doesn't try to modify it.

I hadn't considered git blame though, I don't love the idea of polluting it...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, so clang-format was pushing for it... seems it could do with some reconfiguration in the initializer-list aspect.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After #32813 we could customize the list formatting as well

Copy link
Contributor

@hodlinator hodlinator left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed f33bd2f

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Inline comment in random place).

It seems like the PR has an odd parent commit? Being based upon 11a2d3a which is within another PR's commits, rather than the merge into master which was 9f3dcac.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ooops, not sure what happened there, thanks for noticing!

@@ -140,33 +140,30 @@ class HeadersSyncState {

/** Result data structure for ProcessNextHeaders. */
struct ProcessingResult {
std::vector<CBlockHeader> pow_validated_headers;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for dropping the commit introducing the in/out parameter!

@@ -4149,8 +4149,8 @@ std::vector<unsigned char> ChainstateManager::GenerateCoinbaseCommitment(CBlock&

bool HasValidProofOfWork(const std::vector<CBlockHeader>& headers, const Consensus::Params& consensusParams)
{
return std::all_of(headers.cbegin(), headers.cend(),
[&](const auto& header) { return CheckProofOfWork(header.GetHash(), header.nBits, consensusParams);});
return std::ranges::all_of(headers,
Copy link
Contributor

@hodlinator hodlinator Aug 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can see it belonging in either commit, as one modifies the container and the other modifies what operators we apply to it.

Edit after #32740 (comment) below: Now realize I was mixing up CheckProofOfWork with GetBlockProof - we are still applying the same thing.

m_current_chain_work(chain_start->nChainWork),
m_last_header_received(m_chain_start->GetBlockHeader()),
m_current_height(chain_start->nHeight)
const CBlockIndex& chain_start, const arith_uint256& minimum_required_work) : m_commit_offset(FastRandomContext().randrange<unsigned>(HEADER_COMMITMENT_PERIOD)),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree using style much closer to the original would be very welcome. Listing the member initializers at the indentation level where the function arguments end makes potential comments to the right of those arguments disappear out of sight etc.

Would also be nice to use more common style for CompressedHeader but at least there the initializer list didn't exist before.

src/validation.h Outdated
@@ -393,7 +393,7 @@ bool TestBlockValidity(BlockValidationState& state,
bool fCheckMerkleRoot = true) EXCLUSIVE_LOCKS_REQUIRED(cs_main);

/** Check with the proof of work on each blockheader matches the value in nBits */
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Could fix typo as suggested by LLM (#32740 (comment)).

Suggested change
/** Check with the proof of work on each blockheader matches the value in nBits */
/** Check that the proof of work on each blockheader matches the value in nBits */

sipa and others added 8 commits August 15, 2025 18:38
Just don't call this function when it won't have any effect.
Avoid the need to construct a CBlockIndex object just to compute work for a header,
when its nBits value suffices for that.

Co-Authored-By: Pieter Wuille <pieter@wuille.net>
Co-Authored-By: Pieter Wuille <pieter@wuille.net>
There is no need for a function to convert a CBlock to a CBlockHeader, as it's a child
class of it.
No need to pass consensusParams, as CheckHeadersPow already has access
to m_chainparams.GetConsensus()

Co-Authored-By: maflcko <6399679+maflcko@users.noreply.github.com>
Co-Authored-By: Aurèle Oulès <aurele@oules.com>
Also mark GetFullHeader as const

Co-Authored-By: Aurèle Oulès <aurele@oules.com>
chain_start can never be null, so it's better to pass it as a reference
rather than a raw pointer

Co-Authored-By: maflcko <6399679+maflcko@users.noreply.github.com>
Copy link
Contributor Author

@danielabrozzoni danielabrozzoni left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the reviews!

In my last push:

  • Rebased onto the right commit 😅 (see #32740 (comment))
  • Addressed review comments

@@ -4149,8 +4149,8 @@ std::vector<unsigned char> ChainstateManager::GenerateCoinbaseCommitment(CBlock&

bool HasValidProofOfWork(const std::vector<CBlockHeader>& headers, const Consensus::Params& consensusParams)
{
return std::all_of(headers.cbegin(), headers.cend(),
[&](const auto& header) { return CheckProofOfWork(header.GetHash(), header.nBits, consensusParams);});
return std::ranges::all_of(headers,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it fits slightly better with the span change... since in c++20 span doesn't have a cbegin method, we needed to change the initial code, and update to ranges::all_of

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ooops, not sure what happened there, thanks for noticing!

m_current_chain_work(chain_start->nChainWork),
m_last_header_received(m_chain_start->GetBlockHeader()),
m_current_height(chain_start->nHeight)
const CBlockIndex& chain_start, const arith_uint256& minimum_required_work) : m_commit_offset(FastRandomContext().randrange<unsigned>(HEADER_COMMITMENT_PERIOD)),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I didn't like the initial formatting either, but I thought it was common (looking at other constructors in the repository, no, it's not...)

I updated the formatting both in HeadersSyncState and CompressedHeaders!

Copy link
Contributor

@hodlinator hodlinator left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 50e61f4

PR improves the code in several minor ways, such as:

  • Computing PoW for a CBlockHeader (or CBlock) without having to jump through temporarily constructed CBlockIndex (552b9c3).
  • Removing CBlock::GetBlockHeader() leaves less room for forgetting to properly set fields in the new CBlockHeader instance (67aa62c).
  • Changes const CBlockIndex* HeadersSyncState::m_chain_start to a reference as it is never null and never changed (50e61f4).

m_current_chain_work(chain_start->nChainWork),
m_last_header_received(m_chain_start->GetBlockHeader()),
m_current_height(chain_start->nHeight)
const CBlockIndex& chain_start, const arith_uint256& minimum_required_work) : m_commit_offset(FastRandomContext().randrange<unsigned>(HEADER_COMMITMENT_PERIOD)),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Still have a slight preference for using the original way of formatting the HeadersSyncState initializer-list for optimal git blame integrity, but this is tolerable compared to the previous push, I see it matches PeerManagerImpl).

@DrahtBot DrahtBot requested a review from l0rinc August 18, 2025 15:41
@l0rinc
Copy link
Contributor

l0rinc commented Aug 21, 2025

recrACK 50e61f4

I only left nits, I'm fine with merging it as is.

if (IsAncestorOfBestHeaderOrTip(last_received_header)) {
already_validated_work = true;
}
already_validated_work = already_validated_work || IsAncestorOfBestHeaderOrTip(last_received_header);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: if you touch again, you could explain in the commit message why the LookupBlockIndex cannot be eliminated as well

@@ -31,16 +31,17 @@ struct CompressedHeader {
hashMerkleRoot.SetNull();
}

CompressedHeader(const CBlockHeader& header)
explicit CompressedHeader(const CBlockHeader& header)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: if touching again, the commit message should probably mention the explicit as well. No that important, just noticed it.

@@ -41,7 +44,7 @@ HeadersSyncState::HeadersSyncState(NodeId id, const Consensus::Params& consensus
// exceeds this bound, because it's not possible for a consensus-valid
// chain to be longer than this (at the current time -- in the future we
// could try again, if necessary, to sync a longer chain).
m_max_commitments = 6*(Ticks<std::chrono::seconds>(NodeClock::now() - NodeSeconds{std::chrono::seconds{chain_start->GetMedianTimePast()}}) + MAX_FUTURE_BLOCK_TIME) / HEADER_COMMITMENT_PERIOD;
m_max_commitments = 6 * (Ticks<std::chrono::seconds>(NodeClock::now() - NodeSeconds{std::chrono::seconds{chain_start.GetMedianTimePast()}}) + MAX_FUTURE_BLOCK_TIME) / HEADER_COMMITMENT_PERIOD;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: as mentioned this should be synchronized with the other PR to simplify the merge conflicts

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm happy to put this in draft and wait for #32579 to be merged first... This PR is smaller, and probably easier to rebase. What do you think?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I personally would let the maintainers decide the order, ank keep both open

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I don't mind the order much, especially now that it seems like neither will be in v30. Thanks for asking!

Copy link
Contributor

@l0rinc l0rinc Aug 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

another super-nit: CheckHeadersPow -> CheckHeadersPoW in commit message. Only change if you modify the PR for any other reason.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants