Skip to content

Conversation

rjnrohit
Copy link

@rjnrohit rjnrohit commented Sep 8, 2021

Bitcoin nodes update their TxConfirmStats attributes (eg. confAvg, failAvg etc..) whenever they see a new block. However, during IBD, we know our best block height, so we can skip the parameter updates up to a certain height, when we know they are not going to affect fee estimates.

We can see the max value of decay for parameters is .99931. If we do (max_decay)^6048 which is approximate 0.01, it means, that transactions in blocks older than this height, do not affect the current fee estimates significantly. (i.e ~ Best_Block_Height - 6048).

By skipping these updates we can reduce the computation complexity of fee calculation during IBD significantly: (block_tree_size - 6048)/block_tree_size*100 which is ~ 99%.

I got to know about the functions to optimize by analyzing the profiling data of bitcoind. It can be generated via ./configure CC=clang CXX=clang++ CXXFLAGS="-fprofile-instr-generate=bitcoind-%p.profraw

@maflcko
Copy link
Member

maflcko commented Sep 8, 2021

By skipping these updates we can reduce the computation complexity of fee calculation during IBD significantly: (block_tree_size - 6048)/block_tree_size*100 which is ~ 99%.

Can you explain this a bit better? During IBD the mempool should be empty, so the entries vector is empty as well.

@rjnrohit
Copy link
Author

rjnrohit commented Sep 8, 2021

By skipping these updates we can reduce the computation complexity of fee calculation during IBD significantly: (block_tree_size - 6048)/block_tree_size*100 which is ~ 99%.

Can you explain this a bit better? During IBD the mempool should be empty, so the entries vector is empty as well.

Regardless of size of entries whenever a bitcoin node receives a new block then CBlockPolicyEstimator::processBlock function will be called. And so the UpdateMovingAverages function. The idea is not to call UpdateMovingAverages every time.

@maflcko
Copy link
Member

maflcko commented Sep 8, 2021

See also:

@DrahtBot
Copy link
Contributor

DrahtBot commented Sep 8, 2021

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #22508 (fuzz: replace every fuzzer-controlled while loop with a macro by apoelstra)
  • #10443 (Add fee_est tool for debugging fee estimation code by ryanofsky)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@rjnrohit rjnrohit force-pushed the lazy_update branch 2 times, most recently from cb4cf06 to edd7d17 Compare September 8, 2021 19:19
Bitcoin nodes update their TxConfirmStats attributes (eg. confAvg, failAvg etc..) whenever they see a new block.
However, during IBD, we know our best block height, so we can skip the parameter updates up to a certain height,
when we know they are not going to affect fee estimates.

We can see the max value of decay for parameters is .99931. If we do (max_decay)^6048 which is approximate 0.01,
it means, that transactions in blocks older than this height,
do not affect the current fee estimates significantly. (i.e ~ Best_Block_Height - 6048).

By skipping these updates we can reduce the computation complexity
of fee calculation during IBD significantly:
(block_tree_size - 6048)/block_tree_size*100 which is ~ 99%.
@DrahtBot
Copy link
Contributor

🐙 This pull request conflicts with the target branch and needs rebase.

Want to unsubscribe from rebase notifications on this pull request? Just convert this pull request to a "draft".

@DrahtBot
Copy link
Contributor

DrahtBot commented Jan 3, 2022

There hasn't been much activity lately and the patch still needs rebase. What is the status here?
  • Is it still relevant? ➡️ Please solve the conflicts to make it ready for review and to ensure the CI passes.
  • Is it no longer relevant? ➡️ Please close.
  • Did the author lose interest or time to work on this? ➡️ Please close it and mark it 'Up for grabs' with the label, so that it can be picked up in the future.

@fanquake
Copy link
Member

fanquake commented May 6, 2022

Closing for now.

@fanquake fanquake closed this May 6, 2022
@bitcoin bitcoin locked and limited conversation to collaborators May 6, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants