validation: fix m_best_header tracking and BLOCK_FAILED_CHILD assignment #30666

mzumsande · 2024-08-16T17:36:29Z

m_best_header (the most-work header not known to be on an invalid chain) can be wrong in the context of invalidation / reconsideration of blocks. This can happen naturally (a valid header is received and stored in our block tree db; when the full block arrives, it is found to be invalid) or triggered by the user with the invalidateblock / reconsiderblock rpc.

We don't currently use m_best_header for any critical things (see OP of #16974 for a list that still seems up-to-date), so it being wrong affects mostly rpcs.

This PR proposes to recalculate it if necessary by looping over the block index and finding the best header. It also suggest to mark headers between an invalidatetd block and the previous m_best_header as invalid, so they won't be considered in the recalculation.
It adds tests to rpc_invalidateblock.py and rpc_getchaintips.py that fail on master.

One alternative to this suggested in the past would be to introduce a continuous tracking of header tips (#12138).
While this might be more performant, it is also more complicated, and situations where we need this data are only be remotely triggerable by paying the cost of creating a valid PoW header for an invalid block.
Therefore I think it isn't necessary to optimise for performance here, plus the solution in this PR doesn't perform any extra steps in the normal node operation where no invalidated blocks are encountered.

Fixes #26245

DrahtBot · 2024-08-16T17:36:32Z

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/30666.

Reviews

See the guideline for information on the review process.

Type	Reviewers
ACK	fjahr, TheCharlatan, achow101
Approach ACK	jonatack

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#29700 (kernel, refactor: return error status on all fatal errors by ryanofsky)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

fjahr

tACK c9bf06a

I reviewed the code and confirmed manually that all the added tests do test the behavior changes in their commits.

I would also say this is a bugfix, so the title could be changed to "Fix m_best_header tracking" and get a bugfix label.

src/validation.cpp

test/functional/rpc_getchaintips.py

TheCharlatan

ACK c9bf06a

test/functional/rpc_invalidateblock.py

TheCharlatan · 2024-08-23T08:22:57Z

src/validation.cpp

+        if (!(entry.second.nStatus & BLOCK_FAILED_MASK) && m_best_header->nChainWork < entry.second.nChainWork) {
+            m_best_header = &entry.second;
+        }
+    }


I've been looking the logic over again and think it might be a good idea to call NotifyHeaderTip here too. It basically tracks along m_best_header, but with the current code will only notify when processing the next block or header.

Makes sense. I've added a call to NotifyHeaderTip.

actually, that didn't work because NotifyHeaderTip can't be called when cs_main is held. Reverting this, I'll think about this a bit more.

Ah right, I don't think this a blocker for this PR anyway, just something that came to my mind.

I thought about it a bit and don't see a great non-invasive solution to this: We could add a call to NotifyHeaderTip in the calling functions after cs_main is released, which would be in ActivateBestChain() (plus in the invalidateblock / reconsiderblock rpcs which seem less important), and I'm unsure if we want to refactor this function to always call NotifyBestHeader() just for the special case where a connected block turns out to be invalid.
In any case, since m_best_header is already being reset in InvalidChainFound() in master (just not always correctly) this would not be a regression introduced by this PR, so I think it's ok not to address it here for now.

mzumsande · 2024-08-24T10:45:26Z

src/validation.cpp

@@ -2060,6 +2060,11 @@ void Chainstate::InvalidChainFound(CBlockIndex* pindexNew)
        m_chainman.m_best_invalid = pindexNew;
    }
    if (m_chainman.m_best_header != nullptr && m_chainman.m_best_header->GetAncestor(pindexNew->nHeight) == pindexNew) {
+        CBlockIndex* index_walk{m_chainman.m_best_header};
+        while (index_walk != pindexNew) {
+            index_walk->nStatus |= BLOCK_FAILED_CHILD;


I think that this is not a complete fix. This would only mark indexes between m_best_header and the failed block as BLOCK_FAILED_CHILD, but if this was not a linear chain and there were forked blocks, those wouldn't be marked as BLOCK_FAILED_CHILD. I think I'll change the logic to also set BLOCK_FAILED_CHILD while iterating over the entire block index.

I've reworked this by introducing SetBlockFailureFlags (as a counterpart to ResetBlockFailureFlags that removes the flags after reconsiderblock)

mzumsande · 2024-08-24T11:02:00Z

Thanks for the reviews so far! Ill address feedback soon, plus I want to rework 7d42b0e, see comment above. I'm currently on holidays, so it might take a few days until I make the changes.

One more issue I've noticed while writing the tests this PR is that if we connect an invalid block received via p2p, InvalidChainFound() is called twice:
The first time in ConnectTip(), via InvalidBlockFound() here
The second time in ActivateBestChainStep() after ConnectTip() fails due to an invalid block (here).
Does anyone know if there is a reason for this? It seems to me that the call in ActivateBestChainStep() is unnecessary and could be dropped.

luke-jr · 2024-08-26T23:48:32Z

It also suggest to mark headers between an invalidatetd block and the previous m_best_header as invalid, so they won't be considered in the recalculation.

You mean if you invalidateblock, then reconsiderblock, the rest of the child blocks also need to be reconsidered individually? That seems like a conceptual bug IMO, backward incompatible, and quite annoying.

mzumsande · 2024-08-27T07:05:20Z

You mean if you invalidateblock, then reconsiderblock, the rest of the child blocks also need to be reconsidered individually?

That's already happening, reconsiderblock already goes over the entire block index db on master, looking at each index individually, see Chainstate::ResetBlockFailureFlags (this also has a bug, #30479, but that is unrelated to the failure flags).

mzumsande · 2024-09-16T17:33:14Z

c9bf06a to 321097a:

rebased
addressed feedback
reworked commit f44a403 (fix m_best_header tracking and BLOCK_FAILED_CHILD ) to set BLOCK_FAILED_CHILD for all blocks building on the invalid block, instead of just the ancestors of m_best_header

DrahtBot · 2024-09-16T17:38:45Z

🚧 At least one of the CI tasks failed.
_{Debug: https://github.com/bitcoin/bitcoin/runs/30214184316}

Hints

Make sure to run all tests locally, according to the documentation.

The failure may happen due to a number of reasons, for example:

Possibly due to a silent merge conflict (the changes in this pull request being
incompatible with the current code in the target branch). If so, make sure to rebase on the latest
commit of the target branch.
A sanitizer issue, which can only be found by compiling with the sanitizer and running the
affected test.
An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

jonatack

Approach ACK 321097a on first look

src/validation.h

fjahr · 2024-09-17T13:35:37Z

actually, that didn't work because NotifyHeaderTip can't be called when cs_main is held. Reverting this, I'll think about this a bit more.

Is this ready for review or should we wait until you have addressed this @mzumsande ?

It recalculates m_best_header by looping over the entire block index. Even though this is not very performant, it will only be used in rare situations that cannot be triggered by others without a cost: As part of to invalidateblock / reconsiderblock rpcs, or when a block with an accepted header with valid PoW turns out to be invalid later during full validation.

Co-authored-by: Fabian Jahr <fjahr@protonmail.com>

This means that it is being called in two situations: 1.) As part of the invalidateblock rpc 2.) When we receive a block for which we have a valid header in our block index, but the block turns out to be invalid

…CHILD Without doing so, header-only chains building on a chain that will be marked as invalid would still be eligible for m_best_header. This improves both getblockchaininfo and getchaintips behavior. While this adds an iteration over the entire block index, it can only be triggered by the user (invalidateblock) or by others at a cost (the header needs to be accepted in the first place, so it needs valid PoW). Co-authored-by: TheCharlatan <seb.kung@gmail.com>

Remove whitespace that doesn't conform with pep8 and turn some comments into log messages.

This test would fail to mark the chain tip as "invalid" instead of "headers-only" without the previous commit marking the headers as BLOCK_FAILED_CHILD.

fjahr · 2024-09-17T21:29:37Z

reACK 0bd53d9

TheCharlatan

Re-ACK 0bd53d9

achow101 · 2024-11-14T21:44:50Z

ACK 0bd53d9

This adds checks that 1) descendants of failed blocks are also marked as failed 2) m_best_header points to the most-work, non-invalid block in our block index. The added checks should be satisfied in normal node mode after bitcoin#30666. However, they can be broken in the case of a invalidateblockindex rpc call, which will release cs_main before calling InvalidChainFound(). TBD: How to deal with that - we probably want InvalidateBlock() to stay performant, and we do want to release cs_main - maybe we can apply some more cashing? For now, use this commit only internally for fuzzing.

…7c2ccc920 997c2ccc920 Add hooks for script debug callbacks 8598bc9e5d3 kernel: Add pure kernel bitcoin-chainstate 07530efd10c kernel: Add functions to get the block hash from a block 1a871d96afd kernel: Add block index utility functions to C header a50a97dec8e kernel: Add function to read block undo data from disk to C header 093489acee8 kernel: Add functions to read block from disk to C header 8a4b4fdefab kernel: Add function for copying block data to C header b09983496ae kernel: Add functions for the block validation state to C header c445389d749 kernel: Add validation interface to C header 94f65e6fc8c kernel: Add interrupt function to C header a7eac722561 kernel: Add import blocks function to C header c6a7574247f kernel: Add chainstate load options for in-memory dbs in C header 6f7bb78869c kernel: Add options for reindexing in C header 46f5909e6d8 kernel: Add block validation to C header 1680886ba03 Kernel: Add chainstate loading to kernel C header 4aad77d2665 kernel: Add chainstate manager option for setting worker threads c92849c7b51 kernel: Add chainstate manager object to C header e420bbddf36 kernel: Add notifications context option to C header d3c6127e0d3 kerenl: Add chain params context option to C header 4bc8c494693 kernel: Add kernel library context object df66c7b0ff2 kernel: Add logging to kernel library C header a78aeb9ff0e kernel: Introduce initial kernel C header API 6cd95de2e02 Merge bitcoin/bitcoin#31395: build: Set shared linker flags in toolchain file abeebccc480 Merge bitcoin/bitcoin#31357: cmake: Improve build script correctness 4c9b13841c4 Merge bitcoin/bitcoin#31402: doc: correct libfuzzer-nosan preset flag da4f4fac8df Merge bitcoin/bitcoin#31361: cmake, qt: Use absolute paths for includes in MOC-generated files 16b140f225a doc: correct libfuzzer-nosan preset flag 097c66f6148 Merge bitcoin/bitcoin#30039: dbwrapper: Bump LevelDB max file size to 32 MiB to avoid system slowdown from high disk cache flush rate 68daaea0e48 Merge bitcoin/bitcoin#31390: Remove `src/config` directory 14f162dc5c8 Merge bitcoin/bitcoin#31399: ci, macos: Install `pkgconf` Homebrew's package e2f2698395c ci, macos: Install `pkgconf` Homebrew's package b73d3319377 dbwrapper: Bump max file size to 32 MiB a8e04704f93 build: Set shared linker flags in toolchain file dbc8ba12f3b Merge bitcoin/bitcoin#31371: doc, test: more ephemeral dust follow-ups 935973b315f Remove `src/config` directory 7590e93bc73 Merge bitcoin/bitcoin#30986: contrib: skip missing binaries in gen-manpages b2af068825c Merge bitcoin/bitcoin#30708: rpc: add getdescriptoractivity 144f98db85e Merge bitcoin/bitcoin#31337: build: Fix coverage builds efdb49afb9e Merge bitcoin/bitcoin#31323: guix: swap `moreutils` for just `sponge` 37a5c5d8366 doc: update descriptors.md for getdescriptoractivity ee3ce6a4f4d test: rpc: add no address case for getdescriptoractivity 811f76f3a51 rpc: add getdescriptoractivity ee6185372fc gen-manpages: Prompt error if no binaries are found 70e20ea024c Merge bitcoin/bitcoin#31172: build: increase minimum supported Windows to 10.0 733317ba943 Merge bitcoin/bitcoin#31364: refactor: Fix remaining clang-tidy performance-unnecessary-copy-initialization errors 5a4bc5c0366 Merge bitcoin/bitcoin#31305: refactor: Fix remaining clang-tidy performance-inefficient-vector errors 28fd0bc7316 Merge bitcoin/bitcoin#31365: interpreter: Use the same type for SignatureHash in the definition 72ab35a6d09 Merge bitcoin/bitcoin#31221: ci: Split out native fuzz jobs for macOS and windows (take 2) 160799d9135 test: refactor: introduce `create_ephemeral_dust_package` helper 61e18dec306 doc: ephemeral policy: add missing closing double quote 3305972f7bf refactor: Fix remaining clang-tidy performance-unnecessary-copy-initialization errors 11f3bc229cc refactor: Reserve vectors in fuzz tests 152fefe7a22 refactor: Preallocate PrevectorFillVector(In)Direct without vector resize a774c7a339c refactor: Fix remaining clang-tidy performance-inefficient-vector errors f7144b24be0 Merge bitcoin/bitcoin#31279: policy: ephemeral dust followups c288c790cd9 interpreter: Use the same type for SignatureHash in the definition b031b7910d6 [ci] Split out native fuzz jobs for macOS and windows 6f4128e3a83 cmake, qt: Use absolute paths for includes in MOC-generated files ab5c63edcce cmake: Build `secp256k1` only when required 76a3a540a4c cmake: Ensure script correctness when no targets are specified e8f50c5debe guix: swap moreutils for just sponge 01a7298818d build: Avoid using the `-ffile-prefix-map` compiler option 2638fdb4f93 Merge bitcoin/bitcoin#31338: test: Deduplicate assert_mempool_contents() 17834bd1976 Merge bitcoin/bitcoin#31333: fuzz: Implement G_TEST_GET_FULL_NAME cf577227888 Merge bitcoin/bitcoin#31335: macOS: swap docs & CI from pkg-config to pkgconf fe3457ccfff ci: note that we should install pkgconf in future a0eafc10f94 functional test: Deduplicate assert_mempool_contents() 8d203480b33 doc: migrate from pkg-config to pkgconf in macOS build docs 466e4df3fb8 assert_mempool_contents: assert not duplicates expected ea5db2f2692 functional: only generate required blocks for test d033acb6083 fuzz: package_eval: let fuzzer run out input in main tx creation loop ba35a570c5d CheckEphemeralSpends: return boolean, and set child state and txid outparams cf0cee1617c func: add note about lack of 1P1C propagation in tree submitpackage 84242903043 unit test: ephemeral_tests is using a dust relay rate, not minrelay d9cfa5fc4eb CheckEphemeralSpends: no need to iterate inputs if no parent dust 87b26e3dc07 func: rename test_free_relay to test_no_minrelay_fee e5709a4a41e func: slight elaboration on submitpackage restriction 08e969bd107 RPC: only enforce dust rules on priority when standardness active ca050d12e76 unit test: adapt to changing MAX_DUST_OUTPUTS_PER_TX 7c3490169c9 fuzz: package_eval: move last_tx inside txn ctor 445eaed182a fuzz: use optional status instead of should_rbf_eph_spend 4dfdf615b9d fuzz: remove unused TransactionsDelta validation interface 09ce926e4a1 func: cleanup reorg test comment 768a0c1889e func: cleanup test_dustrelay comments bedca1cb663 fuzz: Directly place transactions in vector c041ad6eccb fuzz: explain package eval coin tracking better bc0d98ea612 fuzz: remove dangling reference to GetEntry 15b6cbf07f5 unit test: make dust index less magical 5fbcfd12b8f unit test: assert txid returned on CheckEphemeralSpends failures ef94d84b4e4 bench: remove unnecessary CMTxn constructors c5c10fd317c ephemeral policy doxygen cleanup dd9044b8d46 ephemeral policy: IWYU c6859ce2de7 Move+rename GetDustIndexes -> GetDust 22ef95dbe3e Merge bitcoin/bitcoin#31288: Add destroy to BlockTemplate schema 92d3d691f09 fuzz: Implement G_TEST_GET_FULL_NAME f34fe0806a0 Merge bitcoin/bitcoin#31122: cluster mempool: Implement changeset interface for mempool b2d952c0f5b Merge bitcoin/bitcoin#31331: doc: add copyright header to p2p_headers_presync 7d3703dec3d doc: add copyright header to p2p_headers_presync 116b8c55736 Merge bitcoin/bitcoin#31213: fuzz: Fix difficulty target generation in `p2p_headers_presync` 15c1f47a005 Merge bitcoin/bitcoin#31327: doc: Correct PR Review Club frequency from weekly to monthly 1209a1082c8 Merge bitcoin/bitcoin#31315: build: Enable -Wbidi-chars=any ab22726def9 Merge bitcoin/bitcoin#31276: guix: scope pkg-config to Linux only 637f437a164 doc: remove PR Review Club frequency e1223099584 Merge bitcoin/bitcoin#31317: test: Revert to random path element 2666d83da59 Merge bitcoin/bitcoin#30893: test: Introduce ensure_for helper faaaf59f71e test: Make g_rng_temp_path rand, not dependent on SeedRandomForTest 746f93b4f0f Merge bitcoin/bitcoin#31307: build: Temporarily disable compiling `fuzz/utxo_snapshot.cpp` with MSVC 25fe087de59 rpc: move-only: move ScriptPubKeyDoc to utils fa80b08fef0 test: Revert to random path element fa7857ccda5 build: Enable -Wbidi-chars=any b2d53610028 build: Temporarily disable compiling `fuzz/utxo_snapshot.cpp` with MSVC 9aa50152c1c Add destroy to BlockTemplate schema ccc2d3abcd3 Merge bitcoin/bitcoin#31287: refactor: Avoid std::string format strings 62016b32300 Use std::ranges for ephemeral policy checks 3ed930a1f41 Have HasDust and PreCheckValidEphemeralTx take CTransaction 04a614bf9a7 Rename CheckValidEphemeralTx to PreCheckEphemeralTx 85bcfeea235 Merge bitcoin/bitcoin#30666: validation: fix m_best_header tracking and BLOCK_FAILED_CHILD assignment 2257c6d68fa Merge bitcoin/bitcoin#30487: ci: skip Github CI on branch pushes for forks 380e1f44e8e Merge bitcoin/bitcoin#30349: benchmark: Improve SipHash_32b accuracy to avoid potential optimization issues 1a8f51e7453 Merge bitcoin/bitcoin#28843: [refactor] Cleanup BlockAssembler mempool usage 2d944e982c4 Merge bitcoin/bitcoin#31285: guix: remove `util-linux` bcd82b13f46 Remove pkgconfig from toolchain file 319a4e82614 depends: drop sqlite pkgconfig file fa1177e3d7c refactor: Avoid std::string format strings a8fe1fd38bf depends: better cleanup after fontconfig 17e79c92607 depends: fully remove libtool archives from Qt build 8ca85651c83 guix: move pkg-config to Linux builds e3e648cf410 depends: drop pkg-config option from Qt build 0d185bd99f9 doc: update depends doc to prefer .cmake outputs e546b4e1a0c Merge bitcoin/bitcoin#31225: doc: Fix grammatical errors in multisig-tutorial.md f44e39c9d0d Merge bitcoin/bitcoin#31174: tinyformat: Add compile-time checking for literal format strings 299e2220e95 gen-manpages: implement --skip-missing-binaries 5736d1ddacc tracing: pass if replaced by tx/pkg to tracepoint a4ec07f1944 doc: add comments for CTxMemPool::ChangeSet 83f814b1d11 Remove m_all_conflicts from SubPackageState d3c8e7dfb63 Ensure that we don't add duplicate transactions in rbf fuzz tests d7dc9fd2f7b Move CalculateChunksForRBF() to the mempool changeset 284a1d33f1d Move prioritisation into changeset 446b08b599b Don't distinguish between direct conflicts and all conflicts when doing cluster-size-2-rbf checks b53041021ab Duplicate transactions are not permitted within a changeset b447416fddc Public mempool removal methods Assume() no changeset is outstanding 2b30f4d36c8 Make RemoveStaged() private 18829194ca6 Enforce that there is only one changeset at a time 7fb62f7db60 Apply mempool changeset transactions directly into the mempool 34b6c5833d1 Clean up FinalizeSubpackage to avoid workspace-specific information 57983b8add7 Move LimitMempoolSize to take place outside FinalizeSubpackage 01e145b9758 Move changeset from workspace to subpackage 802214c0832 Introduce mempool changesets 87d92fa3401 test: Add unit test coverage of package rbf + prioritisetransaction 15d982f91e6 Add package hash to package-rbf log message 4d668549825 ci: remove util-linux from centos CI cdf34be7c94 guix: remove util-linux cbf1a47d606 CheckEphemeralSpends: only compute txid of tx when needed 111465d72dd test: Remove unused attempts parameter from wait_until 5468a23eb9a test: Add check_interval parameter to wait_until 16c87d91fd4 test: Introduce ensure_for helper a6ca8f32439 fuzz: Fix difficulty target generation in p2p_headers_presync 8610bcef9d0 ci: skip Github CI on branch pushes for forks ee1128ead84 doc: update stack-clash-protection comment re mingw-w64 bf47448f152 test: drop check for Windows < 10 35b898c47f8 release: target Windows 10 or later 398754e70bc depends: target Windows 10 when building for mingw-w64 ac286e0d1bd doc: Fix grammatical errors in multisig-tutorial.md fa327c77e34 util: Add ConsumeArithUInt256InRange fuzzing helper 42066f45ff5 Refactor SipHash_32b benchmark to improve accuracy and avoid optimization issues fe39acf88ff tinyformat: Add compile-time checking for literal format strings 184f34f2d0f util: Support dynamic width & precision in ConstevalFormatString 192dac1d337 [refactor] Cleanup BlockAssembler mempool usage 0bd53d913c1 test: add test for getchaintips behavior with invalid chains ccd98ea4c88 test: cleanup rpc_getchaintips.py f5149ddb9b7 validation: mark blocks building on an invalid block as BLOCK_FAILED_CHILD 783cb7337f7 validation: call RecalculateBestHeader in InvalidChainFound 9275e9689a4 rpc: call RecalculateBestHeader as part of reconsiderblock a51e91783aa validation: add RecalculateBestHeader() function REVERT: 403c20980ec kernel: Add pure kernel bitcoin-chainstate REVERT: 2e1b262eff6 kernel: Add functions to get the block hash from a block REVERT: f0b56a6052a kernel: Add block index utility functions to C header REVERT: 9393cf98179 kernel: Add function to read block undo data from disk to C header REVERT: e2ccfba0e3e kernel: Add functions to read block from disk to C header REVERT: 0559575861c kernel: Add function for copying block data to C header REVERT: 7b02e52237e kernel: Add functions for the block validation state to C header REVERT: fdebacca7ad kernel: Add validation interface to C header REVERT: f4ea5f49c6d kernel: Add interrupt function to C header REVERT: 5f4b436aad9 kernel: Add import blocks function to C header REVERT: c95a28fd80c kernel: Add chainstate load options for in-memory dbs in C header REVERT: d6360557ef6 kernel: Add options for reindexing in C header REVERT: a125867b9fe kernel: Add block validation to C header REVERT: b2b75a0ef73 Kernel: Add chainstate loading to kernel C header REVERT: d233003ff2a kernel: Add chainstate manager option for setting worker threads REVERT: 3610be3b138 kernel: Add chainstate manager object to C header REVERT: c194bea41f6 kernel: Add notifications context option to C header REVERT: 691d89d846b kerenl: Add chain params context option to C header REVERT: 407ca750cda kernel: Add kernel library context object REVERT: ee3c4ea92cd kernel: Add logging to kernel library C header REVERT: e6c610a7e03 kernel: Introduce initial kernel C header API git-subtree-dir: libbitcoinkernel-sys/bitcoin git-subtree-split: 997c2ccc920adb0d1c9d8e82dbe94374cfb3c78b

…90df267df 6090df267df kernel: Add pure kernel bitcoin-chainstate 29e3b874303 kernel: Add functions to get the block hash from a block 97d83063cb6 kernel: Add block index utility functions to C header 76ead0878a3 kernel: Add function to read block undo data from disk to C header 94c215c4212 kernel: Add functions to read block from disk to C header aff05fcbe62 kernel: Add function for copying block data to C header a379fbe15eb kernel: Add functions for the block validation state to C header 81db1665213 kernel: Add validation interface to C header 0ee7a5f58a6 kernel: Add interrupt function to C header f68b3dbf919 kernel: Add import blocks function to C header 503aae9afc5 kernel: Add chainstate load options for in-memory dbs in C header fa1335fd809 kernel: Add options for reindexing in C header 0eb020f9c82 kernel: Add block validation to C header 5f471729674 Kernel: Add chainstate loading to kernel C header 90949cbb09d kernel: Add chainstate manager option for setting worker threads a2021b517c6 kernel: Add chainstate manager object to C header 428ea46909b kernel: Add notifications context option to C header b1f5d450323 kerenl: Add chain params context option to C header 7bb4b412db0 kernel: Add kernel library context object e4b63d7f5d9 kernel: Add logging to kernel library C header 47bb56b243e kernel: Introduce initial kernel C header API ff873a20a7f Merge bitcoin/bitcoin#31313: refactor: Clamp worker threads in ChainstateManager constructor c9a7418a8df Merge bitcoin/bitcoin#31096: Package validation: accept packages of size 1 6f24662eb96 Merge bitcoin/bitcoin#31175: rpc: Remove submitblock pre-checks 3867d2421ae Merge bitcoin/bitcoin#31112: Improve parallel script validation error debug logging 8e02b480591 Merge bitcoin/bitcoin#31284: ci: Skip broken Wine64 tests by default 492e1f09943 [validation] merge all ConnectBlock debug logging code paths b49df703f03 [validation] include all logged information in BlockValidationState 7b267c034fd [validation] Add detailed txin/txout information for script error messages 146a3d54268 [validation] Make script error messages uniform for parallel/single validation 1ac1c33f3f1 [checkqueue] support user-defined return type through std::optional ebe4cac38bf Merge bitcoin/bitcoin#30991: test: enable running independent functional test sub-tests 19276741007 Merge bitcoin/bitcoin#31387: doc: Use more precise anchor link to codesigning docs e043618d44d Merge bitcoin/bitcoin#31396: test: simple reordering to reduce run time a25b892ab1f Merge bitcoin/bitcoin#31386: doc: Use more precise anchor links to Xcode SDK extraction eb646111cdc Merge bitcoin/bitcoin#31383: test: Add missing node.setmocktime(self.mocktime) to p2p_ibd_stalling.py 6cd95de2e02 Merge bitcoin/bitcoin#31395: build: Set shared linker flags in toolchain file abeebccc480 Merge bitcoin/bitcoin#31357: cmake: Improve build script correctness 4c9b13841c4 Merge bitcoin/bitcoin#31402: doc: correct libfuzzer-nosan preset flag da4f4fac8df Merge bitcoin/bitcoin#31361: cmake, qt: Use absolute paths for includes in MOC-generated files 16b140f225a doc: correct libfuzzer-nosan preset flag 097c66f6148 Merge bitcoin/bitcoin#30039: dbwrapper: Bump LevelDB max file size to 32 MiB to avoid system slowdown from high disk cache flush rate 68daaea0e48 Merge bitcoin/bitcoin#31390: Remove `src/config` directory 14f162dc5c8 Merge bitcoin/bitcoin#31399: ci, macos: Install `pkgconf` Homebrew's package e2f2698395c ci, macos: Install `pkgconf` Homebrew's package b73d3319377 dbwrapper: Bump max file size to 32 MiB 62f6d9e1a48 test: simple ordering optimization to reduce runtime a8e04704f93 build: Set shared linker flags in toolchain file dbc8ba12f3b Merge bitcoin/bitcoin#31371: doc, test: more ephemeral dust follow-ups 935973b315f Remove `src/config` directory 19f49c7489d doc: Use more precise anchor link to codesigning docs 8bf1b3039cb doc: Use more precise anchor links to Xcode SDK extraction 7590e93bc73 Merge bitcoin/bitcoin#30986: contrib: skip missing binaries in gen-manpages b2af068825c Merge bitcoin/bitcoin#30708: rpc: add getdescriptoractivity 144f98db85e Merge bitcoin/bitcoin#31337: build: Fix coverage builds faa16ed4b9e test: Add missing node.setmocktime(self.mocktime) to p2p_ibd_stalling.py efdb49afb9e Merge bitcoin/bitcoin#31323: guix: swap `moreutils` for just `sponge` 37a5c5d8366 doc: update descriptors.md for getdescriptoractivity ee3ce6a4f4d test: rpc: add no address case for getdescriptoractivity 811f76f3a51 rpc: add getdescriptoractivity ee6185372fc gen-manpages: Prompt error if no binaries are found 70e20ea024c Merge bitcoin/bitcoin#31172: build: increase minimum supported Windows to 10.0 733317ba943 Merge bitcoin/bitcoin#31364: refactor: Fix remaining clang-tidy performance-unnecessary-copy-initialization errors 5a4bc5c0366 Merge bitcoin/bitcoin#31305: refactor: Fix remaining clang-tidy performance-inefficient-vector errors 28fd0bc7316 Merge bitcoin/bitcoin#31365: interpreter: Use the same type for SignatureHash in the definition 72ab35a6d09 Merge bitcoin/bitcoin#31221: ci: Split out native fuzz jobs for macOS and windows (take 2) 160799d9135 test: refactor: introduce `create_ephemeral_dust_package` helper 61e18dec306 doc: ephemeral policy: add missing closing double quote 32fc59796f7 rpc: Allow single transaction through submitpackage 3305972f7bf refactor: Fix remaining clang-tidy performance-unnecessary-copy-initialization errors 11f3bc229cc refactor: Reserve vectors in fuzz tests 152fefe7a22 refactor: Preallocate PrevectorFillVector(In)Direct without vector resize a774c7a339c refactor: Fix remaining clang-tidy performance-inefficient-vector errors f7144b24be0 Merge bitcoin/bitcoin#31279: policy: ephemeral dust followups c288c790cd9 interpreter: Use the same type for SignatureHash in the definition b031b7910d6 [ci] Split out native fuzz jobs for macOS and windows 6f4128e3a83 cmake, qt: Use absolute paths for includes in MOC-generated files ab5c63edcce cmake: Build `secp256k1` only when required 76a3a540a4c cmake: Ensure script correctness when no targets are specified e8f50c5debe guix: swap moreutils for just sponge 01a7298818d build: Avoid using the `-ffile-prefix-map` compiler option 2638fdb4f93 Merge bitcoin/bitcoin#31338: test: Deduplicate assert_mempool_contents() 73db95c65c1 kernel: Make bitcoin-chainstate's block validation mirror submitblock's bb53ce9bdae tests: Add functional test for submitting a previously pruned block 1f7fc738255 rpc: Remove submitblock duplicate pre-check e62a8abd7df rpc: Remove submitblock invalid-duplicate precheck 36dbebafb9b rpc: Remove submitblock coinbase pre-check 17834bd1976 Merge bitcoin/bitcoin#31333: fuzz: Implement G_TEST_GET_FULL_NAME cf577227888 Merge bitcoin/bitcoin#31335: macOS: swap docs & CI from pkg-config to pkgconf fe3457ccfff ci: note that we should install pkgconf in future a0eafc10f94 functional test: Deduplicate assert_mempool_contents() 8d203480b33 doc: migrate from pkg-config to pkgconf in macOS build docs 466e4df3fb8 assert_mempool_contents: assert not duplicates expected ea5db2f2692 functional: only generate required blocks for test d033acb6083 fuzz: package_eval: let fuzzer run out input in main tx creation loop ba35a570c5d CheckEphemeralSpends: return boolean, and set child state and txid outparams cf0cee1617c func: add note about lack of 1P1C propagation in tree submitpackage 84242903043 unit test: ephemeral_tests is using a dust relay rate, not minrelay d9cfa5fc4eb CheckEphemeralSpends: no need to iterate inputs if no parent dust 87b26e3dc07 func: rename test_free_relay to test_no_minrelay_fee e5709a4a41e func: slight elaboration on submitpackage restriction 08e969bd107 RPC: only enforce dust rules on priority when standardness active ca050d12e76 unit test: adapt to changing MAX_DUST_OUTPUTS_PER_TX 7c3490169c9 fuzz: package_eval: move last_tx inside txn ctor 445eaed182a fuzz: use optional status instead of should_rbf_eph_spend 4dfdf615b9d fuzz: remove unused TransactionsDelta validation interface 09ce926e4a1 func: cleanup reorg test comment 768a0c1889e func: cleanup test_dustrelay comments bedca1cb663 fuzz: Directly place transactions in vector c041ad6eccb fuzz: explain package eval coin tracking better bc0d98ea612 fuzz: remove dangling reference to GetEntry 15b6cbf07f5 unit test: make dust index less magical 5fbcfd12b8f unit test: assert txid returned on CheckEphemeralSpends failures ef94d84b4e4 bench: remove unnecessary CMTxn constructors c5c10fd317c ephemeral policy doxygen cleanup dd9044b8d46 ephemeral policy: IWYU c6859ce2de7 Move+rename GetDustIndexes -> GetDust 22ef95dbe3e Merge bitcoin/bitcoin#31288: Add destroy to BlockTemplate schema 92d3d691f09 fuzz: Implement G_TEST_GET_FULL_NAME f34fe0806a0 Merge bitcoin/bitcoin#31122: cluster mempool: Implement changeset interface for mempool b2d952c0f5b Merge bitcoin/bitcoin#31331: doc: add copyright header to p2p_headers_presync 7d3703dec3d doc: add copyright header to p2p_headers_presync 116b8c55736 Merge bitcoin/bitcoin#31213: fuzz: Fix difficulty target generation in `p2p_headers_presync` 15c1f47a005 Merge bitcoin/bitcoin#31327: doc: Correct PR Review Club frequency from weekly to monthly 1209a1082c8 Merge bitcoin/bitcoin#31315: build: Enable -Wbidi-chars=any ab22726def9 Merge bitcoin/bitcoin#31276: guix: scope pkg-config to Linux only 637f437a164 doc: remove PR Review Club frequency e1223099584 Merge bitcoin/bitcoin#31317: test: Revert to random path element 2666d83da59 Merge bitcoin/bitcoin#30893: test: Introduce ensure_for helper faaaf59f71e test: Make g_rng_temp_path rand, not dependent on SeedRandomForTest 746f93b4f0f Merge bitcoin/bitcoin#31307: build: Temporarily disable compiling `fuzz/utxo_snapshot.cpp` with MSVC 25fe087de59 rpc: move-only: move ScriptPubKeyDoc to utils fa80b08fef0 test: Revert to random path element 8f85d36d68a refactor: Clamp worker threads in ChainstateManager constructor fa7857ccda5 build: Enable -Wbidi-chars=any b2d53610028 build: Temporarily disable compiling `fuzz/utxo_snapshot.cpp` with MSVC 9aa50152c1c Add destroy to BlockTemplate schema ccc2d3abcd3 Merge bitcoin/bitcoin#31287: refactor: Avoid std::string format strings 62016b32300 Use std::ranges for ephemeral policy checks 3ed930a1f41 Have HasDust and PreCheckValidEphemeralTx take CTransaction 04a614bf9a7 Rename CheckValidEphemeralTx to PreCheckEphemeralTx 85bcfeea235 Merge bitcoin/bitcoin#30666: validation: fix m_best_header tracking and BLOCK_FAILED_CHILD assignment 2257c6d68fa Merge bitcoin/bitcoin#30487: ci: skip Github CI on branch pushes for forks 380e1f44e8e Merge bitcoin/bitcoin#30349: benchmark: Improve SipHash_32b accuracy to avoid potential optimization issues 1a8f51e7453 Merge bitcoin/bitcoin#28843: [refactor] Cleanup BlockAssembler mempool usage 2d944e982c4 Merge bitcoin/bitcoin#31285: guix: remove `util-linux` bcd82b13f46 Remove pkgconfig from toolchain file 319a4e82614 depends: drop sqlite pkgconfig file fa1177e3d7c refactor: Avoid std::string format strings a8fe1fd38bf depends: better cleanup after fontconfig 17e79c92607 depends: fully remove libtool archives from Qt build 8ca85651c83 guix: move pkg-config to Linux builds e3e648cf410 depends: drop pkg-config option from Qt build 0d185bd99f9 doc: update depends doc to prefer .cmake outputs e546b4e1a0c Merge bitcoin/bitcoin#31225: doc: Fix grammatical errors in multisig-tutorial.md f44e39c9d0d Merge bitcoin/bitcoin#31174: tinyformat: Add compile-time checking for literal format strings 299e2220e95 gen-manpages: implement --skip-missing-binaries 5736d1ddacc tracing: pass if replaced by tx/pkg to tracepoint a4ec07f1944 doc: add comments for CTxMemPool::ChangeSet 83f814b1d11 Remove m_all_conflicts from SubPackageState d3c8e7dfb63 Ensure that we don't add duplicate transactions in rbf fuzz tests d7dc9fd2f7b Move CalculateChunksForRBF() to the mempool changeset 284a1d33f1d Move prioritisation into changeset 446b08b599b Don't distinguish between direct conflicts and all conflicts when doing cluster-size-2-rbf checks b53041021ab Duplicate transactions are not permitted within a changeset b447416fddc Public mempool removal methods Assume() no changeset is outstanding 2b30f4d36c8 Make RemoveStaged() private 18829194ca6 Enforce that there is only one changeset at a time 7fb62f7db60 Apply mempool changeset transactions directly into the mempool 34b6c5833d1 Clean up FinalizeSubpackage to avoid workspace-specific information 57983b8add7 Move LimitMempoolSize to take place outside FinalizeSubpackage 01e145b9758 Move changeset from workspace to subpackage 802214c0832 Introduce mempool changesets 87d92fa3401 test: Add unit test coverage of package rbf + prioritisetransaction 15d982f91e6 Add package hash to package-rbf log message fa5e7064597 ci: Skip broken Wine64 tests by default 4d668549825 ci: remove util-linux from centos CI cdf34be7c94 guix: remove util-linux cbf1a47d606 CheckEphemeralSpends: only compute txid of tx when needed 111465d72dd test: Remove unused attempts parameter from wait_until 5468a23eb9a test: Add check_interval parameter to wait_until 16c87d91fd4 test: Introduce ensure_for helper a6ca8f32439 fuzz: Fix difficulty target generation in p2p_headers_presync 8610bcef9d0 ci: skip Github CI on branch pushes for forks 409d0d62937 test: enable running individual independent functional test methods ee1128ead84 doc: update stack-clash-protection comment re mingw-w64 bf47448f152 test: drop check for Windows < 10 35b898c47f8 release: target Windows 10 or later 398754e70bc depends: target Windows 10 when building for mingw-w64 ac286e0d1bd doc: Fix grammatical errors in multisig-tutorial.md fa327c77e34 util: Add ConsumeArithUInt256InRange fuzzing helper 42066f45ff5 Refactor SipHash_32b benchmark to improve accuracy and avoid optimization issues fe39acf88ff tinyformat: Add compile-time checking for literal format strings 184f34f2d0f util: Support dynamic width & precision in ConstevalFormatString 192dac1d337 [refactor] Cleanup BlockAssembler mempool usage 0bd53d913c1 test: add test for getchaintips behavior with invalid chains ccd98ea4c88 test: cleanup rpc_getchaintips.py f5149ddb9b7 validation: mark blocks building on an invalid block as BLOCK_FAILED_CHILD 783cb7337f7 validation: call RecalculateBestHeader in InvalidChainFound 9275e9689a4 rpc: call RecalculateBestHeader as part of reconsiderblock a51e91783aa validation: add RecalculateBestHeader() function REVERT: 403c20980ec kernel: Add pure kernel bitcoin-chainstate REVERT: 2e1b262eff6 kernel: Add functions to get the block hash from a block REVERT: f0b56a6052a kernel: Add block index utility functions to C header REVERT: 9393cf98179 kernel: Add function to read block undo data from disk to C header REVERT: e2ccfba0e3e kernel: Add functions to read block from disk to C header REVERT: 0559575861c kernel: Add function for copying block data to C header REVERT: 7b02e52237e kernel: Add functions for the block validation state to C header REVERT: fdebacca7ad kernel: Add validation interface to C header REVERT: f4ea5f49c6d kernel: Add interrupt function to C header REVERT: 5f4b436aad9 kernel: Add import blocks function to C header REVERT: c95a28fd80c kernel: Add chainstate load options for in-memory dbs in C header REVERT: d6360557ef6 kernel: Add options for reindexing in C header REVERT: a125867b9fe kernel: Add block validation to C header REVERT: b2b75a0ef73 Kernel: Add chainstate loading to kernel C header REVERT: d233003ff2a kernel: Add chainstate manager option for setting worker threads REVERT: 3610be3b138 kernel: Add chainstate manager object to C header REVERT: c194bea41f6 kernel: Add notifications context option to C header REVERT: 691d89d846b kerenl: Add chain params context option to C header REVERT: 407ca750cda kernel: Add kernel library context object REVERT: ee3c4ea92cd kernel: Add logging to kernel library C header REVERT: e6c610a7e03 kernel: Introduce initial kernel C header API git-subtree-dir: libbitcoinkernel-sys/bitcoin git-subtree-split: 6090df267dfece6192b567fed6582445aa811e7f

20eec64b5e kernel: Add pure kernel bitcoin-chainstate 1522ee9596 kernel: Add functions to get the block hash from a block c543cadc90 kernel: Add block index utility functions to C header 6b5a4fed43 kernel: Add function to read block undo data from disk to C header 6b53c3ba1a kernel: Add functions to read block from disk to C header 7f9908ad99 kernel: Add function for copying block data to C header 1ab71ac371 kernel: Add functions for the block validation state to C header 508dd4db98 kernel: Add validation interface to C header a6a658e5f8 kernel: Add interrupt function to C header 253c3cd36f kernel: Add import blocks function to C header 3ed633b6c4 kernel: Add chainstate load options for in-memory dbs in C header 08ec37f8ce kernel: Add options for reindexing in C header ad0875e397 kernel: Add block validation to C header 2575018d69 Kernel: Add chainstate loading to kernel C header f5d21c94dc kernel: Add chainstate manager option for setting worker threads 783f56f0a2 kernel: Add chainstate manager object to C header 262039e409 kernel: Add notifications context option to C header dc0d406dd5 kerenl: Add chain params context option to C header b5f84de7ad kernel: Add kernel library context object dad0009c86 kernel: Add logging to kernel library C header 27e25aa941 kernel: Introduce initial kernel C header API ff873a20a7 Merge bitcoin/bitcoin#31313: refactor: Clamp worker threads in ChainstateManager constructor c9a7418a8d Merge bitcoin/bitcoin#31096: Package validation: accept packages of size 1 6f24662eb9 Merge bitcoin/bitcoin#31175: rpc: Remove submitblock pre-checks 3867d2421a Merge bitcoin/bitcoin#31112: Improve parallel script validation error debug logging 8e02b48059 Merge bitcoin/bitcoin#31284: ci: Skip broken Wine64 tests by default 492e1f0994 [validation] merge all ConnectBlock debug logging code paths b49df703f0 [validation] include all logged information in BlockValidationState 7b267c034f [validation] Add detailed txin/txout information for script error messages 146a3d5426 [validation] Make script error messages uniform for parallel/single validation 1ac1c33f3f [checkqueue] support user-defined return type through std::optional ebe4cac38b Merge bitcoin/bitcoin#30991: test: enable running independent functional test sub-tests 1927674100 Merge bitcoin/bitcoin#31387: doc: Use more precise anchor link to codesigning docs e043618d44 Merge bitcoin/bitcoin#31396: test: simple reordering to reduce run time a25b892ab1 Merge bitcoin/bitcoin#31386: doc: Use more precise anchor links to Xcode SDK extraction eb646111cd Merge bitcoin/bitcoin#31383: test: Add missing node.setmocktime(self.mocktime) to p2p_ibd_stalling.py 6cd95de2e0 Merge bitcoin/bitcoin#31395: build: Set shared linker flags in toolchain file abeebccc48 Merge bitcoin/bitcoin#31357: cmake: Improve build script correctness 4c9b13841c Merge bitcoin/bitcoin#31402: doc: correct libfuzzer-nosan preset flag da4f4fac8d Merge bitcoin/bitcoin#31361: cmake, qt: Use absolute paths for includes in MOC-generated files 16b140f225 doc: correct libfuzzer-nosan preset flag 097c66f614 Merge bitcoin/bitcoin#30039: dbwrapper: Bump LevelDB max file size to 32 MiB to avoid system slowdown from high disk cache flush rate 68daaea0e4 Merge bitcoin/bitcoin#31390: Remove `src/config` directory 14f162dc5c Merge bitcoin/bitcoin#31399: ci, macos: Install `pkgconf` Homebrew's package e2f2698395 ci, macos: Install `pkgconf` Homebrew's package b73d331937 dbwrapper: Bump max file size to 32 MiB 62f6d9e1a4 test: simple ordering optimization to reduce runtime a8e04704f9 build: Set shared linker flags in toolchain file dbc8ba12f3 Merge bitcoin/bitcoin#31371: doc, test: more ephemeral dust follow-ups 935973b315 Remove `src/config` directory 19f49c7489 doc: Use more precise anchor link to codesigning docs 8bf1b3039c doc: Use more precise anchor links to Xcode SDK extraction 7590e93bc7 Merge bitcoin/bitcoin#30986: contrib: skip missing binaries in gen-manpages b2af068825 Merge bitcoin/bitcoin#30708: rpc: add getdescriptoractivity 144f98db85 Merge bitcoin/bitcoin#31337: build: Fix coverage builds faa16ed4b9 test: Add missing node.setmocktime(self.mocktime) to p2p_ibd_stalling.py efdb49afb9 Merge bitcoin/bitcoin#31323: guix: swap `moreutils` for just `sponge` 37a5c5d836 doc: update descriptors.md for getdescriptoractivity ee3ce6a4f4 test: rpc: add no address case for getdescriptoractivity 811f76f3a5 rpc: add getdescriptoractivity ee6185372f gen-manpages: Prompt error if no binaries are found 70e20ea024 Merge bitcoin/bitcoin#31172: build: increase minimum supported Windows to 10.0 733317ba94 Merge bitcoin/bitcoin#31364: refactor: Fix remaining clang-tidy performance-unnecessary-copy-initialization errors 5a4bc5c036 Merge bitcoin/bitcoin#31305: refactor: Fix remaining clang-tidy performance-inefficient-vector errors 28fd0bc731 Merge bitcoin/bitcoin#31365: interpreter: Use the same type for SignatureHash in the definition 72ab35a6d0 Merge bitcoin/bitcoin#31221: ci: Split out native fuzz jobs for macOS and windows (take 2) 160799d913 test: refactor: introduce `create_ephemeral_dust_package` helper 61e18dec30 doc: ephemeral policy: add missing closing double quote 32fc59796f rpc: Allow single transaction through submitpackage 3305972f7b refactor: Fix remaining clang-tidy performance-unnecessary-copy-initialization errors 11f3bc229c refactor: Reserve vectors in fuzz tests 152fefe7a2 refactor: Preallocate PrevectorFillVector(In)Direct without vector resize a774c7a339 refactor: Fix remaining clang-tidy performance-inefficient-vector errors f7144b24be Merge bitcoin/bitcoin#31279: policy: ephemeral dust followups c288c790cd interpreter: Use the same type for SignatureHash in the definition b031b7910d [ci] Split out native fuzz jobs for macOS and windows 6f4128e3a8 cmake, qt: Use absolute paths for includes in MOC-generated files ab5c63edcc cmake: Build `secp256k1` only when required 76a3a540a4 cmake: Ensure script correctness when no targets are specified e8f50c5deb guix: swap moreutils for just sponge 01a7298818 build: Avoid using the `-ffile-prefix-map` compiler option 2638fdb4f9 Merge bitcoin/bitcoin#31338: test: Deduplicate assert_mempool_contents() 73db95c65c kernel: Make bitcoin-chainstate's block validation mirror submitblock's bb53ce9bda tests: Add functional test for submitting a previously pruned block 1f7fc73825 rpc: Remove submitblock duplicate pre-check e62a8abd7d rpc: Remove submitblock invalid-duplicate precheck 36dbebafb9 rpc: Remove submitblock coinbase pre-check 17834bd197 Merge bitcoin/bitcoin#31333: fuzz: Implement G_TEST_GET_FULL_NAME cf57722788 Merge bitcoin/bitcoin#31335: macOS: swap docs & CI from pkg-config to pkgconf fe3457ccff ci: note that we should install pkgconf in future a0eafc10f9 functional test: Deduplicate assert_mempool_contents() 8d203480b3 doc: migrate from pkg-config to pkgconf in macOS build docs 466e4df3fb assert_mempool_contents: assert not duplicates expected ea5db2f269 functional: only generate required blocks for test d033acb608 fuzz: package_eval: let fuzzer run out input in main tx creation loop ba35a570c5 CheckEphemeralSpends: return boolean, and set child state and txid outparams cf0cee1617 func: add note about lack of 1P1C propagation in tree submitpackage 8424290304 unit test: ephemeral_tests is using a dust relay rate, not minrelay d9cfa5fc4e CheckEphemeralSpends: no need to iterate inputs if no parent dust 87b26e3dc0 func: rename test_free_relay to test_no_minrelay_fee e5709a4a41 func: slight elaboration on submitpackage restriction 08e969bd10 RPC: only enforce dust rules on priority when standardness active ca050d12e7 unit test: adapt to changing MAX_DUST_OUTPUTS_PER_TX 7c3490169c fuzz: package_eval: move last_tx inside txn ctor 445eaed182 fuzz: use optional status instead of should_rbf_eph_spend 4dfdf615b9 fuzz: remove unused TransactionsDelta validation interface 09ce926e4a func: cleanup reorg test comment 768a0c1889 func: cleanup test_dustrelay comments bedca1cb66 fuzz: Directly place transactions in vector c041ad6ecc fuzz: explain package eval coin tracking better bc0d98ea61 fuzz: remove dangling reference to GetEntry 15b6cbf07f unit test: make dust index less magical 5fbcfd12b8 unit test: assert txid returned on CheckEphemeralSpends failures ef94d84b4e bench: remove unnecessary CMTxn constructors c5c10fd317 ephemeral policy doxygen cleanup dd9044b8d4 ephemeral policy: IWYU c6859ce2de Move+rename GetDustIndexes -> GetDust 22ef95dbe3 Merge bitcoin/bitcoin#31288: Add destroy to BlockTemplate schema 92d3d691f0 fuzz: Implement G_TEST_GET_FULL_NAME f34fe0806a Merge bitcoin/bitcoin#31122: cluster mempool: Implement changeset interface for mempool b2d952c0f5 Merge bitcoin/bitcoin#31331: doc: add copyright header to p2p_headers_presync 7d3703dec3 doc: add copyright header to p2p_headers_presync 116b8c5573 Merge bitcoin/bitcoin#31213: fuzz: Fix difficulty target generation in `p2p_headers_presync` 15c1f47a00 Merge bitcoin/bitcoin#31327: doc: Correct PR Review Club frequency from weekly to monthly 1209a1082c Merge bitcoin/bitcoin#31315: build: Enable -Wbidi-chars=any ab22726def Merge bitcoin/bitcoin#31276: guix: scope pkg-config to Linux only 637f437a16 doc: remove PR Review Club frequency e122309958 Merge bitcoin/bitcoin#31317: test: Revert to random path element 2666d83da5 Merge bitcoin/bitcoin#30893: test: Introduce ensure_for helper faaaf59f71 test: Make g_rng_temp_path rand, not dependent on SeedRandomForTest 746f93b4f0 Merge bitcoin/bitcoin#31307: build: Temporarily disable compiling `fuzz/utxo_snapshot.cpp` with MSVC 25fe087de5 rpc: move-only: move ScriptPubKeyDoc to utils fa80b08fef test: Revert to random path element 8f85d36d68 refactor: Clamp worker threads in ChainstateManager constructor fa7857ccda build: Enable -Wbidi-chars=any b2d5361002 build: Temporarily disable compiling `fuzz/utxo_snapshot.cpp` with MSVC 9aa50152c1 Add destroy to BlockTemplate schema ccc2d3abcd Merge bitcoin/bitcoin#31287: refactor: Avoid std::string format strings 62016b3230 Use std::ranges for ephemeral policy checks 3ed930a1f4 Have HasDust and PreCheckValidEphemeralTx take CTransaction 04a614bf9a Rename CheckValidEphemeralTx to PreCheckEphemeralTx 85bcfeea23 Merge bitcoin/bitcoin#30666: validation: fix m_best_header tracking and BLOCK_FAILED_CHILD assignment 2257c6d68f Merge bitcoin/bitcoin#30487: ci: skip Github CI on branch pushes for forks 380e1f44e8 Merge bitcoin/bitcoin#30349: benchmark: Improve SipHash_32b accuracy to avoid potential optimization issues 1a8f51e745 Merge bitcoin/bitcoin#28843: [refactor] Cleanup BlockAssembler mempool usage 2d944e982c Merge bitcoin/bitcoin#31285: guix: remove `util-linux` bcd82b13f4 Remove pkgconfig from toolchain file 319a4e8261 depends: drop sqlite pkgconfig file fa1177e3d7 refactor: Avoid std::string format strings a8fe1fd38b depends: better cleanup after fontconfig 17e79c9260 depends: fully remove libtool archives from Qt build 8ca85651c8 guix: move pkg-config to Linux builds e3e648cf41 depends: drop pkg-config option from Qt build 0d185bd99f doc: update depends doc to prefer .cmake outputs e546b4e1a0 Merge bitcoin/bitcoin#31225: doc: Fix grammatical errors in multisig-tutorial.md f44e39c9d0 Merge bitcoin/bitcoin#31174: tinyformat: Add compile-time checking for literal format strings 299e2220e9 gen-manpages: implement --skip-missing-binaries 5736d1ddac tracing: pass if replaced by tx/pkg to tracepoint a4ec07f194 doc: add comments for CTxMemPool::ChangeSet 83f814b1d1 Remove m_all_conflicts from SubPackageState d3c8e7dfb6 Ensure that we don't add duplicate transactions in rbf fuzz tests d7dc9fd2f7 Move CalculateChunksForRBF() to the mempool changeset 284a1d33f1 Move prioritisation into changeset 446b08b599 Don't distinguish between direct conflicts and all conflicts when doing cluster-size-2-rbf checks b53041021a Duplicate transactions are not permitted within a changeset b447416fdd Public mempool removal methods Assume() no changeset is outstanding 2b30f4d36c Make RemoveStaged() private 18829194ca Enforce that there is only one changeset at a time 7fb62f7db6 Apply mempool changeset transactions directly into the mempool 34b6c5833d Clean up FinalizeSubpackage to avoid workspace-specific information 57983b8add Move LimitMempoolSize to take place outside FinalizeSubpackage 01e145b975 Move changeset from workspace to subpackage 802214c083 Introduce mempool changesets 87d92fa340 test: Add unit test coverage of package rbf + prioritisetransaction 15d982f91e Add package hash to package-rbf log message fa5e706459 ci: Skip broken Wine64 tests by default 4d66854982 ci: remove util-linux from centos CI cdf34be7c9 guix: remove util-linux cbf1a47d60 CheckEphemeralSpends: only compute txid of tx when needed 111465d72d test: Remove unused attempts parameter from wait_until 5468a23eb9 test: Add check_interval parameter to wait_until 16c87d91fd test: Introduce ensure_for helper a6ca8f3243 fuzz: Fix difficulty target generation in p2p_headers_presync 8610bcef9d ci: skip Github CI on branch pushes for forks 409d0d6293 test: enable running individual independent functional test methods ee1128ead8 doc: update stack-clash-protection comment re mingw-w64 bf47448f15 test: drop check for Windows < 10 35b898c47f release: target Windows 10 or later 398754e70b depends: target Windows 10 when building for mingw-w64 ac286e0d1b doc: Fix grammatical errors in multisig-tutorial.md fa327c77e3 util: Add ConsumeArithUInt256InRange fuzzing helper 42066f45ff Refactor SipHash_32b benchmark to improve accuracy and avoid optimization issues fe39acf88f tinyformat: Add compile-time checking for literal format strings 184f34f2d0 util: Support dynamic width & precision in ConstevalFormatString 192dac1d33 [refactor] Cleanup BlockAssembler mempool usage 0bd53d913c test: add test for getchaintips behavior with invalid chains ccd98ea4c8 test: cleanup rpc_getchaintips.py f5149ddb9b validation: mark blocks building on an invalid block as BLOCK_FAILED_CHILD 783cb7337f validation: call RecalculateBestHeader in InvalidChainFound 9275e9689a rpc: call RecalculateBestHeader as part of reconsiderblock a51e91783a validation: add RecalculateBestHeader() function REVERT: 35f8503285 kernel: Add pure kernel bitcoin-chainstate REVERT: 84eb1f952c kernel: Add functions to get the block hash from a block REVERT: 575cb5a033 kernel: Add block index utility functions to C header REVERT: 4c433defd3 kernel: Add function to read block undo data from disk to C header REVERT: 83e48e021b kernel: Add functions to read block from disk to C header REVERT: a4381c560f kernel: Add function for copying block data to C header REVERT: d3e84ac5a6 kernel: Add functions for the block validation state to C header REVERT: deb5b4a5f5 kernel: Add validation interface to C header REVERT: f4ea5f49c6 kernel: Add interrupt function to C header REVERT: 5f4b436aad kernel: Add import blocks function to C header REVERT: c95a28fd80 kernel: Add chainstate load options for in-memory dbs in C header REVERT: d6360557ef kernel: Add options for reindexing in C header REVERT: a125867b9f kernel: Add block validation to C header REVERT: b2b75a0ef7 Kernel: Add chainstate loading to kernel C header REVERT: d233003ff2 kernel: Add chainstate manager option for setting worker threads REVERT: 3610be3b13 kernel: Add chainstate manager object to C header REVERT: c194bea41f kernel: Add notifications context option to C header REVERT: 691d89d846 kerenl: Add chain params context option to C header REVERT: 407ca750cd kernel: Add kernel library context object REVERT: ee3c4ea92c kernel: Add logging to kernel library C header REVERT: e6c610a7e0 kernel: Introduce initial kernel C header API git-subtree-dir: depend/bitcoin git-subtree-split: 20eec64b5e417cac8c68100826c0adf2152a49eb

Summary: This is a series of diffs that fixes a minor bug related to m_best_header sometimes being incorrect. PR description: > m_best_header (the most-work header not known to be on an invalid chain) can be wrong in the context of invalidation / reconsideration of blocks. This can happen naturally (a valid header is received and stored in our block tree db; when the full block arrives, it is found to be invalid) or triggered by the user with the invalidateblock / reconsiderblock rpc. > > We don't currently use m_best_header for any critical things, so it being wrong affects mostly rpcs. > > One alternative to this suggested in the past would be to introduce a continuous tracking of header tips (#12138). > While this might be more performant, it is also more complicated, and situations where we need this data are only be remotely triggerable by paying the cost of creating a valid PoW header for an invalid block. > I think it isn't necessary to optimise for performance here, plus the solution in this PR doesn't perform any extra steps in the normal node operation where no invalidated blocks are encountered. bitcoin/bitcoin@a51e917: > validation: add RecalculateBestHeader() function > > It recalculates m_best_header by looping over the entire > block index. Even though this is not very performant, it > will only be used in rare situations that cannot be > triggered by others without a cost: > As part of to invalidateblock / reconsiderblock rpcs, or when a > block with an accepted header with valid PoW turns out to be invalid > later during full validation. bitcoin/bitcoin@9275e96: > rpc: call RecalculateBestHeader as part of reconsiderblock Co-authored-by: Fabian Jahr <fjahr@protonmail.com> This is a partial backport of [[bitcoin/bitcoin#30666 | core#30666]] Depends on D17998 Test Plan: `ninja all check-all` Reviewers: #bitcoin_abc, Fabien Reviewed By: #bitcoin_abc, Fabien Differential Revision: https://reviews.bitcoinabc.org/D17999

Summary: This means that it is being called in two situations: 1.) As part of the invalidateblock rpc 2.) When we receive a block for which we have a valid header in our block index, but the block turns out to be invalid This is a partial backport of [[bitcoin/bitcoin#30666 | core#30666]] bitcoin/bitcoin@783cb73 Depends on D17999 Test Plan: `ninja all check-all` Reviewers: #bitcoin_abc, Fabien Reviewed By: #bitcoin_abc, Fabien Differential Revision: https://reviews.bitcoinabc.org/D18000

… parents Summary: bitcoin/bitcoin@f5149dd: > Without doing so, header-only chains building on a chain that > will be marked as invalid would still be eligible for m_best_header. > This improves both getblockchaininfo and getchaintips behavior. > > While this adds an iteration over the entire block index, it can only be > triggered by the user (invalidateblock) or by others at a cost (the > header needs to be accepted in the first place, so it needs valid PoW). Co-authored-by: TheCharlatan <seb.kung@gmail.com> bitcoin/bitcoin@ccd98ea bitcoin/bitcoin@0bd53d9 > test: add test for getchaintips behavior with invalid chains > > This test would fail to mark the chain tip as "invalid" instead > of "headers-only" without the previous commit marking the headers > as BLOCK_FAILED_CHILD. Now the unintuitive behavior revealed and documented in D1987 in abc_feature_parkedchain.py becomes the expected behavior. This concludes backport of [[bitcoin/bitcoin#30666 | core#30666]] Depends on D18042 Test Plan: `ninja all check-all` Reviewers: #bitcoin_abc, Fabien Reviewed By: #bitcoin_abc, Fabien Differential Revision: https://reviews.bitcoinabc.org/D18002

f6b782f doc: Improve m_best_header documentation (Martin Zumsande) ee673b9 validation: remove m_failed_blocks (Martin Zumsande) ed764ea validation: Add more checks to CheckBlockIndex() (Martin Zumsande) 9a70883 validation: in invalidateblock, calculate m_best_header right away (Martin Zumsande) 8e39f2d validation: in invalidateblock, mark children as invalid right away (Martin Zumsande) 4c29326 validation: cache all headers with enough PoW in invalidateblock (Martin Zumsande) 15fa5b5 validation: call InvalidBlockFound also from AcceptBlock (Martin Zumsande) Pull request description: Some fields in validation are set opportunistically by "best effort": - The `BLOCK_FAILED_CHILD` status (which means that the block index has an invalid predecessor) - `m_best_header` (the most-work header not known to be invalid). This means that there are known situations in which these fields are not set when they should be, or set to wrong values. This is tolerated because the fields are not used for anything consensus-critical and triggering these situations involved creating invalid blocks with valid PoW header, so would have a cost attached. Also, having stricter guarantees for these fields requires iterating over the entire block index, which has some DoS potential, especially with any header above the checkpoint being accepted int he past (see e.g. #11531). However, there are reasons to change this now: - RPCs use these fields and can report wrong results - There is the constant possibility that someone could add code that expects these fields to be correct, especially because it is not well documented that these fields cannot always be relied upon. - DoS concerns have become less of an issue after #25717 - now an attacker would need to invest much more work because they can't fork off the last checkpoint anymore This PR continues the work from #30666 to ensure that `BLOCK_FAILED_CHILD` status and `m_best_header` are always correct: - it adds a call to `InvalidChainFound()` in `AcceptBlock()`. - it adds checks for `BLOCK_FAILED_CHILD` and `m_best_header` to `CheckBlockIndex()`. In order to be able to do this, the existing cache in the RPC-only `InvalidateBlock()` is adjusted to handle these as well. These are performance optimizations with the goal of avoiding having a call of `InvalidChainFound()` / looping over the block index after each disconnected block. I also wrote a fuzz test to find possible edge cases violating `CheckBlockIndex`, which I will PR separately soon. - it removes the `m_failed_blocks` set, which was a heuristic necessary when we couldn't be sure if a given block index had an invalid predecessor or not. Now that we have that guarantee, the set is no longer needed. ACKs for top commit: stickies-v: re-ACK f6b782f achow101: reACK f6b782f ryanofsky: Code review ACK f6b782f with only minor code & comment updates TheCharlatan: Re-ACK f6b782f Tree-SHA512: 1bee324216eeee6af401abdb683abd098b18212833f9600dbc0a46244e634cb0e6f2a320c937a5675a12af7ec4a7d10fabc1db9e9bc0d9d0712e6e6ca72d084f

…utxo.py The mentioned bug has been fixed in PR bitcoin#30666, so the workaround is not needed anymore.

…feature_assumeutxo.py 206bc05 test: remove unnecessary m_best_header setting hack in feature_assumeutxo.py (Sebastian Falbesoner) Pull request description: The mentioned bug (issue #26245) has been fixed in PR #30666, so the workaround is not needed anymore. ACKs for top commit: fjahr: ACK 206bc05 janb84: ACK 206bc05 mzumsande: utACK 206bc05 Tree-SHA512: 92ad6c09c80318a02e00d8c2a4ca1ec8a6caa6aa310916ff705871c91fe5c960c15ed394ead8ce9c7466b592dba176aa87a635b3228ddc93f57082124f04613f

…utxo.py The mentioned bug has been fixed in PR bitcoin#30666, so the workaround is not needed anymore.

DrahtBot added the Validation label Aug 16, 2024

This was referenced Aug 16, 2024

validation: Improve, document and test logic for chains building on invalid blocks #30207

Closed

bugfix: update chainman best_header after block reconsideration #29913

Closed

DrahtBot mentioned this pull request Aug 17, 2024

assumeutxo: Add dumptxoutset height param, remove shell scripts #29553

Merged

fjahr reviewed Aug 18, 2024

View reviewed changes

src/validation.cpp Outdated Show resolved Hide resolved

test/functional/rpc_getchaintips.py Outdated Show resolved Hide resolved

TheCharlatan approved these changes Aug 18, 2024

View reviewed changes

test/functional/rpc_invalidateblock.py Show resolved Hide resolved

TheCharlatan reviewed Aug 23, 2024

View reviewed changes

mzumsande commented Aug 24, 2024

View reviewed changes

DrahtBot added the Needs rebase label Sep 3, 2024

mzumsande force-pushed the 202404_invalidblock branch from c9bf06a to 5aa9160 Compare September 16, 2024 17:22

mzumsande changed the title ~~validation: improve m_best_header tracking~~ validation: fix m_best_header tracking and BLOCK_FAILED_CHILD assignment Sep 16, 2024

mzumsande force-pushed the 202404_invalidblock branch from 5aa9160 to 321097a Compare September 16, 2024 17:38

DrahtBot added CI failed and removed Needs rebase CI failed labels Sep 16, 2024

jonatack reviewed Sep 16, 2024

View reviewed changes

src/validation.h Outdated Show resolved Hide resolved

src/validation.h Outdated Show resolved Hide resolved

src/validation.h Outdated Show resolved Hide resolved

mzumsande and others added 6 commits September 17, 2024 11:39

rpc: call RecalculateBestHeader as part of reconsiderblock

9275e96

Co-authored-by: Fabian Jahr <fjahr@protonmail.com>

validation: call RecalculateBestHeader in InvalidChainFound

783cb73

This means that it is being called in two situations: 1.) As part of the invalidateblock rpc 2.) When we receive a block for which we have a valid header in our block index, but the block turns out to be invalid

test: cleanup rpc_getchaintips.py

ccd98ea

Remove whitespace that doesn't conform with pep8 and turn some comments into log messages.

test: add test for getchaintips behavior with invalid chains

0bd53d9

This test would fail to mark the chain tip as "invalid" instead of "headers-only" without the previous commit marking the headers as BLOCK_FAILED_CHILD.

mzumsande force-pushed the 202404_invalidblock branch from 321097a to 0bd53d9 Compare September 17, 2024 15:41

DrahtBot requested review from jonatack and TheCharlatan September 17, 2024 21:29

DrahtBot mentioned this pull request Oct 3, 2024

Add -pausebackgroundsync startup option #31023

Closed

1 task

TheCharlatan approved these changes Oct 10, 2024

View reviewed changes

DrahtBot added CI failed and removed CI failed labels Oct 18, 2024

DrahtBot mentioned this pull request Nov 5, 2024

kernel, refactor: return error status on all fatal errors #29700

Draft

achow101 merged commit 85bcfee into bitcoin:master Nov 14, 2024
16 checks passed

This comment was marked as off-topic.

Sign in to view

mzumsande deleted the 202404_invalidblock branch November 21, 2024 19:59

mzumsande mentioned this pull request Dec 2, 2024

validation: stricter internal handling of invalid blocks #31405

Merged

pinheadmz mentioned this pull request May 23, 2025

getchaintips doesn't mark headers-only chain as invalid #8050

Closed

theStack added a commit to theStack/bitcoin that referenced this pull request Jun 13, 2025

test: remove unnecessary m_best_header setting hack in feature_assume…

206bc05

…utxo.py The mentioned bug has been fixed in PR bitcoin#30666, so the workaround is not needed anymore.

theStack mentioned this pull request Jun 13, 2025

test: remove unnecessary m_best_header setting hack in feature_assumeutxo.py #32746

Merged

PeterWrighten pushed a commit to PeterWrighten/bitcoin that referenced this pull request Jul 15, 2025

test: remove unnecessary m_best_header setting hack in feature_assume…

219afa0

…utxo.py The mentioned bug has been fixed in PR bitcoin#30666, so the workaround is not needed anymore.

saikiran57 pushed a commit to saikiran57/bitcoin that referenced this pull request Jul 28, 2025

test: remove unnecessary m_best_header setting hack in feature_assume…

a7890c3

…utxo.py The mentioned bug has been fixed in PR bitcoin#30666, so the workaround is not needed anymore.

validation: fix m_best_header tracking and BLOCK_FAILED_CHILD assignment #30666

validation: fix m_best_header tracking and BLOCK_FAILED_CHILD assignment #30666

Uh oh!

Conversation

mzumsande commented Aug 16, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DrahtBot commented Aug 16, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Coverage & Benchmarks

Reviews

Conflicts

Uh oh!

fjahr left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

TheCharlatan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

TheCharlatan Aug 23, 2024

Choose a reason for hiding this comment

Uh oh!

mzumsande Sep 16, 2024

Choose a reason for hiding this comment

Uh oh!

mzumsande Sep 16, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

TheCharlatan Sep 17, 2024

Choose a reason for hiding this comment

Uh oh!

mzumsande Sep 17, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This comment was marked as off-topic.

Uh oh!

mzumsande Aug 24, 2024

Choose a reason for hiding this comment

Uh oh!

mzumsande Sep 16, 2024

Choose a reason for hiding this comment

Uh oh!

mzumsande commented Aug 24, 2024

Uh oh!

luke-jr commented Aug 26, 2024

Uh oh!

mzumsande commented Aug 27, 2024

Uh oh!

mzumsande commented Sep 16, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DrahtBot commented Sep 16, 2024

Uh oh!

jonatack left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

fjahr commented Sep 17, 2024

Uh oh!

fjahr commented Sep 17, 2024

Uh oh!

TheCharlatan left a comment

Choose a reason for hiding this comment

Uh oh!

achow101 commented Nov 14, 2024

Uh oh!

Uh oh!

This comment was marked as off-topic.

Uh oh!

mzumsande commented Aug 16, 2024 •

edited

Loading

DrahtBot commented Aug 16, 2024 •

edited

Loading

mzumsande Sep 16, 2024 •

edited

Loading

mzumsande Sep 17, 2024 •

edited

Loading

mzumsande commented Sep 16, 2024 •

edited

Loading