rpc: dumptxoutset height parameter follow-ups (29553) #30808

fjahr · 2024-09-04T00:29:16Z

First, this addresses two left-over comments in #29553:

When running dumptxoutset the network gets disabled in the beginning and then re-enabled at the end. The network would be re-enabled even if the user had already disabled the network themself before running dumptxoutset. The network is now not re-enabled anymore since that might not be what the user wants.
The -rpcclienttimeout=0 option is added to loadtxoutset examples in documentation

Additionally, pablomartin4btc notified me that he found his node stuck at the invalidated height after some late testing after #29553 was merged. We could not find the actual source of the issue since his logs got lost. However, it seems likely that some kind of disruption stopped the process before the node could roll forward again. We fixed this issue for network disablement with a RAII class previously and it seems logical that this can happen the same way for the rollback part so I suggest to also fix it the same way.

An example to reproduce the issue described above as I think it happened: Remove the ! in the following line in PrepareUTXOSnapshot() to simulate an issue occurring during GetUTXOStats().

if (!maybe_stats) {

This leaves the node in the following state on master:

$ build/src/bitcoin-cli -rpcclienttimeout=0 -named dumptxoutset utxo-859750.dat rollback=859750
error code: -32603
error message:
Unable to read UTXO set
$ build/src/bitcoin-cli getchaintips
[
  {
    "height": 859762,
    "hash": "00000000000000000002ec7a0fcca3aeca5b35545b52eb925766670aacc704ad",
    "branchlen": 12,
    "status": "headers-only"
  },
  {
    "height": 859750,
    "hash": "0000000000000000000010897b6b88a18f9478050200d8d048013c58bfd6229e",
    "branchlen": 0,
    "status": "active"
  },

(Note that the first tip is headers-only and not invalid only because I started dumptxoutset before my node had fully synced to the tip. pablomartin4btc saw it as invalid.)

DrahtBot · 2024-09-04T00:29:19Z

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage

For detailed information about the code coverage, see the test coverage report.

Reviews

See the guideline for information on the review process.

Type	Reviewers
ACK	maflcko, pablomartin4btc, achow101

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

fjahr · 2024-09-04T00:29:32Z

cc @pablomartin4btc

maflcko

review ACK e6812ff 💦

Show signature

Signature:

untrusted comment: signature from minisign secret key on empty file; verify via: minisign -Vm "${path_to_any_empty_file}" -P RWTRmVTMeKV5noAMqVlsMugDDCyyTSbA3Re5AkUrhvLVln0tSaFWglOw -x "${path_to_this_whole_four_line_signature_blob}"
RUTRmVTMeKV5npGrKx1nqXCw5zeVHdtdYURB/KlyA/LMFgpNCs+SkW9a8N95d+U4AP1RJMi+krxU1A3Yux4bpwZNLvVBKy0wLgM=
trusted comment: review ACK e6812ffec338035a1ebe949015e95ba335c94d08 💦
W+/4H97jPWfOdV0n5HR2P1oOkiKMX4xhTey8kpWlh8OcWiie4vaGV7+XKN2A/MMaEg8gluzE110nCa+zLZZ+Dg==

maflcko · 2024-09-04T11:02:14Z

src/rpc/blockchain.cpp

+    NodeContext& m_node;
+    const CBlockIndex* m_invalidate_index;
+public:
+    TemporaryRollback(NodeContext& node, const CBlockIndex* index) : m_node(node), m_invalidate_index(index) {
+        InvalidateBlock(*m_node.chainman, m_invalidate_index->GetBlockHash());
+    };
+    ~TemporaryRollback() {
+        ReconsiderBlock(*m_node.chainman, m_invalidate_index->GetBlockHash());


Suggested change

NodeContext& m_node;

const CBlockIndex* m_invalidate_index;

public:

TemporaryRollback(NodeContext& node, const CBlockIndex* index) : m_node(node), m_invalidate_index(index) {

InvalidateBlock(*m_node.chainman, m_invalidate_index->GetBlockHash());

};

~TemporaryRollback() {

ReconsiderBlock(*m_node.chainman, m_invalidate_index->GetBlockHash());

ChainstateManager& m_chainman;

const CBlockIndex& m_invalidate_index;

public:

TemporaryRollback(ChainstateManager& chainman, const CBlockIndex& index) : m_chainman{chainman}, m_invalidate_index{index} {

InvalidateBlock(m_chainman, m_invalidate_index.GetBlockHash());

};

~TemporaryRollback() {

ReconsiderBlock(m_chainman, m_invalidate_index.GetBlockHash());

nit in e6812ff: Doesn't change the behavior here, but generally it is a bit better to avoid pointers, when the value can not be nullptr. This avoids accidental undefined behavior in the future, when the nullpointer is accidentally dereferenced.

Forgot the CBlockIndex * -> & ? 😅

Ah, yeah, I overlooked that. Fixed.

src/rpc/blockchain.cpp

Also fixes a typo in the RPC help text.

maflcko

re-ACK 769dc84 🎃

Show signature

Signature:

untrusted comment: signature from minisign secret key on empty file; verify via: minisign -Vm "${path_to_any_empty_file}" -P RWTRmVTMeKV5noAMqVlsMugDDCyyTSbA3Re5AkUrhvLVln0tSaFWglOw -x "${path_to_this_whole_four_line_signature_blob}"
RUTRmVTMeKV5npGrKx1nqXCw5zeVHdtdYURB/KlyA/LMFgpNCs+SkW9a8N95d+U4AP1RJMi+krxU1A3Yux4bpwZNLvVBKy0wLgM=
trusted comment: re-ACK 769dc84852e6f14876d595a7d628cc02fa074c75 🎃
NpAXsv7bFi2YYBPoRYnRhIhn+RpXg/ohmiETp1/YL5H7ZaOZFcdFFODR6KX2Ku9cOh7cXGUwsjR2ocRElLCVCQ==

maflcko · 2024-09-04T14:00:22Z

src/rpc/blockchain.cpp

@@ -2823,22 +2841,14 @@ static RPCHelpMan dumptxoutset()
        // be activated as the new tip and we would not get to new_tip_index.
        if (target_index != chainstate->m_chain.Tip()) {
            LogInfo("Failed to roll back to requested height, reverting to tip.\n");


nit in the last commit: Shouldn't this be a LogWarning, so that the user knows that the log is related to an RPC error that happened? Also, it could include the name of the RPC that triggered the log.

Edit: To explain, -logsourcelocations doesn't help here, because the function name here would just be operator().

I checked the developer notes on the definition again and I think you are right that a LogWarning is a better fit. I think it's not a "severe problem" necessarily but it does need to be addressed by the node admin if they want to run dumptxoutset successfully.

maflcko · 2024-09-04T14:07:45Z

re-ACK a3108a7 🐸

Show signature

Signature:

untrusted comment: signature from minisign secret key on empty file; verify via: minisign -Vm "${path_to_any_empty_file}" -P RWTRmVTMeKV5noAMqVlsMugDDCyyTSbA3Re5AkUrhvLVln0tSaFWglOw -x "${path_to_this_whole_four_line_signature_blob}"
RUTRmVTMeKV5npGrKx1nqXCw5zeVHdtdYURB/KlyA/LMFgpNCs+SkW9a8N95d+U4AP1RJMi+krxU1A3Yux4bpwZNLvVBKy0wLgM=
trusted comment: re-ACK a3108a7c5692d137b70b8442b4741936277e89be 🐸
NK9JYrH/xtml5V8cLt4zlJuPtyWae9nucXxnYumLdL5Ac+7ZUcB01Qzi/JwdvIIDsDoiGyVgBO2UNqBhhlH1DQ==

pablomartin4btc

cr ACK a3108a7

Thanks for addressing the 3 follow-ups. I'll test this soon.

Is it worth it to add a test to produce the situation where the reconsider didn't happen and the height stayed at the rollback/ invalidated height/ stale tip (test should fail in prod and not with this PR)?

Here some more details (rpc: dumptxoutset, getchaintips & getcblockchaininfo) about the situation described at the top.

./src/bitcoin-cli -datadir=${AU_DATADIR} -rpcclienttimeout=0 -named dumptxoutset utxo-858000.dat rollback=858000
{
  "coins_written": 184931392,
  "base_hash": "00000000000000000001b44004f45de683f7f5dc792519b47f867baac2d4bf56",
  "base_height": 858000,
  "path": "/home/pablo/.test_utxo_840/utxo-858000.dat",
  "txoutset_hash": "d0b6dd9367746570658e9052ff814b864643d8aeccb2ccc81d438123242db63b",
  "nchaintx": 1064936785
}

./src/bitcoin-cli -datadir=${AU_DATADIR} getchaintips
[
  {
    "height": 859145,
    "hash": "0000000000000000000109e94f6e25bd92d6424d960427e3bd10e7c65cd1996d",
    "branchlen": 1145,
    "status": "invalid"
  },
  {
    "height": 858000,
    "hash": "00000000000000000001b44004f45de683f7f5dc792519b47f867baac2d4bf56",
    "branchlen": 0,
    "status": "active"
  }
]

./src/bitcoin-cli -datadir=${AU_DATADIR} getblockchaininfo
{
  "chain": "main",
  "blocks": 858000,
  "headers": 858000,
  "bestblockhash": "00000000000000000001b44004f45de683f7f5dc792519b47f867baac2d4bf56",
  "difficulty": 86871474313761.95,
  "time": 1724377947,
  "mediantime": 1724371333,
  "verificationprogress": 0.9936410953572635,
  "initialblockdownload": true,
  "chainwork": "00000000000000000000000000000000000000008a60c229d43fb91def135e8e",
  "size_on_disk": 678404041445,
  "pruned": false,
  "warnings": [
    "This is a pre-release test build - use at your own risk - do not use for mining or merchant applications"
  ]
}

Just to clarify, when this happened, testing #29553, I didn't realised the state of the node after a few other PRs reviews I made, the snapshot file was generated completely and I still have it, perhaps I shutdown the node during the reconsiderblock process...

achow101 · 2024-09-04T15:28:44Z

ACK a3108a7

ryanofsky · 2024-09-04T17:09:47Z

src/rpc/blockchain.cpp

@@ -2770,6 +2787,7 @@ static RPCHelpMan dumptxoutset()
    CConnman& connman = EnsureConnman(node);
    const CBlockIndex* invalidate_index{nullptr};
    std::unique_ptr<NetworkDisable> disable_network;
+    std::unique_ptr<TemporaryRollback> temporary_rollback;


Not important, but it would probably make sense for both disable_network and temporary_rollback variables to be std::optional instead of std::unique_ptr since they aren't passed around it would simplify the way they are initialized.

fjahr · 2024-09-04T18:16:33Z

Yo dawg, I heard you like follow-ups so I made a follow-up to my follow-up ;)

#30817 addresses the std::optional comment from @ryanofsky and adds a small test as suggested by @pablomartin4btc

pablomartin4btc · 2024-09-04T18:41:02Z

post-merge tACK a3108a7

(manipulated maybe_stats as suggested and other manual tests: rolling back, shutdown node during invalidate & reconsider)

c2b779d refactor: Manage dumptxoutset RAII classes with std::optional (Fabian Jahr) 4b5bf33 test: Add coverage for failing dumptxoutset behavior (Fabian Jahr) Pull request description: This adds a test that checks that network activity is not suspended if dumptxoutset fails in the middle of its process which is implemented with the `NetworkDisable` RAII class. I would have liked to add coverage for the `TemporaryRollback` RAII class but that seems a lot more tricky since the failure needs to happen at some point after the rollback and on the scale of our test chain here I couldn't find a way to do it yet. This was requested by pablomartin4btc here: #30808 (review). To test the test you can comment out the content of the destructor of `NetworkDisable`. It also addresses the feedback by ryanofsky to use `std::optional` instead of `std::unique_ptr` for the management of the RAII object: #30808 (comment) ACKs for top commit: achow101: ACK c2b779d pablomartin4btc: cr & tACK c2b779d tdb3: ACK c2b779d BrandonOdiwuor: Code Review ACK c2b779d theStack: ACK c2b779d Tree-SHA512: 9556e75014a2599bb870b70faf887608b332f2312626333f771d4ec11c04f863a2cf17e223ec473d4e8b0c9e8008394a4e0c321561f7ef3a2eec713dcfaea58a

…6e67b2a d8b6e67b2a kernel: Add check if the chainstate maanger is busy loading blocks f12a131af8 kernel: Process transactions 75d48e6718 kernel: Add optional mempool 889632bb70 kernel: Add support for handling transactions 476c754f3b kernel: Add check for when a block has been mutated. ebef1b465b kernel: Add support for handling block headers a56c79a2b5 kernel: Add utxo set iteration and value retrieval 33c71843e3 kernel: Add pure kernel bitcoin-chainstate ccb2bb59f9 kernel: Add block index utility functions to C header c953e1f530 kernel: Add function to read block undo data from disk to C header 55673f7f70 kernel: Add functions to read block from disk to C header 348e1c0a22 kernel: Add function for copying block data to C header 4c71ffbef6 kernel: Add functions for the block validation state to C header 160343c9c2 kernel: Add validation interface and task runner to C header 55a60d8c3f kernel: Add interrupt function to C header 0d59e95b82 kernel: Add import blocks function to C header 804e064ff6 kernel: Add chainstate load options for in-memory dbs in C header 1c829ce9f4 kernel: Add options for reindexing in C header c8a2917004 kernel: Add block validation to C header 8015b8374c Kernel: Add chainstate loading to kernel C header b3c8467b62 kernel: Add chainstate manager object to C header 07c55185c2 kernel: Add notifications context option to C header 71c45c0680 kerenl: Add chain params context option to C header d2ad67bd5f kernel: Add kernel library context object 1c6716c609 kernel: Add logging to kernel library C header 63a83b8dad kernel: Introduce initial kernel C header API REVERT: 8777c555fc kernel: Add pure kernel bitcoin-chainstate REVERT: fcc69abdb9 kernel: Add block index utility functions to C header REVERT: 266d21c9c6 kernel: Add function to read block undo data from disk to C header REVERT: 62b2926c0a kernel: Add functions to read block from disk to C header REVERT: 440d7a7a86 kernel: Add function for copying block data to C header REVERT: 8827ebd5ea kernel: Add functions for the block validation state to C header REVERT: 1ad10251b4 kernel: Add validation interface and task runner to C header REVERT: b96eb0c49c kernel: Add interrupt function to C header REVERT: d96ffb9165 kernel: Add import blocks function to C header REVERT: 28dc294a13 kernel: Add chainstate load options for in-memory dbs in C header REVERT: 7de1202a01 kernel: Add options for reindexing in C header REVERT: 2e28c5516d kernel: Add block validation to C header REVERT: f511ce1489 Kernel: Add chainstate loading to kernel C header REVERT: 7ed0a89a3e kernel: Add chainstate manager object to C header REVERT: 3fb9cec5d4 kernel: Add notifications context option to C header REVERT: d71bbbab94 kerenl: Add chain params context option to C header REVERT: 7cfc892152 kernel: Add kernel library context object REVERT: ba01b6bd0e kernel: Add logging to kernel library C header REVERT: 17b98b95eb kernel: Introduce initial kernel C header API REVERT: 0c4ff18ee9 Merge bitcoin/bitcoin#30896: kernel: Move background load thread to node context REVERT: 87d54500bf Merge bitcoin/bitcoin#30892: test: Check already deactivated network stays suspended after dumptxoutset REVERT: 71af7435ef Merge bitcoin/bitcoin#30233: refactor: move m_is_inbound out of CNodeState REVERT: 1d5b2406bb Merge bitcoin/bitcoin#30877: code style: update .editorconfig file REVERT: fea550b480 Merge bitcoin/bitcoin#30890: doc: unit test runner help fixup and improvements REVERT: 95560616fb code style: update .editorconfig file REVERT: 282f0e9255 Unit test runner documentation fix and improvements REVERT: 06a9f7789e Merge bitcoin/bitcoin#30433: build: add `standard branch-protection` to hardening flags for aarch64-linux REVERT: bc7900f33d kernel: Move background load thread to node context REVERT: e43ce250c6 Merge bitcoin-core/gui#835: Fix crash when closing wallet REVERT: 001b1cf010 build: use standard branch-protection for aarch64-linux REVERT: a965f2bc07 gui: fix crash when closing wallet REVERT: 72c9a1fe94 test: Check that network stays suspended after dumptxoutset if it was off before REVERT: cf0120ff02 Merge bitcoin/bitcoin#30880: test: Wait for local services to update in feature_assumeutxo REVERT: e46bebb444 Merge bitcoin/bitcoin#30546: util: Use consteval checked format string in FatalErrorf, LogConnectFailure REVERT: be768dbd18 Merge bitcoin/bitcoin#30618: test: support std::optional in BOOST_CHECK_* and increase FromUserHex fuzz feature coverage REVERT: 07c7c96022 Merge bitcoin/bitcoin#30883: build: Revert "Minimize I/O operations in `GenerateHeaderFrom{Json,Raw}.cmake`" REVERT: 24817e8b15 Merge bitcoin/bitcoin#30814: kernel: Create usable static kernel library REVERT: fdeb717e78 Revert "build: Minimize I/O operations in `GenerateHeaderFrom{Json,Raw}.cmake`" REVERT: 07f4cebe57 refactor: move m_is_inbound out of CNodeState REVERT: 19f4a7c95a test: Wait for local services to update in feature_assumeutxo REVERT: 7d43bca052 Merge bitcoin/bitcoin#30872: test: fix exclude parsing for functional runner REVERT: cf786eccd7 Merge bitcoin/bitcoin#30865: build: Skip secp256k1 ctime tests when tests are not being built REVERT: 23eedc5d1e build: Skip secp256k1 ctime tests when tests are not being built REVERT: fa5bc450d5 util: Use compile-time check for LogConnectFailure REVERT: fa7087b896 util: Use compile-time check for FatalErrorf REVERT: faa62c0112 util: Add ConstevalFormatString REVERT: 72b46f28bf test: fix exclude parsing for functional runner REVERT: a5e99669cc Merge bitcoin/bitcoin#30733: test: remove unused src_dir param from run_tests after CMake migration REVERT: 0c1e507278 Merge bitcoin/bitcoin#30871: build: Add more cmake presets REVERT: fcb61bbc8d Merge bitcoin/bitcoin#27038: security-check: test for `_FORTIFY_SOURCE` usage in release binaries REVERT: 85833cf05f Merge bitcoin/bitcoin#30847: test: Drop no longer needed workarounds REVERT: 11e2f9fff4 Merge bitcoin/bitcoin#30835: build: Introduce "Kernel" installation component REVERT: db8350b0e3 Merge bitcoin/bitcoin#30803: build: Minor build system fixes and amendments REVERT: a86e7a476d Merge bitcoin/bitcoin#30838: build: Use CMake's default permissions in macOS `deploy` target REVERT: f0eb63399a Merge bitcoin/bitcoin#30841: ci: Post CMake-migration fixes and amendments REVERT: 155963768a Merge bitcoin/bitcoin#30842: build: Minimize I/O operations in `GenerateHeaderFrom{Json,Raw}.cmake` REVERT: c773618886 Merge bitcoin/bitcoin#30867: build: Fix `ENABLE_WALLET` option REVERT: 349632e022 Merge bitcoin/bitcoin#30807: Fix peers abruptly disconnecting from AssumeUTXO nodes during IBD REVERT: f6298a878f Merge bitcoin/bitcoin#30840: docs: Updated debug build instructions for cmake REVERT: a8809aeb6e Merge bitcoin/bitcoin#30870: docs: updated developer notes for --with-sanitizers to -DSANITIZERS REVERT: f15e817811 build: add more CMake presets (dev-mode, libfuzzer, libfuzzer-nosan) REVERT: fae7b83eb5 lint: Remove forbidden functions from lint-format-strings.py REVERT: 4b1ce3cac8 docs: updated developer notes for --with-sanitizers to -DSANITIZERS and removed resource for -fsanitze flags REVERT: 1eac96a503 Compare FromUserHex result against other hex validators and parsers REVERT: 19947863e1 Use BOOST_CHECK_EQUAL for optional, arith_uint256, uint256, uint160 REVERT: 0725a37494 Merge bitcoin/bitcoin#30805: test: Add explicit onion bind to p2p_permissions REVERT: 0037d53d1a build: Fix `ENABLE_WALLET` option REVERT: 992f83bb6f test: add coverage for assumeUTXO honest peers disconnection REVERT: 6d5812e5c8 assumeUTXO: fix peers disconnection during sync REVERT: 082779d606 test: Add explicit onion bind to p2p_permissions REVERT: c66c68345e Merge bitcoin/bitcoin#30773: Remove unsafe uint256S() and test-only uint160S() REVERT: 2756797eca Merge bitcoin/bitcoin#30065: init: fixes file descriptor accounting REVERT: 5ba03e7d35 build: Use CMake's default permissions in macOS `deploy` target REVERT: e4fb97a512 Merge bitcoin/bitcoin#30791: build: Use correct variable name REVERT: df3f63ccfa Merge bitcoin/bitcoin#30509: multiprocess: Add -ipcbind option to bitcoin-node REVERT: 743ac30e34 Add std::optional support to Boost's equality check REVERT: 712a2b5453 Merge bitcoin/bitcoin#30817: test: Add coverage for dumptxoutset failure robustness REVERT: fb52023ee6 Merge bitcoin/bitcoin#30684: init: fix init fatal error on invalid negated option value REVERT: 746f88000e Merge bitcoin/bitcoin#30401: fix: increase consistency of rpcauth parsing REVERT: 2d68c3b1c2 build: Use correct variables when passing `-fsanitize` to libsecp256k1 REVERT: df86a4f333 Merge bitcoin/bitcoin#30845: Update libsecp256k1 subtree to latest master REVERT: be4f78275f contrib: test for FORTIFY_SOURCE in security-check.py REVERT: 94bc3c4cc0 Merge bitcoin/bitcoin#30824: cmake: decouple `FORTIFY_SOURCE` check from `Debug` build type REVERT: ba84c2774d Merge bitcoin/bitcoin#30823: cmake: add `USE_SOURCE_PERMISSIONS` to all `configure_file()` usage REVERT: da3f4cb8ee Merge bitcoin/bitcoin#30850: doc: fix minor typo REVERT: 7a669fde18 docs: Fix minor typo REVERT: 1cc93fe7b4 build: Delete dead code that implements `IF_CHECK_FAILED` option REVERT: 0b003e1ff7 docs: Updated debug build instructions for cmake REVERT: 341ad23809 build: Delete MSVC special case for `BUILD_FOR_FUZZING` option REVERT: 5c80192ff6 test: Drop no longer needed workarounds REVERT: ff54395de4 Update secp256k1 subtree to latest master REVERT: 611562806c Squashed 'src/secp256k1/' changes from 642c885b61..2f2ccc4695 REVERT: b07fe666f2 build: Minimize I/O operations in `GenerateHeaderFrom{Json,Raw}.cmake` REVERT: c45186ca54 ci: Switch from `make` to `cmake --build` REVERT: 6e5f33af58 ci: Handle log files regardless of CMake's version REVERT: fdad128b52 build: Stop enabling CMake's CMP0141 policy REVERT: b2a6f545b4 doc: Drop `ctest` command from Windows cross-compiling instructions REVERT: 73b618582d build: Print `CMAKE_CXX_COMPILER_ARG1` in summary REVERT: f03c942095 build, test: Add missed log options REVERT: 6f2cb0eafd doc: Amend comment about ZeroMQ config files REVERT: 0dd16d7118 build: Add a pkg-config file for libbitcoinkernel REVERT: 45be32f838 build: Produce a usable static kernel library REVERT: 43cd83b0c7 test: move uint256_tests/operator_with_self to arith_uint256_tests REVERT: c6c994cb2b test: remove test-only uint160S REVERT: 62cc4656e2 test: remove test-only uint256S REVERT: adc00ad728 test: remove test-only arith_uint256S REVERT: a5fa90706a Merge bitcoin/bitcoin#30834: test: Work around boost compilation error REVERT: 7b04fabe2d build: Introduce "Kernel" installation component REVERT: fa9d7d5d20 test: Work around boost compilation error REVERT: fa3ecdf778 Revert "build: work around issue with Boost <= 1.80 and Clang >= 18" REVERT: 30073e6b3a multiprocess: Add -ipcbind option to bitcoin-node REVERT: bbf95c0cc5 Merge bitcoin/bitcoin#30755: ci: Add missed configuration options to "Win64 native" job REVERT: 73fe7d7230 multiprocess: Add unit tests for connect, serve, and listen functions REVERT: 955d4077aa multiprocess: Add IPC connectAddress and listenAddress methods REVERT: 4da20434d4 depends: Update libmultiprocess library for CustomMessage function and ThreadContext bugfix REVERT: ee22bf55e3 doc: Update and amend MSVC build guide REVERT: c07fdd6546 fuzz: Don't compile BDB-specific code on MSVC in `wallet_bdb_parser.cpp` REVERT: e07a3ede52 ci: Add missed configuration options to "Win64 native" job REVERT: 1f054eca4e cmake: add USE_SOURCE_PERMISSIONS to all configure_file usage REVERT: 0e5cd608da Merge bitcoin/bitcoin#30415: contrib: fix check-deps.sh to check for weak symbols REVERT: 118b55c462 Merge bitcoin/bitcoin#30790: bench: Remove redundant logging benchmarks REVERT: c0cbe26a86 Merge bitcoin/bitcoin#30748: test: Pin and document TEST_DIR_PATH_ELEMENT, SeedRand::FIXED_SEED REVERT: c3af4b1ec3 Merge bitcoin/bitcoin#30822: cmake: scope Boost Test check to `vcpkg` REVERT: 7f472e9bcd Merge bitcoin/bitcoin#30821: build: work around issue with Boost <= 1.80 and Clang >= 18 REVERT: d4c7c4009d init: error out if -maxconnections is negative REVERT: c773649481 init: improves file descriptors accounting and docs REVERT: 29008a7ff4 init: fixes fd accounting regarding poll/select REVERT: 30803a35d5 cmake: decouple FORTIFY_SOURCE check from Debug build type REVERT: a7a4e11db8 cmake: scope Boost Test check to vcpkg REVERT: d661e2b1b7 Merge bitcoin/bitcoin#30812: lint: Check for release note snippets in the wrong folder REVERT: cd062d6684 build: work around issue with Boost <= 1.80 and Clang >= 18 REVERT: d6a1b94ffd Merge bitcoin-core/gui#834: qt, build: remove unneeded `Q_IMPORT_PLUGIN` macro calls REVERT: 6852d1d487 Merge bitcoin/bitcoin#30796: test: Use std::span and std::string_view for raw data REVERT: fa3a7ebe5b lint: Check for release note snippets in the wrong folder REVERT: fa05ee0517 Merge bitcoin/bitcoin#30772: build: Fix / improve coverage scripts REVERT: 79772cd26e Merge bitcoin/bitcoin#30743: depends: build libevent with `-D_GNU_SOURCE` REVERT: faecca9a85 test: Use span for raw data REVERT: c2b779da4e refactor: Manage dumptxoutset RAII classes with std::optional REVERT: 4b5bf335ad test: Add coverage for failing dumptxoutset behavior REVERT: f794a0d5f4 Merge bitcoin/bitcoin#30819: doc: fix assumeutxo design doc link REVERT: fadbcd51fc bench: Remove redundant logging benchmarks REVERT: fa8dd952e2 bench: Use LogInfo instead of the deprecated alias LogPrintf REVERT: e5f7272ad3 doc: fix assumeutxo design doc link REVERT: 93e48240bf Merge bitcoin/bitcoin#30244: ci: parse TEST_RUNNER_EXTRA into an array REVERT: f640b323bd Merge bitcoin/bitcoin#30723: lint: Speed up and fix flake8 checks REVERT: 3ae35b427f ci: run check-deps.sh as part of clang-tidy job REVERT: 0aaa1298a0 contrib: fix check-deps.sh when libraries do not import symbols REVERT: 3c99f5a38a contrib: fix check-deps.sh to check for weak symbols REVERT: 86c80e9cf2 contrib: make check-deps.sh script work with cmake REVERT: 5373aa30e2 Merge bitcoin/bitcoin#30788: test: fixing failing system_tests/run_command under some Locales REVERT: 3210d87dfc Merge bitcoin/bitcoin#29043: fuzz: make FuzzedDataProvider usage deterministic REVERT: 81276540d3 Merge bitcoin/bitcoin#30148: cli: restrict multiple exclusive argument usage in bitcoin-cli REVERT: 210210c923 Merge bitcoin/bitcoin#29566: test: update satoshi_round function REVERT: b0c3de6847 Merge bitcoin/bitcoin#28417: contrib/signet/miner updates REVERT: cb65ac469a Merge bitcoin/bitcoin#29605: net: Favor peers from addrman over fetching seednodes REVERT: b8d2f58e06 Merge bitcoin/bitcoin#30808: rpc: dumptxoutset height parameter follow-ups (29553) REVERT: f51b237723 refactor: rpc: use uint256::FromHex for ParseHashV REVERT: d9fcbfc372 build: Add `JOBS` variable support to `CoverageFuzz.cmake` script REVERT: e7cf4a6f27 build: Add missed `-g` for "Coverage" build configuration REVERT: fe2003ab12 build: Add `COMMAND_ERROR_IS_FATAL` to every process in coverage scrips REVERT: a3108a7c56 rpc: Manage dumptxoutset rollback with RAII class REVERT: c5eaae3b89 doc: Add -rpcclienttimeout=0 to loadtxoutset examples REVERT: 598b9bba5a rpc: Don't re-enable previously disabled network after dumptxoutset REVERT: ae48a22a3d test: fixing failing system_tests/run_command under some Locales REVERT: fac973647d test: Use string_view for json_tests REVERT: 5567754087 depends: build libevent with -D_GNU_SOURCE REVERT: 7346b01092 qt, build: remove unneeded `Q_IMPORT_PLUGIN` macro calls REVERT: fa84f9decd test: Pin and document TEST_DIR_PATH_ELEMENT REVERT: 2ad560139b Remove unused src_dir param from run_tests REVERT: 2222f7a874 test: Rename SeedRand::SEED to FIXED_SEED for clarity REVERT: fafdb7df34 lint: Speed up flake8 checks REVERT: faf17df7fb lint: Document missing py_lint dependency REVERT: faebeb828f lint: Remove python whitespace and shadowing lint rules REVERT: 7777047835 lint: Remove python lint rules that are SyntaxError REVERT: faaf3e53f0 test: [refactor] Fix F841 flake8 REVERT: 444421db69 test: [refactor] Fix E714 pycodestyle REVERT: ee47ca29d6 init: fix fatal error on '-wallet' negated option value REVERT: 27c976d11a fix: increase consistency of rpcauth parsing REVERT: 2ad3689512 test: add norpcauth test REVERT: 67df0dec1a test: blank rpcauth CLI interaction REVERT: fb6d51eb25 signet/miner: Use argparse exclusive groups REVERT: ec317bc44b test: update satoshi_round function REVERT: ecc98ccff2 test: add cases for blank rpcauth REVERT: c8e6771af0 test: restrict multiple CLI arguments REVERT: 8838c4f171 common/args.h: automate check for multiple cli commands REVERT: 6eeb188d40 test: adds seednode functional tests REVERT: 3270f0adad net: Favor peers from addrman over fetching seednodes REVERT: 8131bf7483 ci: parse TEST_RUNNER_EXTRA into an array REVERT: c4762b0aa0 test: allow excluding func test by name and arg REVERT: 338a266a9a signet/miner: add support for a poolnum/poolid tag in mined blocks REVERT: 409ab7d35b signet/miner: add Generate.mine function REVERT: 7b31332370 signet/miner: add Generate.gbt function REVERT: 85c5c0bea9 signet/miner: add Generate.next_block_time function REVERT: 5540e6ca49 signet/miner: move next_block_* functions into new Generator class REVERT: 35f4631196 signet/miner: rename do_decode_psbt to decode_psbt REVERT: aac040b439 signet/miner: drop create_coinbase function REVERT: 16951f549e signet/miner: drop do_createpsbt function REVERT: 3aed0a4284 signet/miner: drop get_reward_address function REVERT: 01960c53c7 fuzz: make FuzzedDataProvider usage deterministic git-subtree-dir: libbitcoinkernel-sys/bitcoin git-subtree-split: d8b6e67b2a12332f188320e90262e2d741e27f7b

…e7b1f5e d5ee7b1f5e kernel: Add check if the chainstate maanger is busy loading blocks 604752a1da kernel: Process transactions 37d74344e8 kernel: Add optional mempool fdfb45e273 kernel: Add support for handling transactions 5ed4aa699c kernel: Add check for when a block has been mutated. b8f6a2740d kernel: Add support for handling block headers 1aad96a695 kernel: Add utxo set iteration and value retrieval 8777c555fc kernel: Add pure kernel bitcoin-chainstate fcc69abdb9 kernel: Add block index utility functions to C header 266d21c9c6 kernel: Add function to read block undo data from disk to C header 62b2926c0a kernel: Add functions to read block from disk to C header 440d7a7a86 kernel: Add function for copying block data to C header 8827ebd5ea kernel: Add functions for the block validation state to C header 1ad10251b4 kernel: Add validation interface and task runner to C header b96eb0c49c kernel: Add interrupt function to C header d96ffb9165 kernel: Add import blocks function to C header 28dc294a13 kernel: Add chainstate load options for in-memory dbs in C header 7de1202a01 kernel: Add options for reindexing in C header 2e28c5516d kernel: Add block validation to C header f511ce1489 Kernel: Add chainstate loading to kernel C header 7ed0a89a3e kernel: Add chainstate manager object to C header 3fb9cec5d4 kernel: Add notifications context option to C header d71bbbab94 kerenl: Add chain params context option to C header 7cfc892152 kernel: Add kernel library context object ba01b6bd0e kernel: Add logging to kernel library C header 17b98b95eb kernel: Introduce initial kernel C header API 0c4ff18ee9 Merge bitcoin/bitcoin#30896: kernel: Move background load thread to node context 87d54500bf Merge bitcoin/bitcoin#30892: test: Check already deactivated network stays suspended after dumptxoutset 71af7435ef Merge bitcoin/bitcoin#30233: refactor: move m_is_inbound out of CNodeState 1d5b2406bb Merge bitcoin/bitcoin#30877: code style: update .editorconfig file fea550b480 Merge bitcoin/bitcoin#30890: doc: unit test runner help fixup and improvements 95560616fb code style: update .editorconfig file 282f0e9255 Unit test runner documentation fix and improvements 06a9f7789e Merge bitcoin/bitcoin#30433: build: add `standard branch-protection` to hardening flags for aarch64-linux bc7900f33d kernel: Move background load thread to node context e43ce250c6 Merge bitcoin-core/gui#835: Fix crash when closing wallet 001b1cf010 build: use standard branch-protection for aarch64-linux a965f2bc07 gui: fix crash when closing wallet 72c9a1fe94 test: Check that network stays suspended after dumptxoutset if it was off before cf0120ff02 Merge bitcoin/bitcoin#30880: test: Wait for local services to update in feature_assumeutxo e46bebb444 Merge bitcoin/bitcoin#30546: util: Use consteval checked format string in FatalErrorf, LogConnectFailure be768dbd18 Merge bitcoin/bitcoin#30618: test: support std::optional in BOOST_CHECK_* and increase FromUserHex fuzz feature coverage 07c7c96022 Merge bitcoin/bitcoin#30883: build: Revert "Minimize I/O operations in `GenerateHeaderFrom{Json,Raw}.cmake`" 24817e8b15 Merge bitcoin/bitcoin#30814: kernel: Create usable static kernel library fdeb717e78 Revert "build: Minimize I/O operations in `GenerateHeaderFrom{Json,Raw}.cmake`" 07f4cebe57 refactor: move m_is_inbound out of CNodeState 19f4a7c95a test: Wait for local services to update in feature_assumeutxo 7d43bca052 Merge bitcoin/bitcoin#30872: test: fix exclude parsing for functional runner cf786eccd7 Merge bitcoin/bitcoin#30865: build: Skip secp256k1 ctime tests when tests are not being built 23eedc5d1e build: Skip secp256k1 ctime tests when tests are not being built fa5bc450d5 util: Use compile-time check for LogConnectFailure fa7087b896 util: Use compile-time check for FatalErrorf faa62c0112 util: Add ConstevalFormatString 72b46f28bf test: fix exclude parsing for functional runner a5e99669cc Merge bitcoin/bitcoin#30733: test: remove unused src_dir param from run_tests after CMake migration 0c1e507278 Merge bitcoin/bitcoin#30871: build: Add more cmake presets fcb61bbc8d Merge bitcoin/bitcoin#27038: security-check: test for `_FORTIFY_SOURCE` usage in release binaries 85833cf05f Merge bitcoin/bitcoin#30847: test: Drop no longer needed workarounds 11e2f9fff4 Merge bitcoin/bitcoin#30835: build: Introduce "Kernel" installation component db8350b0e3 Merge bitcoin/bitcoin#30803: build: Minor build system fixes and amendments a86e7a476d Merge bitcoin/bitcoin#30838: build: Use CMake's default permissions in macOS `deploy` target f0eb63399a Merge bitcoin/bitcoin#30841: ci: Post CMake-migration fixes and amendments 155963768a Merge bitcoin/bitcoin#30842: build: Minimize I/O operations in `GenerateHeaderFrom{Json,Raw}.cmake` c773618886 Merge bitcoin/bitcoin#30867: build: Fix `ENABLE_WALLET` option 349632e022 Merge bitcoin/bitcoin#30807: Fix peers abruptly disconnecting from AssumeUTXO nodes during IBD f6298a878f Merge bitcoin/bitcoin#30840: docs: Updated debug build instructions for cmake a8809aeb6e Merge bitcoin/bitcoin#30870: docs: updated developer notes for --with-sanitizers to -DSANITIZERS f15e817811 build: add more CMake presets (dev-mode, libfuzzer, libfuzzer-nosan) fae7b83eb5 lint: Remove forbidden functions from lint-format-strings.py 4b1ce3cac8 docs: updated developer notes for --with-sanitizers to -DSANITIZERS and removed resource for -fsanitze flags 1eac96a503 Compare FromUserHex result against other hex validators and parsers 19947863e1 Use BOOST_CHECK_EQUAL for optional, arith_uint256, uint256, uint160 0725a37494 Merge bitcoin/bitcoin#30805: test: Add explicit onion bind to p2p_permissions 0037d53d1a build: Fix `ENABLE_WALLET` option 992f83bb6f test: add coverage for assumeUTXO honest peers disconnection 6d5812e5c8 assumeUTXO: fix peers disconnection during sync 082779d606 test: Add explicit onion bind to p2p_permissions c66c68345e Merge bitcoin/bitcoin#30773: Remove unsafe uint256S() and test-only uint160S() 2756797eca Merge bitcoin/bitcoin#30065: init: fixes file descriptor accounting 5ba03e7d35 build: Use CMake's default permissions in macOS `deploy` target e4fb97a512 Merge bitcoin/bitcoin#30791: build: Use correct variable name df3f63ccfa Merge bitcoin/bitcoin#30509: multiprocess: Add -ipcbind option to bitcoin-node 743ac30e34 Add std::optional support to Boost's equality check 712a2b5453 Merge bitcoin/bitcoin#30817: test: Add coverage for dumptxoutset failure robustness fb52023ee6 Merge bitcoin/bitcoin#30684: init: fix init fatal error on invalid negated option value 746f88000e Merge bitcoin/bitcoin#30401: fix: increase consistency of rpcauth parsing 2d68c3b1c2 build: Use correct variables when passing `-fsanitize` to libsecp256k1 df86a4f333 Merge bitcoin/bitcoin#30845: Update libsecp256k1 subtree to latest master be4f78275f contrib: test for FORTIFY_SOURCE in security-check.py 94bc3c4cc0 Merge bitcoin/bitcoin#30824: cmake: decouple `FORTIFY_SOURCE` check from `Debug` build type ba84c2774d Merge bitcoin/bitcoin#30823: cmake: add `USE_SOURCE_PERMISSIONS` to all `configure_file()` usage da3f4cb8ee Merge bitcoin/bitcoin#30850: doc: fix minor typo 7a669fde18 docs: Fix minor typo 1cc93fe7b4 build: Delete dead code that implements `IF_CHECK_FAILED` option 0b003e1ff7 docs: Updated debug build instructions for cmake 341ad23809 build: Delete MSVC special case for `BUILD_FOR_FUZZING` option 5c80192ff6 test: Drop no longer needed workarounds ff54395de4 Update secp256k1 subtree to latest master 611562806c Squashed 'src/secp256k1/' changes from 642c885b61..2f2ccc4695 b07fe666f2 build: Minimize I/O operations in `GenerateHeaderFrom{Json,Raw}.cmake` c45186ca54 ci: Switch from `make` to `cmake --build` 6e5f33af58 ci: Handle log files regardless of CMake's version fdad128b52 build: Stop enabling CMake's CMP0141 policy b2a6f545b4 doc: Drop `ctest` command from Windows cross-compiling instructions 73b618582d build: Print `CMAKE_CXX_COMPILER_ARG1` in summary f03c942095 build, test: Add missed log options 6f2cb0eafd doc: Amend comment about ZeroMQ config files 0dd16d7118 build: Add a pkg-config file for libbitcoinkernel 45be32f838 build: Produce a usable static kernel library 43cd83b0c7 test: move uint256_tests/operator_with_self to arith_uint256_tests c6c994cb2b test: remove test-only uint160S 62cc4656e2 test: remove test-only uint256S adc00ad728 test: remove test-only arith_uint256S a5fa90706a Merge bitcoin/bitcoin#30834: test: Work around boost compilation error 7b04fabe2d build: Introduce "Kernel" installation component fa9d7d5d20 test: Work around boost compilation error fa3ecdf778 Revert "build: work around issue with Boost <= 1.80 and Clang >= 18" 30073e6b3a multiprocess: Add -ipcbind option to bitcoin-node bbf95c0cc5 Merge bitcoin/bitcoin#30755: ci: Add missed configuration options to "Win64 native" job 73fe7d7230 multiprocess: Add unit tests for connect, serve, and listen functions 955d4077aa multiprocess: Add IPC connectAddress and listenAddress methods 4da20434d4 depends: Update libmultiprocess library for CustomMessage function and ThreadContext bugfix ee22bf55e3 doc: Update and amend MSVC build guide c07fdd6546 fuzz: Don't compile BDB-specific code on MSVC in `wallet_bdb_parser.cpp` e07a3ede52 ci: Add missed configuration options to "Win64 native" job 1f054eca4e cmake: add USE_SOURCE_PERMISSIONS to all configure_file usage 0e5cd608da Merge bitcoin/bitcoin#30415: contrib: fix check-deps.sh to check for weak symbols 118b55c462 Merge bitcoin/bitcoin#30790: bench: Remove redundant logging benchmarks c0cbe26a86 Merge bitcoin/bitcoin#30748: test: Pin and document TEST_DIR_PATH_ELEMENT, SeedRand::FIXED_SEED c3af4b1ec3 Merge bitcoin/bitcoin#30822: cmake: scope Boost Test check to `vcpkg` 7f472e9bcd Merge bitcoin/bitcoin#30821: build: work around issue with Boost <= 1.80 and Clang >= 18 d4c7c4009d init: error out if -maxconnections is negative c773649481 init: improves file descriptors accounting and docs 29008a7ff4 init: fixes fd accounting regarding poll/select 30803a35d5 cmake: decouple FORTIFY_SOURCE check from Debug build type a7a4e11db8 cmake: scope Boost Test check to vcpkg d661e2b1b7 Merge bitcoin/bitcoin#30812: lint: Check for release note snippets in the wrong folder cd062d6684 build: work around issue with Boost <= 1.80 and Clang >= 18 d6a1b94ffd Merge bitcoin-core/gui#834: qt, build: remove unneeded `Q_IMPORT_PLUGIN` macro calls 6852d1d487 Merge bitcoin/bitcoin#30796: test: Use std::span and std::string_view for raw data fa3a7ebe5b lint: Check for release note snippets in the wrong folder fa05ee0517 Merge bitcoin/bitcoin#30772: build: Fix / improve coverage scripts 79772cd26e Merge bitcoin/bitcoin#30743: depends: build libevent with `-D_GNU_SOURCE` faecca9a85 test: Use span for raw data c2b779da4e refactor: Manage dumptxoutset RAII classes with std::optional 4b5bf335ad test: Add coverage for failing dumptxoutset behavior f794a0d5f4 Merge bitcoin/bitcoin#30819: doc: fix assumeutxo design doc link fadbcd51fc bench: Remove redundant logging benchmarks fa8dd952e2 bench: Use LogInfo instead of the deprecated alias LogPrintf e5f7272ad3 doc: fix assumeutxo design doc link 93e48240bf Merge bitcoin/bitcoin#30244: ci: parse TEST_RUNNER_EXTRA into an array f640b323bd Merge bitcoin/bitcoin#30723: lint: Speed up and fix flake8 checks 3ae35b427f ci: run check-deps.sh as part of clang-tidy job 0aaa1298a0 contrib: fix check-deps.sh when libraries do not import symbols 3c99f5a38a contrib: fix check-deps.sh to check for weak symbols 86c80e9cf2 contrib: make check-deps.sh script work with cmake 5373aa30e2 Merge bitcoin/bitcoin#30788: test: fixing failing system_tests/run_command under some Locales 3210d87dfc Merge bitcoin/bitcoin#29043: fuzz: make FuzzedDataProvider usage deterministic 81276540d3 Merge bitcoin/bitcoin#30148: cli: restrict multiple exclusive argument usage in bitcoin-cli 210210c923 Merge bitcoin/bitcoin#29566: test: update satoshi_round function b0c3de6847 Merge bitcoin/bitcoin#28417: contrib/signet/miner updates cb65ac469a Merge bitcoin/bitcoin#29605: net: Favor peers from addrman over fetching seednodes b8d2f58e06 Merge bitcoin/bitcoin#30808: rpc: dumptxoutset height parameter follow-ups (29553) f51b237723 refactor: rpc: use uint256::FromHex for ParseHashV d9fcbfc372 build: Add `JOBS` variable support to `CoverageFuzz.cmake` script e7cf4a6f27 build: Add missed `-g` for "Coverage" build configuration fe2003ab12 build: Add `COMMAND_ERROR_IS_FATAL` to every process in coverage scrips a3108a7c56 rpc: Manage dumptxoutset rollback with RAII class c5eaae3b89 doc: Add -rpcclienttimeout=0 to loadtxoutset examples 598b9bba5a rpc: Don't re-enable previously disabled network after dumptxoutset ae48a22a3d test: fixing failing system_tests/run_command under some Locales fac973647d test: Use string_view for json_tests 5567754087 depends: build libevent with -D_GNU_SOURCE 7346b01092 qt, build: remove unneeded `Q_IMPORT_PLUGIN` macro calls fa84f9decd test: Pin and document TEST_DIR_PATH_ELEMENT 2ad560139b Remove unused src_dir param from run_tests 2222f7a874 test: Rename SeedRand::SEED to FIXED_SEED for clarity fafdb7df34 lint: Speed up flake8 checks faf17df7fb lint: Document missing py_lint dependency faebeb828f lint: Remove python whitespace and shadowing lint rules 7777047835 lint: Remove python lint rules that are SyntaxError faaf3e53f0 test: [refactor] Fix F841 flake8 444421db69 test: [refactor] Fix E714 pycodestyle ee47ca29d6 init: fix fatal error on '-wallet' negated option value 27c976d11a fix: increase consistency of rpcauth parsing 2ad3689512 test: add norpcauth test 67df0dec1a test: blank rpcauth CLI interaction fb6d51eb25 signet/miner: Use argparse exclusive groups ec317bc44b test: update satoshi_round function ecc98ccff2 test: add cases for blank rpcauth c8e6771af0 test: restrict multiple CLI arguments 8838c4f171 common/args.h: automate check for multiple cli commands 6eeb188d40 test: adds seednode functional tests 3270f0adad net: Favor peers from addrman over fetching seednodes 8131bf7483 ci: parse TEST_RUNNER_EXTRA into an array c4762b0aa0 test: allow excluding func test by name and arg 338a266a9a signet/miner: add support for a poolnum/poolid tag in mined blocks 409ab7d35b signet/miner: add Generate.mine function 7b31332370 signet/miner: add Generate.gbt function 85c5c0bea9 signet/miner: add Generate.next_block_time function 5540e6ca49 signet/miner: move next_block_* functions into new Generator class 35f4631196 signet/miner: rename do_decode_psbt to decode_psbt aac040b439 signet/miner: drop create_coinbase function 16951f549e signet/miner: drop do_createpsbt function 3aed0a4284 signet/miner: drop get_reward_address function 01960c53c7 fuzz: make FuzzedDataProvider usage deterministic REVERT: d8b6e67b2a kernel: Add check if the chainstate maanger is busy loading blocks REVERT: f12a131af8 kernel: Process transactions REVERT: 75d48e6718 kernel: Add optional mempool REVERT: 889632bb70 kernel: Add support for handling transactions REVERT: 476c754f3b kernel: Add check for when a block has been mutated. REVERT: ebef1b465b kernel: Add support for handling block headers REVERT: a56c79a2b5 kernel: Add utxo set iteration and value retrieval REVERT: 33c71843e3 kernel: Add pure kernel bitcoin-chainstate REVERT: ccb2bb59f9 kernel: Add block index utility functions to C header REVERT: c953e1f530 kernel: Add function to read block undo data from disk to C header REVERT: 55673f7f70 kernel: Add functions to read block from disk to C header REVERT: 348e1c0a22 kernel: Add function for copying block data to C header REVERT: 4c71ffbef6 kernel: Add functions for the block validation state to C header REVERT: 160343c9c2 kernel: Add validation interface and task runner to C header REVERT: 55a60d8c3f kernel: Add interrupt function to C header REVERT: 0d59e95b82 kernel: Add import blocks function to C header REVERT: 804e064ff6 kernel: Add chainstate load options for in-memory dbs in C header REVERT: 1c829ce9f4 kernel: Add options for reindexing in C header REVERT: c8a2917004 kernel: Add block validation to C header REVERT: 8015b8374c Kernel: Add chainstate loading to kernel C header REVERT: b3c8467b62 kernel: Add chainstate manager object to C header REVERT: 07c55185c2 kernel: Add notifications context option to C header REVERT: 71c45c0680 kerenl: Add chain params context option to C header REVERT: d2ad67bd5f kernel: Add kernel library context object REVERT: 1c6716c609 kernel: Add logging to kernel library C header REVERT: 63a83b8dad kernel: Introduce initial kernel C header API git-subtree-dir: libbitcoinkernel-sys/bitcoin git-subtree-split: d5ee7b1f5ee19e454a74d00502a0bf7be849c260

Summary: This adds a height parameter to the dumptxoutset RPC. This internalizes the workflow that was previously done by scripts: roll back the chain to the height we actually want the snapshot from, create the snapshot, roll forward to the real tip again. The nice thing about internalizing this functionality is that we can write tests for the code and it gives us more options to make the functionality robust. The shell scripts we have so far will be more cumbersome to maintain in the long run, especially since we will only notice later when we have broken them. This is a partial backport of [[bitcoin/bitcoin#29553 | core#29553]], [[bitcoin/bitcoin#18836 | core#18836]], [[bitcoin/bitcoin#30808 | core#30808]] and [[bitcoin/bitcoin#30817 | core#30817]] ------ > RPC: Add type parameter to dumptxoutset bitcoin/bitcoin@993cafe ------ > test: Test for dumptxoutset at specific height bitcoin/bitcoin@8426850 ------ > tests: Add a sha256sum_file function to util bitcoin/bitcoin@092fc43 ------ > assumeutxo: Remove devtools/utxo_snapshot.sh bitcoin/bitcoin@b29c21f ------ > rpc: Don't re-enable previously disabled network after dumptxoutset > > Also fixes a typo in the RPC help text. bitcoin/bitcoin@598b9bb ------ > refactor: Manage dumptxoutset RAII classes with std::optional (partial) bitcoin/bitcoin@c2b779d Test Plan: `ninja all check-all` Reviewers: #bitcoin_abc, Fabien Reviewed By: #bitcoin_abc, Fabien Differential Revision: https://reviews.bitcoinabc.org/D18047

Summary: This is a partial backport of [[bitcoin/bitcoin#30808 | core#30808]] and [[bitcoin/bitcoin#30817 | core#30817]] bitcoin/bitcoin@a3108a7 bitcoin/bitcoin@c2b779d Depends on D18048 Test Plan: `ninja all check-all` Reviewers: #bitcoin_abc, Fabien Reviewed By: #bitcoin_abc, Fabien Differential Revision: https://reviews.bitcoinabc.org/D18049

Summary: This new doc will be available on the bitcoinabc.org/doc/assumeutxo.html URL after the next deployment of the website, and can be given to users. When can add a link to snapshot files when the hosting is sorted out (see TODO note left in the document). I added a section to document the current Chronik incompatibility, similar to what is in the release notes. > Also fixes some outdated information in the remaining design doc. bitcoin/bitcoin@e868a6e > Add -rpcclienttimeout=0 to loadtxoutset examples bitcoin/bitcoin@c5eaae3 > fix assumeutxo design doc link This concludes backport of [[bitcoin/bitcoin#29553 | core#29553]], [[bitcoin/bitcoin#30808 | core#30808]] and [[bitcoin/bitcoin#30819 | core#30819]] Depends D18125 Test Plan: `ninja all check-all` Reviewers: #bitcoin_abc, Fabien Reviewed By: #bitcoin_abc, Fabien Subscribers: Fabien Differential Revision: https://reviews.bitcoinabc.org/D18126

Christewart · 2025-07-29T23:40:07Z

I believe this deserves a release note as it can break existing calls to dumptxoutset if you were previously omitting the type parameter?

This parameter can be omitted if a separate "rollback" named parameter is specified indicating the height or hash of a specific historical block.

This is the error i am seeing when upgrading to v29

Error -8: Invalid snapshot type "" specified. Please specify "rollback" or "latest"

ryanofsky · 2025-07-30T00:53:19Z

I believe this deserves a release note as it can break existing calls to dumptxoutset if you were previously omitting the type parameter?

Yes it would be good to add a release note saying that dumptxoutset can no longer be called with just a filename to produce a snapshot with the latest UTXO state. It now needs an additional type argument that can be set to "latest" to match previous behavior. It can also take other arguments and is now capable of rolling back the chain and producing UTXO snapshots at arbitrary heights or block hashes.

I had to look at #29553 discussion history to remind myself why we made this incompatible change. The original implementation of #29553 actually didn't require any new parameters but just changed dumptxoutset to roll back the chain to the hardcoded assumeutxo snapshot height by default. But martin pointed out #29553 (comment) that having a default behavior that rolls back the chain was dangerous because it could make the node unusable if the height was far back enough. And I pointed out that dumping the latest UTXO state by default wasn't great either because it would produce snapshots that couldn't be loaded. So we decided not to have any default and require requesting either the latest snapshot or rollback explicitly.

maflcko · 2025-07-30T05:46:39Z

29.0 is already tagged (archived), built, and released for a long time, so it isn't possible to change the release notes there now.

ryanofsky · 2025-07-30T11:26:21Z

29.0 is already tagged (archived), built, and released for a long time, so it isn't possible to change the release notes there now.

Thanks, this makes me think the nicest place to document compatibility information would probably be in RPC documentation itself. E.g. a note like "In version 29.0, non-optional type and height parameters were added. Prior behavior was equivalent to passing type=latest". This way if an incompatible change is made and your script stops working, you coul just check RPC help to figure out what to do instead of searching through release notes. I guess changes like this are not very common though so it might not be worth it.

Christewart · 2025-07-30T11:30:39Z

Alternatively I believe 29.1 is publishing RC's, maybe it could be documented in that release with a note about how this change wasn't documented in the 29.0 release? 🤷‍♂️

fjahr changed the title ~~rpd: dumptxoutset height parameter followups (29553)~~ rpd: dumptxoutset height parameter follow-ups (29553) Sep 4, 2024

fjahr changed the title ~~rpd: dumptxoutset height parameter follow-ups (29553)~~ rpc: dumptxoutset height parameter follow-ups (29553) Sep 4, 2024

DrahtBot added the RPC/REST/ZMQ label Sep 4, 2024

maflcko approved these changes Sep 4, 2024

View reviewed changes

fjahr added 2 commits September 4, 2024 15:49

rpc: Don't re-enable previously disabled network after dumptxoutset

598b9bb

Also fixes a typo in the RPC help text.

doc: Add -rpcclienttimeout=0 to loadtxoutset examples

c5eaae3

fjahr force-pushed the 2024-09-dumptxoutset-height-followup branch from e6812ff to 769dc84 Compare September 4, 2024 13:49

maflcko approved these changes Sep 4, 2024

View reviewed changes

fjahr force-pushed the 2024-09-dumptxoutset-height-followup branch from 769dc84 to 063a63c Compare September 4, 2024 14:01

rpc: Manage dumptxoutset rollback with RAII class

a3108a7

fjahr force-pushed the 2024-09-dumptxoutset-height-followup branch from 063a63c to a3108a7 Compare September 4, 2024 14:04

fjahr mentioned this pull request Sep 4, 2024

assumeutxo: Add dumptxoutset height param, remove shell scripts #29553

Merged

pablomartin4btc reviewed Sep 4, 2024

View reviewed changes

achow101 merged commit b8d2f58 into bitcoin:master Sep 4, 2024
15 of 16 checks passed

ryanofsky reviewed Sep 4, 2024

View reviewed changes

fjahr mentioned this pull request Sep 4, 2024

test: Add coverage for dumptxoutset failure robustness #30817

Merged

pablomartin4btc mentioned this pull request Sep 12, 2024

rpc: add relevant_blocks to scanblocks status #30713

Closed

Christewart mentioned this pull request Jul 30, 2025

2025 07 30 Add release note for dumptxoutset breaking change #33103

Closed

rpc: dumptxoutset height parameter follow-ups (29553) #30808

rpc: dumptxoutset height parameter follow-ups (29553) #30808

Uh oh!

Conversation

fjahr commented Sep 4, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DrahtBot commented Sep 4, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Coverage

Reviews

Uh oh!

fjahr commented Sep 4, 2024

Uh oh!

maflcko left a comment

Choose a reason for hiding this comment

Uh oh!

maflcko Sep 4, 2024

Choose a reason for hiding this comment

Uh oh!

fjahr Sep 4, 2024

Choose a reason for hiding this comment

Uh oh!

maflcko Sep 4, 2024

Choose a reason for hiding this comment

Uh oh!

fjahr Sep 4, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

maflcko left a comment

Choose a reason for hiding this comment

Uh oh!

maflcko Sep 4, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fjahr Sep 4, 2024

Choose a reason for hiding this comment

Uh oh!

maflcko commented Sep 4, 2024

Uh oh!

pablomartin4btc left a comment

Choose a reason for hiding this comment

Uh oh!

achow101 commented Sep 4, 2024

Uh oh!

Uh oh!

ryanofsky Sep 4, 2024

Choose a reason for hiding this comment

Uh oh!

fjahr commented Sep 4, 2024

Uh oh!

pablomartin4btc commented Sep 4, 2024

Uh oh!

Christewart commented Jul 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ryanofsky commented Jul 30, 2025

Uh oh!

maflcko commented Jul 30, 2025

Uh oh!

ryanofsky commented Jul 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Christewart commented Jul 30, 2025

Uh oh!

Uh oh!

fjahr commented Sep 4, 2024 •

edited

Loading

DrahtBot commented Sep 4, 2024 •

edited

Loading

maflcko Sep 4, 2024 •

edited

Loading

Christewart commented Jul 29, 2025 •

edited

Loading

ryanofsky commented Jul 30, 2025 •

edited

Loading