build: Make G_FUZZING constexpr, require -DBUILD_FOR_FUZZING=ON to fuzz #31191
Conversation
The fuzz binary is still compiled. This is required for the next commit.
|
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. Code Coverage & BenchmarksFor details see: https://corecheck.dev/bitcoin/bitcoin/pulls/31191. ReviewsSee the guideline for information on the review process.
If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update. ConflictsReviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first. |
fafbf8a to
fafbfdc
Compare
|
🚧 At least one of the CI tasks failed. HintsTry to run the tests locally, according to the documentation. However, a CI failure may still
Leave a comment here, if you need help tracking down a confusing failure. |
fafbfdc to
fafbf8a
Compare
|
Tested ACK fafbf8a |
|
Tested ACK fafbf8a This solves #31178
It also seems to reasonably solve #30950 & #31057 which were the motivation of #31093 which introduced the regression. I don't have a view on this, but just want to document that this PR does not address the described use case of building for fuzzing without building the fuzz binary (#31057 (comment)) which is what motivated making
|
Can you explain this a bit better? This pull request does not change anything about being able to build with these options and be able to see if the fuzz binary build breaks. Specifically,
Maybe I am misunderstanding, so it would help to share exact steps to reproduce of the use case that you are claiming is not addressed. |
|
Code review ACK fafbf8a but approach -0, because this approach means libraries built for fuzz testing do not function correctly if used in a release, and libraries built for releases are mostly useless for fuzz testing. So I would like to at least consider other solutions to this problem even if we go with this one. #31178 makes it pretty clear that if we want to be able to write Also I don't really buy the idea that if you put an So I think a better alternative to this PR might just be to provide a better alternative to Assume(). I think most uses of Assume() are fine as they are, but a few really are on hot paths, and I tried adding a simple counter to identify them in the linearizeoptimallyexample11 benchmark. If I disable these checks, the speedup is equivalent to this PR without drawbacks of this PR. The change I would propose based on this is: diff
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -225,6 +225,7 @@ if(BUILD_FOR_FUZZING)
set(BUILD_GUI_TESTS OFF)
set(BUILD_BENCH OFF)
set(BUILD_FUZZ_BINARY ON)
+ target_compile_definitions(core_interface INTERFACE ENABLE_SLOWCHECK)
endif()
include(ProcessConfigurations)
--- a/cmake/module/ProcessConfigurations.cmake
+++ b/cmake/module/ProcessConfigurations.cmake
@@ -126,6 +126,7 @@ target_compile_definitions(core_interface_debug INTERFACE
DEBUG_LOCKCONTENTION
RPC_DOC_CHECK
ABORT_ON_FAILED_ASSUME
+ ENABLE_SLOWCHECK
)
# We leave assertions on.
if(MSVC)
--- a/src/cluster_linearize.h
+++ b/src/cluster_linearize.h
@@ -337,7 +337,7 @@ struct SetInfo
/** Add a transaction to this SetInfo (which must not yet be in it). */
void Set(const DepGraph<SetType>& depgraph, ClusterIndex pos) noexcept
{
- Assume(!transactions[pos]);
+ SLOWCHECK(!transactions[pos]);
transactions.Set(pos);
feerate += depgraph.FeeRate(pos);
}
--- a/src/util/bitset.h
+++ b/src/util/bitset.h
@@ -281,7 +281,7 @@ class MultiIntBitSet
/** Progress to the next 1 bit (only if != IteratorEnd). */
constexpr Iterator& operator++() noexcept
{
- Assume(m_idx < N);
+ SLOWCHECK(m_idx < N);
m_val &= m_val - I{1U};
if (m_val == 0) {
while (true) {
@@ -301,7 +301,7 @@ class MultiIntBitSet
/** Get the current bit position (only if != IteratorEnd). */
constexpr unsigned operator*() const noexcept
{
- Assume(m_idx < N);
+ SLOWCHECK(m_idx < N);
return m_pos;
}
};
@@ -316,7 +316,7 @@ public:
/** Set a bit to 1. */
void constexpr Set(unsigned pos) noexcept
{
- Assume(pos < MAX_SIZE);
+ SLOWCHECK(pos < MAX_SIZE);
m_val[pos / LIMB_BITS] |= I{1U} << (pos % LIMB_BITS);
}
/** Set a bit to the specified value. */
--- a/src/util/check.h
+++ b/src/util/check.h
@@ -81,6 +81,12 @@ constexpr T&& inline_assertion_check(LIFETIMEBOUND T&& val, [[maybe_unused]] con
/**
* Assume is the identity function.
*
+ * Assume() should be used instead of Assert() in cases where if the condition
+ * is not true, it indicates there is a bug, and you want to handle the bug by
+ * crashing in debug builds and in fuzz tests, but want to avoid unnecessary
+ * crashes in release builds and handle it with logging, warning, or other
+ * fallback behavior.
+ *
* - Should be used to run non-fatal checks. In debug builds it behaves like
* Assert()/assert() to notify developers and testers about non-fatal errors.
* In production it doesn't warn or log anything.
@@ -90,6 +96,24 @@ constexpr T&& inline_assertion_check(LIFETIMEBOUND T&& val, [[maybe_unused]] con
*/
#define Assume(val) inline_assertion_check<false>(val, __FILE__, __LINE__, __func__, #val)
+/**
+ * SLOWCHECK() can be used to perform checks that are enabled in debug and fuzz
+ * builds but are skipped in release builds. It is meant to be used for checks
+ * that are slow, either because they occur in hot code paths, or because the
+ * checks themselves are expensive. Assert() should be preferred for critical
+ * checks which are not in performance-sensitive code.
+ *
+ * SLOWCHECK() is basically equivalent to the traditional C/C++ assert() macro
+ * For historic reasons, Bitcoin Core cannot be compiled with NDEBUG so there is
+ * no way to skip assert() checks in release builds, and SLOWCHECK() restores
+ * this lost functionality.
+ */
+#ifdef ENABLE_SLOWCHECK
+#define SLOWCHECK(val) assert(val);
+#else
+#define SLOWCHECK(val) assert(1 || (val));
+#endif
+
/**
* NONFATAL_UNREACHABLE() is a macro that is used to mark unreachable code. It throws a NonFatalCheckError.
*/
--- a/src/util/vecdeque.h
+++ b/src/util/vecdeque.h
@@ -74,7 +74,7 @@ class VecDeque
/** What index in the buffer does logical entry number pos have? */
size_t BufferIndex(size_t pos) const noexcept
{
- Assume(pos < m_capacity);
+ SLOWCHECK(pos < m_capacity);
// The expression below is used instead of the more obvious (pos + m_offset >= m_capacity),
// because the addition there could in theory overflow with very large deques.
if (pos >= m_capacity - m_offset) {I would curious to know from @sipa and others if they think this approach makes sense and does not add too much of a burden. Specifically results I saw testing this with the linearizeoptimallyexample11 benchmark were 1.42 op/sec on master, 1.77 op/sec reverting 9f243cd (#31093), 1.77 op/sec cherry-picking fafbf8a from this PR, and 1.83 op/sec with the proposed change. |
I don't think it is a supported use-case to take libraries from one build (with different build options) and drop them into another build (with different build options) and expect it to work, or be a supported use-case. This has also been the case up until two weeks ago, before commit 9f243cd. Also your
I don't think I've claimed this. It has always been the assumption that
I agree and I don't think I've claimed otherwise. I just don't see the use-case to mix-match libraries from different builds and I think this change makes sense even if you go ahead with your
It may be best to submit this as a separate pull request. I think it is fine if this one keeps sitting for a few more weeks and it can be closed/merged, possibly depending on the result of your SLOWCHECK pull request. |
Agree it should not be a generally supported use case. And I do think it's nice to have a BUILD_FOR_FUZZING option that enables the best options for builds specifically doing fuzzing. But for normal builds, I don't think the whole codebase should need to be recompiled with different options just to get a useful fuzz binary. And I don't think it is good for BUILD_FOR_FUZZING option to create strange libraries that seem functional but skip proof of work checks. I did ACK this PR in case we think these problems are worth having so
Yes sorry, the comments above about Assume() usage were meant for @sipa who wrote "I have been using Assume in many places assuming it'll be optimized out entirely in production code" and similar things in the other issues.
Yes, will do if it seems like something that is helpful. |
I don't think this conceptual issue is addressed by your alternative |
Didn't think of this, that makes sense to me. I also may have misunderstood or misrepresented the use-case/concern I tried to describe, but it's not my own and I don't have any view on it, so it seems silly that I even mentioned it. I still ACK fafbf8a for fixing the regression measured in #31178. |
|
The use-case which I think this breaks and should be supported is the ability to use fuzzing in a normal build instead of a dedicated build. As an analogy, if you want use a debugger, the best place to use it is in a dedicated debug build, but you should also be able to generate debug symbols and use debuggers with some limitations in release builds. Or if you want to check memory safety, the best way to do it may be to run MSan or valgrind in dedicated builds, but it should also be possible to use ASan in normal builds. Similarly, if you want to run fuzz tests, the best way to run them is in a dedicated build with -DBUILD_FOR_FUZZING=ON, but I think you should also be able to run fuzz tests in a normal build with other functioning executables. This flexibility lets you have a single build supporting most developer tools, even if it is not the ideal build for running every tool. I think it lowers the barrier to start trying new tools if using them only requires toggling build options not creating entirely new build configurations. AFAICT before #31093, libraries with BUILD_FOR_FUZZING did not function normally because they skipped proof of work checks, and libraries without BUILD_FOR_FUZZING were not very useful for fuzzing because they skipped most Assume() checks. #31093 fixed both of those issues and this PR breaks them again. Maybe this is acceptable, but doesn't seem good, and an alternate approach like the one in #31191 (comment) might be better. I'd be happy to open a PR with that approach it makes sense, or to learn that I am wrong and it doesn't make sense, or to get feedback that it is not a good solution for other reasons. I also think this PR is ok, but I don't like the tradeoff it is making, so I'm just pointing out the disadvantages I see here. |
I see what you're saying about tools being more accessible if they're able to be used within a single build. However, I do think with cmake it's straightforward to maintain separate builds for different purposes. And fuzz testing might be involved enough to warrant its own build. I don't see many advantages to fuzzing in a normal build that isn't optimized for it. If we're still able to verify that the fuzz binary builds properly and run pre-fuzz checks, as mentioned in #31191 (comment), then to me it feels reasonable to require BUILD_FOR_FUZZING when we intend to do some actual fuzz testing. I also think it's cleaner to isolate fuzz-specific code paths (like skipping the PoW check) to a dedicated build. Of course, I'm open to reviewing other potential solutions. |
|
Went ahead and merged this since it has enough ACKs, and it sounds like the concerns I have about drawbacks of this PR don't seem to be shared by the other reviewers. I think it would be nice if fuzzing could be turned on without turning every else off, and this approach will probably make me a little less likely to start writing fuzz tests, and prefer writing unit tests instead for convenience. But status after this PR isn't worse than the status before #31093, and this approach can always be revisited later. |
I understand your concerns, but did you actually encounter them in reality? Generally, for writing fuzz tests, you'll want to go with a fuzz engine (and ideally sanitizers). Otherwise, generating fuzz inputs and exploring the fuzz target reachable code will be harder. Once you link a fuzz engine, you'll have to go with a separate fuzz build anyway. My understanding is that the only real-world developer use-case this affects is running the fuzz target(s) on a fixed set of fuzz input(s). However, this is used rarely or even not at all recently by developers (I think), and was regularly only done in CI. |
Yes, I've never written or run a fuzz test because they require separate setup and because I'm reasonably happy writing unit tests. If I could run fuzz tests in my default build, I'd be more likely to write fuzz tests as an alternative to unit tests. I understand fuzz tests are not always an alternative to unit tests, but I've written a lot of unit tests where I'm just handcrafting inputs to test edge cases, and then checking invariants, and probably fuzz tests could save effort and do a better job. Not just letting developers add -fsanitize=fuzzer,address to their flags and run an extra binary seems like adding an unnecessary obstacle to starting to use the fuzzer. Maybe the obstacle has to exist for reasons I'm too uninformed to know about, but from the outside treating fuzzing compiler options differently than other compiler options seems confusing and unnecessarily complex. |
|
@ryanofsky As long as we're talking about libfuzzer-based fuzzing (which is what we mostly use in this project, though not exclusively), the
These pretty much inherently require a separate build: you can't do fuzzing without (1), because the mutations that libfuzzer makes to the corpus are based on coverage information. And in the other direction, (2) is incompatible with a normal build because it adds its own main function. Now, in theory there is FWIW, for me personally, the overhead of needing a separate configure for fuzzing has reduced significantly since the introduction of cmake, as I just have a separate build directory for it. |
I see. I think what you are asking for could be achieved by reverting this pull request, possibly hiding the performance impact behind |
|
Thanks for clarifications and ideas. Iit sounds like doing what I want would require |
…158303fe2 48158303fe2 kernel: Add pure kernel bitcoin-chainstate bf80d2f5009 kernel: Add block index utility functions to C header a6ab5345e3b kernel: Add function to read block undo data from disk to C header 845b824d6c7 kernel: Add functions to read block from disk to C header 9324c8c4f67 kernel: Add function for copying block data to C header 368fc93fd80 kernel: Add functions for the block validation state to C header eb6e25ac007 kernel: Add validation interface to C header cdce4484005 kernel: Add interrupt function to C header 7e47ec78768 kernel: Add import blocks function to C header 2b803d50747 kernel: Add chainstate load options for in-memory dbs in C header ea92eb13c4a kernel: Add options for reindexing in C header 8254f2035a7 kernel: Add block validation to C header ad7b880346e Kernel: Add chainstate loading to kernel C header 583820c4487 kernel: Add chainstate manager object to C header ec137a086a0 kernel: Add notifications context option to C header 62a89689266 kerenl: Add chain params context option to C header bb482dcbd30 kernel: Add kernel library context object d114ccfdf8a kernel: Add logging to kernel library C header 44c65c46c43 kernel: Introduce initial kernel C header API 69c03134440 Merge bitcoin/bitcoin#31269: validation: Remove RECENT_CONSENSUS_CHANGE validation result 42282592943 Merge bitcoin/bitcoin#31000: bench: add support for custom data directory 36f5effa178 Merge bitcoin/bitcoin#31235: addrman: cap the `max_pct` to not exceed the maximum number of addresses 98ad249b69f Merge bitcoin/bitcoin#31277: doc: mention `descriptorprocesspsbt` in psbt.md b0222bbb494 Merge bitcoin/bitcoin#30239: Ephemeral Dust 1dda1892b6b Merge bitcoin/bitcoin#31037: test: enhance p2p_orphan_handling 5c2e291060c bench: Add basic CheckEphemeralSpends benchmark 3f6559fa581 Add release note for ephemeral dust 71a6ab4b33d test: unit test for CheckEphemeralSpends 21d28b2f362 fuzz: add ephemeral_package_eval harness 127719f516a test: Add CheckMempoolEphemeralInvariants e2e30e89ba4 functional test: Add ephemeral dust tests 4e68f901390 rpc: disallow in-mempool prioritisation of dusty tx e1d3e81ab4d policy: Allow dust in transactions, spent in-mempool 04b2714fbbc functional test: Add new -dustrelayfee=0 test case ebb6cd82baf doc: mention `descriptorprocesspsbt` in psbt.md 2b33322169b Merge bitcoin/bitcoin#31249: test: Add combinerawtransaction test to rpc_createmultisig 3fb6229dcfd Merge bitcoin/bitcoin#31271: doc: correct typos fa66e0887ca bench: add support for custom data directory ad9c2cceda9 test, bench: specialize working directory name 9c5775c331e addrman: cap the `max_pct` to not exceed the maximum number of addresses 8d340be9247 Merge bitcoin/bitcoin#31181: cmake: Revamp `FindLibevent` module 9a8e5adb161 Merge bitcoin/bitcoin#31267: refactor: Drop deprecated space in operator""_mst 726cbee9553 doc: correct typos 9fdfb73ca84 doc: fix typos af6088701a2 Merge bitcoin/bitcoin#31237: doc: Add missing 'blank=true' option in offline-signing-tutorial.md 7a526653022 Merge bitcoin/bitcoin#31239: test: clarify log messages when handling SOCKS5 proxy connections 900b17239fb Merge bitcoin/bitcoin#31259: doc: Fix missing comma in JSON example in REST-interface.md faf21625652 refactor: Drop deprecated space in operator""_mst c889890e4a3 Merge bitcoin/bitcoin#31264: doc: Fixup bitcoin-wallet manpage chain selection args 0f6d20e43f2 Merge bitcoin/bitcoin#31163: scripted-diff: get rid of remaining "command" terminology in protocol.{h,cpp} 5acd5e7f874 Merge bitcoin/bitcoin#31257: ci: make ctest stop on failure 19f277711eb Merge bitcoin/bitcoin#26593: tracing: Only prepare tracepoint arguments when actually tracing e80e4c6ff91 validation: Remove RECENT_CONSENSUS_CHANGE validation result fa729ab4a27 doc: Fixup bitcoin-wallet manpage chain selection args 5e3b444022c doc: Fix missing comma in JSON example in REST-interface.md 0903ce8dbc2 Merge bitcoin/bitcoin#30592: Remove mempoolfullrbf f842d0801e1 Merge bitcoin/bitcoin#29686: Update manpage descriptions 36a22e56833 ci: make ctest stop on failure 83fab3212c9 test: Add combinerawtransaction test to rpc_createmultisig 018e5fcc462 Merge bitcoin/bitcoin#31190: TxDownloadManager followups 3a5f6027e16 Merge bitcoin/bitcoin#31171: depends: Specify CMake generator explicitly 99d9a093cf6 test: clarify log messages when handling SOCKS5 proxy connections c9e67e214f0 Merge bitcoin/bitcoin#31238: fuzz: Limit wallet_notifications iterations 564238aabf1 Merge bitcoin/bitcoin#31164: net: Use actual memory size in receive buffer accounting fa461d7a43a fuzz: Limit wallet_notifications iterations ec375de39ff doc: Add missing 'blank=true' option in offline-signing-tutorial.md 5a96767e3f5 depends, libevent: Do not install *.pc files and remove patches for them ffda355b5a2 cmake, refactor: Move `HAVE_EVHTTP_...` to `libevent` interface b619bdc3303 cmake: Revamp `FindLibevent` module 2c90f8e08c4 Merge bitcoin/bitcoin#31232: ci: `add second_deadlock_stack=1` to TSAN options 5dc94d13d41 fuzz fix: assert MAX_PEER_TX_ANNOUNCEMENTS is not exceeded 45e2f8f87d8 Merge bitcoin/bitcoin#31173: cmake: Add `FindQRencode` module and enable `libqrencode` package for MSVC 80cb630bd94 Merge bitcoin/bitcoin#31216: Update secp256k1 subtree to v0.6.0 5161c2618cd ci: add second_deadlock_stack=1 to TSAN options 85224f92d52 Merge bitcoin/bitcoin#30811: build: Unify `-logsourcelocations` format 9719d373dc2 Merge bitcoin/bitcoin#30634: ci: Use clang-19 from apt.llvm.org 97235c446e9 build: Disable secp256k1 musig module 9e5089dbb02 build, msvc: Enable `libqrencode` vcpkg package 30089b0cb61 cmake: Add `FindQRencode` module 65b19419366 Merge bitcoin/bitcoin#31186: msvc: Update vcpkg manifest d3388720837 Merge bitcoin/bitcoin#31206: doc: Use relative hyperlinks in release-process.md ffc05fca6f7 Merge bitcoin/bitcoin#31220: doc: Fix word order in developer-notes.md 9f2c8287a24 Merge bitcoin/bitcoin#31192: depends, doc: List packages required to build `qt` package separately 03cff2c1421 Merge bitcoin/bitcoin#31191: build: Make G_FUZZING constexpr, require -DBUILD_FOR_FUZZING=ON to fuzz 44939e5de1b doc: Fix word order in developer-notes.md b934954ad10 Merge bitcoin/bitcoin#30670: doc: Extend developer-notes with file-name-only debugging fix 05aebe3790f Merge bitcoin/bitcoin#30930: netinfo: add peer services column and outbound-only option 0ba680d41b4 Update secp256k1 subtree to v0.6.0 2d46a89386d Squashed 'src/secp256k1/' changes from 2f2ccc46954..0cdc758a563 d22a234ed27 net: Use actual memory size in receive buffer accounting 047b5e2af1f streams: add DataStream::GetMemoryUsage c3a6722f34a net: Use DynamicUsage(m_type) in CSerializedNetMsg::GetMemoryUsage c6594c0b142 memusage: Add DynamicUsage for std::string 7596282a556 memusage: Allow counting usage of vectors with different allocators 6463117a292 Merge bitcoin/bitcoin#31208: doc: archive release notes for v27.2 788c1324f3d build: Unify `-logsourcelocations` format 4747f030956 depends, doc: List packages required to build `qt` package separately 1a05c86ae47 doc: archive release notes for v27.2 9f71cff6ab3 doc: Use relative hyperlinks in release-process.md f1bcf3edc50 Merge bitcoin/bitcoin#31139: test: added test to assert TX decode rpc error on submitpackage rpc 975b115e1a2 Merge bitcoin/bitcoin#31198: init: warn, don't error, when '-upnp' is set 4a0251c05dd Merge bitcoin/bitcoin#31187: ci: Do not error on unused-member-function in test each commit e001dc3dc6e Merge bitcoin/bitcoin#31203: fuzz: fix `implicit-integer-sign-change` in wallet_create_transaction 5a26cf7773e fuzz: fix `implicit-integer-sign-change` in wallet_create_transaction a1b3ccae4be init: warn, don't error, when '-upnp' is set c189eec848e doc: release note for mempoolrullrbf removal d47297c6aab rpc: Mark fullrbf and bip125-replaceable as deprecated 04a5dcee8ab docs: remove requirement to signal bip125 fafbf8acf41 Make G_FUZZING constexpr, require -DBUILD_FOR_FUZZING=ON to execute a fuzz target fae3cf0ffa6 ci: Temporarily disable macOS/Windows fuzz step f6577b71741 build, msvc: Update vcpkg manifest baseline 16e16013bfa build, msvc: Document `libevent` version pinning ec47cd2b508 build, msvc: Drop no longer needed `liblzma` version pinning 9a0734df5f1 build, msvc: Reorder keys in `vcpkg.json` 8351562bec6 [fuzz] allow negative time jumps in txdownloadman_impl 917ab810d93 [doc] comment fixups from n30110 f07a533dfcb Merge bitcoin/bitcoin#24214: Fix unsigned integer overflows in interpreter 62516105536 Merge bitcoin/bitcoin#31015: build: have "make test" depend on "make all" 4a31f8ccc9d Merge bitcoin/bitcoin#31156: test: Don't enforce BIP94 on regtest unless specified by arg 02be3dced71 Merge bitcoin/bitcoin#31166: key: clear out secret data in `DecodeExtKey` 54d07dd37d5 ci: Do not error on unused-member-function in test each commit 47f50c7af55 doc: add bitcoin-qt man description 40b82e3ab0a doc: add bitcoin-util man description a7bf80f3a2d doc: add bitcoin-tx man description 3f9a5168323 doc: add bitcoin-wallet man description d8c0bb23ef8 doc: add bitcoin-cli man description 09abccfa772 doc: add bitcoind man description 97b790e844a Merge bitcoin/bitcoin#29420: test: extend the SOCKS5 Python proxy to actually connect to a destination 6b73eb9a1a2 Merge bitcoin/bitcoin#31064: init: Correct coins db cache size setting 27d12cf17f2 Merge bitcoin/bitcoin#31043: rpc: getorphantxs follow-up 7b66815b16b Merge bitcoin/bitcoin#30110: refactor: TxDownloadManager + fuzzing dc97e7f6dba Merge bitcoin/bitcoin#30903: cmake: Add `FindZeroMQ` module 1b0b9b4c787 Extend possible debugging fixes with file-name-only da10e0bab4a Merge bitcoin/bitcoin#30942: test: Remove dead code from interface_zmq test 111a23d9b36 Remove -mempoolfullrbf option e96ffa98b04 Merge bitcoin/bitcoin#31142: test: fix intermittent failure in p2p_seednode.py, don't connect to random IPs 54c4b09f083 Merge bitcoin/bitcoin#31042: build: Rename `PACKAGE_*` variables to `CLIENT_*` e60cecc8115 doc: add release note for 31156 fc7dfb3df5b test: Don't enforce BIP94 on regtest unless specified by arg fabe90c8242 ci: Use clang-19 from apt.llvm.org 0de3e96e333 tracing: use bitcoind pid in bcc tracing examples 411c6cfc6c2 tracing: only prepare tracepoint args if attached d524c1ec066 tracing: dedup TRACE macros & rename to TRACEPOINT 70713303b63 scripted-diff: Rename `PACKAGE_*` variables to `CLIENT_*` 332655cb52c build: Rename `PACKAGE_*` variables to `CLIENT_*` e6e29e3c94c scripted-diff: Clarify "user agent" variable name e2ba8236715 depends: Specify CMake generator explicitly 1c7ca6e64de Merge bitcoin/bitcoin#31093: Introduce `g_fuzzing` global for fuzzing checks 6e21dedbf2b Merge bitcoin/bitcoin#31130: Drop miniupnp dependency d7fd766feb2 test: added test to assert TX decode rpc error on submitpackage rpc 559a8dd9c0a key: clear out secret data in `DecodeExtKey` 4120c7543ee scripted-diff: get rid of remaining "command" terminology in protocol.{h,cpp} 2a52718d734 Merge bitcoin/bitcoin#31152: functional test: Additional package evaluation coverage 9de9c858d5a test: enhance p2p_orphan_handling 33af14b62e4 test: reduce assert_debug_log reliance 0ea84bc362f test: explicitly check boolean verbosity is disallowed 7a2e6b68cd9 doc: add rpc guidance for boolean verbosity avoidance 698f302df8b rpc: disallow boolean verbosity in getorphantxs 63f5e6ec795 test: add entry and expiration time checks 808a708107e rpc: add entry time to getorphantxs 56bf3027144 refactor: rename rpc_getorphantxs to rpc_orphans 7824f6b0770 test: check that getorphantxs is hidden ac68fcca701 rpc: disallow undefined verbosity in getorphantxs 25dacae9c7f Merge bitcoin/bitcoin#31040: test: Assert that when we add the max orphan amount that we cannot add anymore and that a random orphan gets dropped 40e5f26a3ff mapport: remove dead code in DispatchMapPort 38fdf7c1fb1 mapport: drop outdated comments 915640e191b depends: zeromq: don't install .pc files and remove patches for them 6b8a74463b5 cmake: Add `FindZeroMQ` module 9a7206a34e3 Merge bitcoin/bitcoin#29536: fuzz: fuzz connman with non-empty addrman + ASMap d4abaf8c9d9 Merge bitcoin/bitcoin#29608: optimization: Preallocate addresses in GetAddr based on nNodes b7b24352906 doc: add release note for #31130 1b6dec98da3 depends: drop miniupnpc 953533d0214 doc: remove mentions of UPnP 94ad614482f ci: remove UPnP options f32c34d0c3d functional test: Additional package evaluation coverage 87532fe5585 netinfo: allow setting an outbound-only peer list 9f243cd7fa6 Introduce `g_fuzzing` global for fuzzing checks b95adf057a4 Merge bitcoin/bitcoin#31150: util: Treat Assume as Assert when evaluating at compile-time 8f24e492e20 Merge bitcoin/bitcoin#29991: depends: sqlite 3.46.1 2ef5004f78c Merge bitcoin/bitcoin#31146: ci: Temporary workaround for old CCACHE_DIR cirrus env 8c12fe828de Merge bitcoin/bitcoin#29936: fuzz: wallet: add target for `CreateTransaction` 5c299ecafe6 test: Assert that when we add the max orphan amount that we cannot add anymore and that a random orphan gets dropped 0f4bc635854 [fuzz] txdownloadman and txdownload_impl 699643f23a1 [unit test] MempoolRejectedTx fa584cbe727 [p2p] add TxDownloadOptions bool to make TxRequestTracker deterministic f803c8ce8dd [p2p] filter 1p1c for child txid in recent rejects 5269d57e6d7 [p2p] don't process orphan if in recent rejects 2266eba43a9 [p2p] don't find 1p1cs for reconsiderable txns that are AlreadyHaveTx fa7027d0fc1 [refactor] add CheckIsEmpty and GetOrphanTransactions, remove access to TxDownloadMan internals 969b07237b9 [refactor] wrap {Have,Get}TxToReconsider in txdownload f150fb94e7d [refactor] make AlreadyHaveTx and Find1P1CPackage private to TxDownloadImpl 1e08195135b [refactor] move new tx logic to txdownload 257568eab5b [refactor] move invalid package processing to TxDownload c4ce0c1218d [refactor] move invalid tx processing to TxDownload c6b21749ca0 [refactor] move valid tx processing to TxDownload a8cf3b6e845 [refactor] move Find1P1CPackage to txdownload f497414ce76 [refactor] put peerman tasks at the end of ProcessInvalidTx 6797bc42a76 [p2p] restrict RecursiveDynamicUsage of orphans added to vExtraTxnForCompact 798cc8f5aac [refactor] move Find1P1CPackage into ProcessInvalidTx 416fbc952b2 [refactor] move new orphan handling to ProcessInvalidTx c8e67b9169b [refactor] move ProcessInvalidTx and ProcessValidTx definitions down 3a41926d1b5 [refactor] move notfound processing to txdownload 042a97ce7fc [refactor] move tx inv/getdata handling to txdownload 58e09f244b4 [p2p] don't log tx invs when in IBD 288865338f5 [refactor] rename maybe_add_extra_compact_tx to first_time_failure f48d36cd97e [refactor] move peer (dis)connection logic to TxDownload f61d9e4b4b8 [refactor] move AlreadyHaveTx to TxDownload 84e4ef843db [txdownload] add read-only reference to mempool af918349de5 [refactor] move ValidationInterface functions to TxDownloadManager f6c860efb12 [doc] fix typo in m_lazy_recent_confirmed_transactions doc 5f9004e1550 [refactor] add TxDownloadManager wrapping TxOrphanage, TxRequestTracker, and bloom filters 947f2925d55 Merge bitcoin/bitcoin#31124: util: Remove RandAddSeedPerfmon 7640cfdd624 Merge bitcoin/bitcoin#31118: doc: replace `-?` with `-h` and `-help` 74fb19317ae Merge bitcoin/bitcoin#30849: refactor: migrate `bool GetCoin` to return `optional<Coin>` c16e909b3e2 Merge bitcoin/bitcoin#28574: wallet: optimize migration process, batch db transactions a9598e5eaab build: drop miniupnpc dependency a5fcfb7385c interfaces: remove now unused 'use_upnp' arg from 'mapPort' 038bbe7b200 daemon: remove UPnP support 844770b05eb qt: remove UPnP settings dd92911732d Merge bitcoin/bitcoin#31148: ci: display logs of failed unit tests automatically fa69a5f4b76 util: Treat Assume as Assert when evaluating at compile-time 0c79c343a9f Merge bitcoin/bitcoin#31147: cmake, qt, test: Remove problematic code 8523d8c0fc8 ci: display logs of failed tests automatically 2f40e453ccd Merge bitcoin/bitcoin#29450: build: replace custom `MAC_OSX` macro with existing `__APPLE__` cb7c5ca824e Add gdb and lldb links to debugging troubleshooting 6c6b2442eda build: Replace MAC_OSX macro with existing __APPLE__ fb46d57d4e7 cmake, qt, test: Remove problematic code fa9747a8961 ci: Temporary workaround for old CCACHE_DIR cirrus env 6c9fe7b73ea test: Prevent connection attempts to random IPs in p2p_seednodes.py bb97b1ffa9f test: fix intermittent timeout in p2p_seednodes.py 57529ac4dbb test: set P2PConnection.p2p_connected_to_node in peer_connect_helper() 22cd0e888c7 test: support WTX INVs from P2PDataStore and fix a comment ebe42c00aa4 test: extend the SOCKS5 Python proxy to actually connect to a destination 9bb92c0e7ff util: Remove RandAddSeedPerfmon c98fc36d094 wallet: migration, consolidate external wallets db writes 7c9076a2d2e wallet: migration, consolidate main wallet db writes 9ef20e86d7f wallet: provide WalletBatch to 'SetupDescriptorScriptPubKeyMans' 34bf0795fc0 wallet: refactor ApplyMigrationData to return util::Result<void> aacaaaa0d3a wallet: provide WalletBatch to 'RemoveTxs' 57249ff6697 wallet: introduce active db txn listeners 91e065ec175 wallet: remove post-migration signals connection 055c0532fc8 wallet: provide WalletBatch to 'DeleteRecords' 122d103ca22 wallet: introduce 'SetWalletFlagWithDB' 6052c7891dc wallet: decouple default descriptors creation from external signer setup f2541d09e13 wallet: batch MigrateToDescriptor() db transactions 66c9936455f bench: add coverage for wallet migration process 33a28e252a7 Change default help arg to `-help` and mention `-h` and `-?` as alternatives f0130ab1a1e doc: replace `-?` with `-h` for bench_bitcoin help 681ebcceca7 netinfo: rename and hoist max level constant to use in top-level help e7d307ce8cf netinfo: clarify relaytxes and addr_relay_enabled help docs eef2a9d4062 netinfo: add peer services column 3a4a788ee0d init: Correct coins db cache size setting 2957ca96119 build: have "make test" depend on "make all" bbbbaa0d9ac Fix unsigned integer overflows in interpreter c4dc81f9c69 test: Remove dead code from interface_zmq c495731a316 fuzz: wallet: add target for `CreateTransaction` 3db68e29ec6 wallet: move `ImportDescriptors`/`FuzzedWallet` to util 552cae243a1 fuzz: cover `ASMapHealthCheck` in connman target 33b0f3ae966 fuzz: use `ConsumeNetGroupManager` in connman target 18c8a0945bd fuzz: move `ConsumeNetGroupManager` to util fe624631aeb fuzz: fuzz `connman` with a non-empty addrman 0a12cff2a8e fuzz: move `AddrManDeterministic` to util 4feaa287284 refactor: Rely on returned value of GetCoin instead of parameter 46dfbf169b4 refactor: Return optional of Coin in GetCoin e31bfb26c21 refactor: Remove unrealistic simulation state ba621ffb9cb test: improve debug log message from P2PConnection::connection_made() def6dd0c597 depends: sqlite 3.46.1 66082ca3488 Preallocate addresses in GetAddr based on nNodes REVERT: 1047757ea3b kernel: Add pure kernel bitcoin-chainstate REVERT: c568fdf75fd kernel: Add block index utility functions to C header REVERT: 0f1da1dcba5 kernel: Add function to read block undo data from disk to C header REVERT: 45af559c9f6 kernel: Add functions to read block from disk to C header REVERT: 2a7f8a8240c kernel: Add function for copying block data to C header REVERT: b19f5336c03 kernel: Add functions for the block validation state to C header REVERT: 9c0ffa913f4 kernel: Add validation interface to C header REVERT: a93318c6152 kernel: Add interrupt function to C header REVERT: 51053f33720 kernel: Add import blocks function to C header REVERT: 6b0ada2af42 kernel: Add chainstate load options for in-memory dbs in C header REVERT: 34427bfa9c7 kernel: Add options for reindexing in C header REVERT: ca57311c969 kernel: Add block validation to C header REVERT: 44156d84838 Kernel: Add chainstate loading to kernel C header REVERT: 2cee46cdcc1 kernel: Add chainstate manager object to C header REVERT: 7102c7ae45e kernel: Add notifications context option to C header REVERT: ed628a2a3c4 kerenl: Add chain params context option to C header REVERT: 27643297ff7 kernel: Add kernel library context object REVERT: 2ba22cf3f90 kernel: Add logging to kernel library C header REVERT: 873874c03e9 kernel: Introduce initial kernel C header API git-subtree-dir: libbitcoinkernel-sys/bitcoin git-subtree-split: 48158303fe276cb2f8fbc53ff31a4162d8f55c84
g_fuzzingis used insideAssumeat runtime, causing significant overhead in hot paths. See #31178One could simply remove the
g_fuzzingcheck from theAssume, but this would make fuzzing a bit less useful. Also, it would be unclear ifg_fuzzingadds a runtime overhead in other code paths today or in the future.Fix all issues by making
G_FUZZINGequal to the build optionBUILD_FOR_FUZZING, and for consistency in fuzzing, require it to be set when executing any fuzz target.Fixes #31178
Temporarily this drops fuzzing from two CI tasks, but they can be re-added in a follow-up with something like #31073