tracing: Only prepare tracepoint arguments when actually tracing #26593

0xB10C · 2022-11-28T20:50:26Z

Currently, if the tracepoints are compiled (e.g. in depends and release builds), we always prepare the tracepoint arguments regardless of the tracepoints being used or not. We made sure that the argument preparation is as cheap as possible, but we can almost completely eliminate any overhead for users not interested in the tracepoints (the vast majority), by gating the tracepoint argument preparation with an if(something is attached to this tracepoint). To achieve this, we use the optional semaphore feature provided by SystemTap.

The first commit simplifies and deduplicates our tracepoint macros from 13 TRACEx macros to a single TRACEPOINT macro. This makes them easier to use and also avoids more duplicate macro definitions in the second commit.

The Linux tracing tools I'm aware of (bcc, bpftrace, libbpf, and systemtap) all support the semaphore gating feature. Thus, all existing tracepoints and their argument preparation is gated in the second commit. For details, please refer to the commit messages and the updated documentation in doc/tracing.md.

Also adding unit tests that include all tracepoint macros to make sure there are no compiler problems with them (e.g. some varadiac extension not supported).

Reviewers might want to check:

Do the tracepoints still work for you? Do the examples in contrib/tracing/ run on your system (as bpftrace frequently breaks on every new version, please test master too if it should't work for you)? Do the CI interface tests still pass?
Is the new documentation clear?
The TRACEPOINT_SEMAPHORE(event, context) macros places global variables in our global namespace. Is this something we strictly want to avoid or maybe move to all TRACEPOINT_SEMAPHOREs to a separate .cpp file or even namespace? I like having the TRACEPOINT_SEMAPHORE() in same file as the TRACEPOINT(), but open for suggestion on alternative approaches.
Are newly added tracepoints in the unit tests visible when using readelf -n build/src/test/test_bitcoin? You can run the new unit tests with ./build/src/test/test_bitcoin --run_test=util_trace_tests* --log_level=all.

Two of the added unit tests demonstrate that we are only processing the tracepoint arguments when attached by having a test-failure condition in the tracepoint argument preparation. The following bpftrace script can be used to demonstrate that the tests do indeed fail when attached to the tracepoints.

fail_tests.bt:

#!/usr/bin/env bpftrace

usdt:./build/src/test/test_bitcoin:test:check_if_attached {
  printf("the 'check_if_attached' test should have failed\n");
}

usdt:./build/src/test/test_bitcoin:test:expensive_section {
  printf("the 'expensive_section' test should have failed\n");
}

Run the unit tests with ./build/src/test/test_bitcoin and start bpftrace fail_tests.bt -p $(pidof test_bitcoin) in a separate terminal. The unit tests should fail with:

Running 594 test cases...
test/util_trace_tests.cpp(31): error: in "util_trace_tests/test_tracepoint_check_if_attached": check false has failed
test/util_trace_tests.cpp(51): error: in "util_trace_tests/test_tracepoint_manual_tracepoint_active_check": check false has failed

*** 2 failures are detected in the test module "Bitcoin Core Test Suite"

These links might provide more contextual information for reviewers:

How SystemTap Userspace Probes Work by eklitzke (actually an example on Bitcoin Core; mentions that with semaphores "the overhead for an untraced process is effectively zero.")
libbpf comment on USDT semaphore handling (can recommend the whole comment for background on how the tracepoints and tracing tools work together)
https://sourceware.org/systemtap/wiki/UserSpaceProbeImplementation#Semaphore_Handling

DrahtBot · 2022-11-28T20:50:29Z

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/26593.

Reviews

See the guideline for information on the review process.

Type	Reviewers
ACK	willcl-ark, vasild, laanwj, jb55
Concept ACK	kouloumos
Approach ACK	virtu

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#31161 (cmake: Set top-level target output locations by hebasto)
#31122 (cluster mempool: Implement changeset interface for mempool by sdaftuar)
#30611 (validation: write chainstate to disk every hour by andrewtoth)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

src/util/trace.h

0xB10C · 2022-11-29T16:28:37Z

Addressed tracing: Only prepare tracepoint arguments when actually tracing #26593 (comment) by using __VA_ARGS__. Also using a separate TRACEPOINT0 macro for tracepoints without arguments, as using the TRACEPOINT macro without arguments is only be possible with GCC (clang warns that we are using gcc extensions). Tested with both clang and gcc.
Adressed tracing: Only prepare tracepoint arguments when actually tracing #26593 (comment) by dropping the TRACEPOINT_UNCHECKED macro.
Added unit tests for the TRACEPOINT macros. The tracepoints in the unit tests don't do anything in particular, but the tests show that the macros work. Through the CI we cover ENABLE_TRACING being defined and undefined. The test case test_tracepoint_check_if_attached also shows that the TRACEPOINT_ACTIVE macro is not broken (it's tested that it's working in the functional tests).

jb55 · 2022-11-29T17:25:55Z

genius. Concept ACK

vasild · 2022-12-02T07:30:12Z

Concept ACK

virtu · 2022-12-12T09:52:55Z

Approach ACK

Super cool stuff. This effectively removes any runtime overhead of tracepoints on modern processors with branch prediction and speculative execution when no probes are attached .

Some feedback on testing with 61ca0f9:

functional tests executed successfully on x86_64 and arm64
demo scripts in contrib/tracing: right now, bpftrace scripts work, bcc scripts don't

generated assembly looks as expected:

tracing code is semaphore-gated using compare and branch instructions:

# x86_64
cmp    WORD PTR [rip+0x8f816b],0x0        # 0x55d32c25d6c0 <net_outbound_message_semaphore>
jne    0x55d32b965800 <_ZN8CConnman11PushMessageEP5CNodeO17CSerializedNetMsg+1488>

# arm64
ldrh    w1, [x0, #1776]
cbnz    w1, 0xaaaab2413fa4 <_ZN8CConnman11PushMessageEP5CNodeO17CSerializedNetMsg+1228>

tracing code uses nop when no probe is attached vs. int3 (x86_64) / brk (arm64) to trap when probe is attached

used gdb to inspect semaphores: they are correctly initialized to zero and set to one if one or more(!) probes are attached. The documentation I read stated semaphores act as probe reference counters, so I was expecting semaphore values to reflect the number of probes. However, it looks like semaphores and counters were separated. This does not impact correct behavior though:

(gdb) # no probe attached
(gdb) x/hx &net_outbound_message_semaphore
0xaaaacc8506f0 <net_outbound_message_semaphore>:        0x0000
(gdb) # started one instance of contrib/tracing/log_p2p_traffic.bt
(gdb) x/hx &net_outbound_message_semaphore
0xaaaacc8506f0 <net_outbound_message_semaphore>:        0x0001
(gdb) # started another instance of contrib/tracing/log_p2p_traffic.bt
(gdb) x/hx &net_outbound_message_semaphore
0xaaaacc8506f0 <net_outbound_message_semaphore>:        0x0001
(gdb) # stopped first instance
(gdb) x/hx &net_outbound_message_semaphore
0xaaaacc8506f0 <net_outbound_message_semaphore>:        0x0001
(gdb) # stopped second instance
(gdb) x/hx &net_outbound_message_semaphore
0xaaaacc8506f0 <net_outbound_message_semaphore>:        0x0000

0xB10C · 2022-12-13T08:34:36Z

Thanks for the detailed review and testing. I've too noticed that the current example bcc scripts have a problem with the semaphores. The functional tests, also using bcc, work as intended as they use the PID to attach to the bitcoind process. Will add a commit to allow running the example scripts by specifying a PID.

kouloumos · 2022-12-15T18:37:52Z

Concept ACK!
I've though about something similar while working on #25541 and even more when you mentioned performance impact on #25832.

0xB10C · 2022-12-20T12:41:14Z

demo scripts in contrib/tracing: right now, bpftrace scripts work, bcc scripts don't -- (#26593 (comment))

Rebased and added a commit that fixes the bcc example scripts by using the PID of bitcoind instead of the file path. We already do this in the tests (220a5a2).

…-arguments` to compile without warnings 5197660 tracepoints: Disables `-Wgnu-zero-variadic-macro-arguments` to compile without warnings (Martin Leitner-Ankerl) Pull request description: Fixes bitcoin#26916 by disabling the warning `-Wgnu-zero-variadic-macro-arguments` when clang is used as the compiler. Also see the comments * Proposed changes in the bug bitcoin#26916 (comment) * Proposed changes when moving to a variadic maro: bitcoin#26593 (comment) ACKs for top commit: hebasto: ACK 5197660, I've reconsidered my [comment](bitcoin#27401 (comment)) and I think the current localized approach is optimal. fanquake: ACK 5197660 - checked that this fixes the warnings under Clang. Tree-SHA512: c3dda3bcbb2540af6283ffff65885a9937bfdaaef3b00dc7d60b9f9740031d5c36ac9cb3d3d8756dbadce4812201a9754f5b8770df0d5e0d5ee690ba8a7135d2

This deduplicates the TRACEx macros by using systemtaps STAP_PROBEV[0] variadic macro instead of the DTrace compability DTRACE_PROBE[1] macros. Bitcoin Core never had DTrace tracepoints, so we don't need to use the drop-in replacement for it. As noted in pr25541[2], these macros aren't compatibile with DTrace on macOS anyway. This also renames the TRACEx macro to TRACEPOINT to clarify what the macro does: inserting a tracepoint vs tracing (logging) something. [0]: https://sourceware.org/git/?p=systemtap.git;a=blob;f=includes/sys/sdt.h;h=24d5e01c37805e55c36f7202e5d4e821b85167a1;hb=ecab2afea46099b4e7dfd551462689224afdbe3a#l407 [1]: https://sourceware.org/git/?p=systemtap.git;a=blob;f=includes/sys/sdt.h;h=24d5e01c37805e55c36f7202e5d4e821b85167a1;hb=ecab2afea46099b4e7dfd551462689224afdbe3a#l490 [2]: https://github.com/bitcoin/bitcoin/pull/25541/files#diff-553886c5f808e01e3452c7b21e879cc355da388ef7680bf310f6acb926d43266R30-R31 Co-authored-by: Martin Leitner-Ankerl <martin.ankerl@gmail.com>

Before this commit, we would always prepare tracepoint arguments regardless of the tracepoint being used or not. While we already made sure not to include expensive arguments in our tracepoints, this commit introduces gating to make sure the arguments are only prepared if the tracepoints are actually used. This is a win-win improvement to our tracing framework. For users not interested in tracing, the overhead is reduced to a cheap 'greater than 0' compare. As the semaphore-gating technique used here is available in bpftrace, bcc, and libbpf, users interested in tracing don't have to change their tracing scripts while profiting from potential future tracepoints passing slightly more expensive arguments. An example are mempool tracepoints that pass serialized transactions. We've avoided the serialization in the past as it was too expensive. Under the hood, the semaphore-gating works by placing a 2-byte semaphore in the '.probes' ELF section. The address of the semaphore is contained in the ELF note providing the tracepoint information (`readelf -n ./src/bitcoind | grep NT_STAPSDT`). Tracing toolkits like bpftrace, bcc, and libbpf increase the semaphore at the address upon attaching to the tracepoint. We only prepare the arguments and reach the tracepoint if the semaphore is greater than zero. The semaphore is decreased when detaching from the tracepoint. This also extends the "Adding a new tracepoint" documentation to include information about the semaphores and updated step-by-step instructions on how to add a new tracepoint.

BCC needs the PID of a bitcoind process to attach to the tracepoints (instead of the binary path used before) when the tracepoints have a semaphore. For reference, we already use the PID in our tracepoint interface tests. See 220a5a2.

willcl-ark · 2024-10-30T10:51:21Z

utACK 0de3e96

Based on git range-diff 38e24df...0de3e96e333090548a43e5e870c4cb8941d6baf1 .

The cmake tidy-up looks good to me, and I examined newly built binaries for probe and semaphore info. I didn't re-run the testing from my previous ACK.

vasild

ACK 0de3e96

laanwj · 2024-11-08T20:14:29Z

re-ACK 0de3e96

jb55 · 2024-11-08T20:44:41Z

utACK 0de3e96

0xB10C · 2024-11-11T11:57:20Z

Less overhead for everyone not hooking into the tracepoints 🥳. Now that this is merged, here are a few ideas I had for future tracepoint interface work. Noting them for prosperity.

We could internalize the relevant macro parts of systemtap's sys/sdt.h for the Linux tracepoints. This would allow us to drop the external dependency on systemtap, as we don't use 99% of it. Some prior commentary on this can be found here: cmake: Switch from tri-state options to boolean. Stage TWO hebasto/bitcoin#162 (comment)
In the past I've managed to use a simple macro build a bitcoind with tracepoints on macOS. While our ebpf based demo scripts aren't compatible, @kouloumos DTrace scripts from tracing: macOS USDT support #25541 are. This could look similar to https://github.com/0xB10C/bitcoin/blob/13b0ce221600fc7040502c834c51433ca96f91c3/src/util/trace.h#L35-L63. However, I currently don't have access to a macOS system to further work on this - I'm looking to rent one.
The same could possible on FreeBSD with e.g. these macros https://github.com/0xB10C/bitcoin/blob/13b0ce221600fc7040502c834c51433ca96f91c3/src/util/trace.h#L104-L119. I haven't tested this on FreeBSD yet. In build: Detect USDT the same way how it is used in the code #27458 (review), vasild mentiones he'd interested in FreeBSD tracepoints. My understanding is that the same macOS DTrace scripts from 25541 would work there too.
At least for macOS, we'd need an per-tracepoint interface definition similar to https://github.com/0xB10C/bitcoin/blob/13b0ce221600fc7040502c834c51433ca96f91c3/src/util/trace.h#L121-L236. With some more commentary, these could replace the list of tracepoints in https://github.com/bitcoin/bitcoin/blob/master/doc/tracing.md#tracepoint-documentation. This would solve something similar to tracing: explicitly cast block_connected duration to nanoseconds #29877 (comment).

Even if we don't do 4. (because we e.g. don't want to do 2.), casting the tracepoint arguments to the type we expect to pass would be worthwhile to avoid problems like tracing: explicitly cast block_connected duration to nanoseconds #29877. For some of our traceponts, we already do this: e.g.

bitcoin/src/validation.cpp

Lines 2902 to 2907 in 900b172

    
           TRACEPOINT(utxocache, flush, 
        
                  int64_t{Ticks<std::chrono::microseconds>(SteadyClock::now() - nNow)}, 
        
                  (uint32_t)mode, 
        
                  (uint64_t)coins_count, 
        
                  (uint64_t)coins_mem_usage, 
        
                  (bool)fFlushForPrune);

Two other ideas that were mentioned in the past:

We could drop the example scripts from /contrib/tracing/* and maintain them, along with proper tests in a CI, Bitcoin Core version compatibility information, possibly libbpf-based C or Rust tools (Porting bcc tools to libbpf #30298), ... in an external repository like, for example, 0xb10c/tracing-scripts, bitcoin-core/tracing-scripts, or bitcoin-monitoring/tracing-scripts (what ever would works best).
If we at some point decide that maintaining the tracepoints in Bitcoin Core adds too much maintenance burden compared to the actual usage they're getting, we could drop the tracepoints but keep the tracepoint interface. We now have a unit test that includes a few nop tracepoints to check that the interface will still compile (https://github.com/0xB10C/bitcoin/blob/0de3e96e333090548a43e5e870c4cb8941d6baf1/src/test/util_trace_tests.cpp). This would allow us to drop the bcc python dependency in the CI and to remove the interface_usdt_* functional tests (which need to run in VM and can't run in a container). Tracepoint users could maintain a patch on Bitcoin Core adding the tracepoints (and tests) they want back in. We'd however loose the tracepoints in release (or actually all) builds which currently allow e.g. everyone experiencing problems with their node to hook into them and extract data without needing to restart it.

…158303fe2 48158303fe2 kernel: Add pure kernel bitcoin-chainstate bf80d2f5009 kernel: Add block index utility functions to C header a6ab5345e3b kernel: Add function to read block undo data from disk to C header 845b824d6c7 kernel: Add functions to read block from disk to C header 9324c8c4f67 kernel: Add function for copying block data to C header 368fc93fd80 kernel: Add functions for the block validation state to C header eb6e25ac007 kernel: Add validation interface to C header cdce4484005 kernel: Add interrupt function to C header 7e47ec78768 kernel: Add import blocks function to C header 2b803d50747 kernel: Add chainstate load options for in-memory dbs in C header ea92eb13c4a kernel: Add options for reindexing in C header 8254f2035a7 kernel: Add block validation to C header ad7b880346e Kernel: Add chainstate loading to kernel C header 583820c4487 kernel: Add chainstate manager object to C header ec137a086a0 kernel: Add notifications context option to C header 62a89689266 kerenl: Add chain params context option to C header bb482dcbd30 kernel: Add kernel library context object d114ccfdf8a kernel: Add logging to kernel library C header 44c65c46c43 kernel: Introduce initial kernel C header API 69c03134440 Merge bitcoin/bitcoin#31269: validation: Remove RECENT_CONSENSUS_CHANGE validation result 42282592943 Merge bitcoin/bitcoin#31000: bench: add support for custom data directory 36f5effa178 Merge bitcoin/bitcoin#31235: addrman: cap the `max_pct` to not exceed the maximum number of addresses 98ad249b69f Merge bitcoin/bitcoin#31277: doc: mention `descriptorprocesspsbt` in psbt.md b0222bbb494 Merge bitcoin/bitcoin#30239: Ephemeral Dust 1dda1892b6b Merge bitcoin/bitcoin#31037: test: enhance p2p_orphan_handling 5c2e291060c bench: Add basic CheckEphemeralSpends benchmark 3f6559fa581 Add release note for ephemeral dust 71a6ab4b33d test: unit test for CheckEphemeralSpends 21d28b2f362 fuzz: add ephemeral_package_eval harness 127719f516a test: Add CheckMempoolEphemeralInvariants e2e30e89ba4 functional test: Add ephemeral dust tests 4e68f901390 rpc: disallow in-mempool prioritisation of dusty tx e1d3e81ab4d policy: Allow dust in transactions, spent in-mempool 04b2714fbbc functional test: Add new -dustrelayfee=0 test case ebb6cd82baf doc: mention `descriptorprocesspsbt` in psbt.md 2b33322169b Merge bitcoin/bitcoin#31249: test: Add combinerawtransaction test to rpc_createmultisig 3fb6229dcfd Merge bitcoin/bitcoin#31271: doc: correct typos fa66e0887ca bench: add support for custom data directory ad9c2cceda9 test, bench: specialize working directory name 9c5775c331e addrman: cap the `max_pct` to not exceed the maximum number of addresses 8d340be9247 Merge bitcoin/bitcoin#31181: cmake: Revamp `FindLibevent` module 9a8e5adb161 Merge bitcoin/bitcoin#31267: refactor: Drop deprecated space in operator""_mst 726cbee9553 doc: correct typos 9fdfb73ca84 doc: fix typos af6088701a2 Merge bitcoin/bitcoin#31237: doc: Add missing 'blank=true' option in offline-signing-tutorial.md 7a526653022 Merge bitcoin/bitcoin#31239: test: clarify log messages when handling SOCKS5 proxy connections 900b17239fb Merge bitcoin/bitcoin#31259: doc: Fix missing comma in JSON example in REST-interface.md faf21625652 refactor: Drop deprecated space in operator""_mst c889890e4a3 Merge bitcoin/bitcoin#31264: doc: Fixup bitcoin-wallet manpage chain selection args 0f6d20e43f2 Merge bitcoin/bitcoin#31163: scripted-diff: get rid of remaining "command" terminology in protocol.{h,cpp} 5acd5e7f874 Merge bitcoin/bitcoin#31257: ci: make ctest stop on failure 19f277711eb Merge bitcoin/bitcoin#26593: tracing: Only prepare tracepoint arguments when actually tracing e80e4c6ff91 validation: Remove RECENT_CONSENSUS_CHANGE validation result fa729ab4a27 doc: Fixup bitcoin-wallet manpage chain selection args 5e3b444022c doc: Fix missing comma in JSON example in REST-interface.md 0903ce8dbc2 Merge bitcoin/bitcoin#30592: Remove mempoolfullrbf f842d0801e1 Merge bitcoin/bitcoin#29686: Update manpage descriptions 36a22e56833 ci: make ctest stop on failure 83fab3212c9 test: Add combinerawtransaction test to rpc_createmultisig 018e5fcc462 Merge bitcoin/bitcoin#31190: TxDownloadManager followups 3a5f6027e16 Merge bitcoin/bitcoin#31171: depends: Specify CMake generator explicitly 99d9a093cf6 test: clarify log messages when handling SOCKS5 proxy connections c9e67e214f0 Merge bitcoin/bitcoin#31238: fuzz: Limit wallet_notifications iterations 564238aabf1 Merge bitcoin/bitcoin#31164: net: Use actual memory size in receive buffer accounting fa461d7a43a fuzz: Limit wallet_notifications iterations ec375de39ff doc: Add missing 'blank=true' option in offline-signing-tutorial.md 5a96767e3f5 depends, libevent: Do not install *.pc files and remove patches for them ffda355b5a2 cmake, refactor: Move `HAVE_EVHTTP_...` to `libevent` interface b619bdc3303 cmake: Revamp `FindLibevent` module 2c90f8e08c4 Merge bitcoin/bitcoin#31232: ci: `add second_deadlock_stack=1` to TSAN options 5dc94d13d41 fuzz fix: assert MAX_PEER_TX_ANNOUNCEMENTS is not exceeded 45e2f8f87d8 Merge bitcoin/bitcoin#31173: cmake: Add `FindQRencode` module and enable `libqrencode` package for MSVC 80cb630bd94 Merge bitcoin/bitcoin#31216: Update secp256k1 subtree to v0.6.0 5161c2618cd ci: add second_deadlock_stack=1 to TSAN options 85224f92d52 Merge bitcoin/bitcoin#30811: build: Unify `-logsourcelocations` format 9719d373dc2 Merge bitcoin/bitcoin#30634: ci: Use clang-19 from apt.llvm.org 97235c446e9 build: Disable secp256k1 musig module 9e5089dbb02 build, msvc: Enable `libqrencode` vcpkg package 30089b0cb61 cmake: Add `FindQRencode` module 65b19419366 Merge bitcoin/bitcoin#31186: msvc: Update vcpkg manifest d3388720837 Merge bitcoin/bitcoin#31206: doc: Use relative hyperlinks in release-process.md ffc05fca6f7 Merge bitcoin/bitcoin#31220: doc: Fix word order in developer-notes.md 9f2c8287a24 Merge bitcoin/bitcoin#31192: depends, doc: List packages required to build `qt` package separately 03cff2c1421 Merge bitcoin/bitcoin#31191: build: Make G_FUZZING constexpr, require -DBUILD_FOR_FUZZING=ON to fuzz 44939e5de1b doc: Fix word order in developer-notes.md b934954ad10 Merge bitcoin/bitcoin#30670: doc: Extend developer-notes with file-name-only debugging fix 05aebe3790f Merge bitcoin/bitcoin#30930: netinfo: add peer services column and outbound-only option 0ba680d41b4 Update secp256k1 subtree to v0.6.0 2d46a89386d Squashed 'src/secp256k1/' changes from 2f2ccc46954..0cdc758a563 d22a234ed27 net: Use actual memory size in receive buffer accounting 047b5e2af1f streams: add DataStream::GetMemoryUsage c3a6722f34a net: Use DynamicUsage(m_type) in CSerializedNetMsg::GetMemoryUsage c6594c0b142 memusage: Add DynamicUsage for std::string 7596282a556 memusage: Allow counting usage of vectors with different allocators 6463117a292 Merge bitcoin/bitcoin#31208: doc: archive release notes for v27.2 788c1324f3d build: Unify `-logsourcelocations` format 4747f030956 depends, doc: List packages required to build `qt` package separately 1a05c86ae47 doc: archive release notes for v27.2 9f71cff6ab3 doc: Use relative hyperlinks in release-process.md f1bcf3edc50 Merge bitcoin/bitcoin#31139: test: added test to assert TX decode rpc error on submitpackage rpc 975b115e1a2 Merge bitcoin/bitcoin#31198: init: warn, don't error, when '-upnp' is set 4a0251c05dd Merge bitcoin/bitcoin#31187: ci: Do not error on unused-member-function in test each commit e001dc3dc6e Merge bitcoin/bitcoin#31203: fuzz: fix `implicit-integer-sign-change` in wallet_create_transaction 5a26cf7773e fuzz: fix `implicit-integer-sign-change` in wallet_create_transaction a1b3ccae4be init: warn, don't error, when '-upnp' is set c189eec848e doc: release note for mempoolrullrbf removal d47297c6aab rpc: Mark fullrbf and bip125-replaceable as deprecated 04a5dcee8ab docs: remove requirement to signal bip125 fafbf8acf41 Make G_FUZZING constexpr, require -DBUILD_FOR_FUZZING=ON to execute a fuzz target fae3cf0ffa6 ci: Temporarily disable macOS/Windows fuzz step f6577b71741 build, msvc: Update vcpkg manifest baseline 16e16013bfa build, msvc: Document `libevent` version pinning ec47cd2b508 build, msvc: Drop no longer needed `liblzma` version pinning 9a0734df5f1 build, msvc: Reorder keys in `vcpkg.json` 8351562bec6 [fuzz] allow negative time jumps in txdownloadman_impl 917ab810d93 [doc] comment fixups from n30110 f07a533dfcb Merge bitcoin/bitcoin#24214: Fix unsigned integer overflows in interpreter 62516105536 Merge bitcoin/bitcoin#31015: build: have "make test" depend on "make all" 4a31f8ccc9d Merge bitcoin/bitcoin#31156: test: Don't enforce BIP94 on regtest unless specified by arg 02be3dced71 Merge bitcoin/bitcoin#31166: key: clear out secret data in `DecodeExtKey` 54d07dd37d5 ci: Do not error on unused-member-function in test each commit 47f50c7af55 doc: add bitcoin-qt man description 40b82e3ab0a doc: add bitcoin-util man description a7bf80f3a2d doc: add bitcoin-tx man description 3f9a5168323 doc: add bitcoin-wallet man description d8c0bb23ef8 doc: add bitcoin-cli man description 09abccfa772 doc: add bitcoind man description 97b790e844a Merge bitcoin/bitcoin#29420: test: extend the SOCKS5 Python proxy to actually connect to a destination 6b73eb9a1a2 Merge bitcoin/bitcoin#31064: init: Correct coins db cache size setting 27d12cf17f2 Merge bitcoin/bitcoin#31043: rpc: getorphantxs follow-up 7b66815b16b Merge bitcoin/bitcoin#30110: refactor: TxDownloadManager + fuzzing dc97e7f6dba Merge bitcoin/bitcoin#30903: cmake: Add `FindZeroMQ` module 1b0b9b4c787 Extend possible debugging fixes with file-name-only da10e0bab4a Merge bitcoin/bitcoin#30942: test: Remove dead code from interface_zmq test 111a23d9b36 Remove -mempoolfullrbf option e96ffa98b04 Merge bitcoin/bitcoin#31142: test: fix intermittent failure in p2p_seednode.py, don't connect to random IPs 54c4b09f083 Merge bitcoin/bitcoin#31042: build: Rename `PACKAGE_*` variables to `CLIENT_*` e60cecc8115 doc: add release note for 31156 fc7dfb3df5b test: Don't enforce BIP94 on regtest unless specified by arg fabe90c8242 ci: Use clang-19 from apt.llvm.org 0de3e96e333 tracing: use bitcoind pid in bcc tracing examples 411c6cfc6c2 tracing: only prepare tracepoint args if attached d524c1ec066 tracing: dedup TRACE macros & rename to TRACEPOINT 70713303b63 scripted-diff: Rename `PACKAGE_*` variables to `CLIENT_*` 332655cb52c build: Rename `PACKAGE_*` variables to `CLIENT_*` e6e29e3c94c scripted-diff: Clarify "user agent" variable name e2ba8236715 depends: Specify CMake generator explicitly 1c7ca6e64de Merge bitcoin/bitcoin#31093: Introduce `g_fuzzing` global for fuzzing checks 6e21dedbf2b Merge bitcoin/bitcoin#31130: Drop miniupnp dependency d7fd766feb2 test: added test to assert TX decode rpc error on submitpackage rpc 559a8dd9c0a key: clear out secret data in `DecodeExtKey` 4120c7543ee scripted-diff: get rid of remaining "command" terminology in protocol.{h,cpp} 2a52718d734 Merge bitcoin/bitcoin#31152: functional test: Additional package evaluation coverage 9de9c858d5a test: enhance p2p_orphan_handling 33af14b62e4 test: reduce assert_debug_log reliance 0ea84bc362f test: explicitly check boolean verbosity is disallowed 7a2e6b68cd9 doc: add rpc guidance for boolean verbosity avoidance 698f302df8b rpc: disallow boolean verbosity in getorphantxs 63f5e6ec795 test: add entry and expiration time checks 808a708107e rpc: add entry time to getorphantxs 56bf3027144 refactor: rename rpc_getorphantxs to rpc_orphans 7824f6b0770 test: check that getorphantxs is hidden ac68fcca701 rpc: disallow undefined verbosity in getorphantxs 25dacae9c7f Merge bitcoin/bitcoin#31040: test: Assert that when we add the max orphan amount that we cannot add anymore and that a random orphan gets dropped 40e5f26a3ff mapport: remove dead code in DispatchMapPort 38fdf7c1fb1 mapport: drop outdated comments 915640e191b depends: zeromq: don't install .pc files and remove patches for them 6b8a74463b5 cmake: Add `FindZeroMQ` module 9a7206a34e3 Merge bitcoin/bitcoin#29536: fuzz: fuzz connman with non-empty addrman + ASMap d4abaf8c9d9 Merge bitcoin/bitcoin#29608: optimization: Preallocate addresses in GetAddr based on nNodes b7b24352906 doc: add release note for #31130 1b6dec98da3 depends: drop miniupnpc 953533d0214 doc: remove mentions of UPnP 94ad614482f ci: remove UPnP options f32c34d0c3d functional test: Additional package evaluation coverage 87532fe5585 netinfo: allow setting an outbound-only peer list 9f243cd7fa6 Introduce `g_fuzzing` global for fuzzing checks b95adf057a4 Merge bitcoin/bitcoin#31150: util: Treat Assume as Assert when evaluating at compile-time 8f24e492e20 Merge bitcoin/bitcoin#29991: depends: sqlite 3.46.1 2ef5004f78c Merge bitcoin/bitcoin#31146: ci: Temporary workaround for old CCACHE_DIR cirrus env 8c12fe828de Merge bitcoin/bitcoin#29936: fuzz: wallet: add target for `CreateTransaction` 5c299ecafe6 test: Assert that when we add the max orphan amount that we cannot add anymore and that a random orphan gets dropped 0f4bc635854 [fuzz] txdownloadman and txdownload_impl 699643f23a1 [unit test] MempoolRejectedTx fa584cbe727 [p2p] add TxDownloadOptions bool to make TxRequestTracker deterministic f803c8ce8dd [p2p] filter 1p1c for child txid in recent rejects 5269d57e6d7 [p2p] don't process orphan if in recent rejects 2266eba43a9 [p2p] don't find 1p1cs for reconsiderable txns that are AlreadyHaveTx fa7027d0fc1 [refactor] add CheckIsEmpty and GetOrphanTransactions, remove access to TxDownloadMan internals 969b07237b9 [refactor] wrap {Have,Get}TxToReconsider in txdownload f150fb94e7d [refactor] make AlreadyHaveTx and Find1P1CPackage private to TxDownloadImpl 1e08195135b [refactor] move new tx logic to txdownload 257568eab5b [refactor] move invalid package processing to TxDownload c4ce0c1218d [refactor] move invalid tx processing to TxDownload c6b21749ca0 [refactor] move valid tx processing to TxDownload a8cf3b6e845 [refactor] move Find1P1CPackage to txdownload f497414ce76 [refactor] put peerman tasks at the end of ProcessInvalidTx 6797bc42a76 [p2p] restrict RecursiveDynamicUsage of orphans added to vExtraTxnForCompact 798cc8f5aac [refactor] move Find1P1CPackage into ProcessInvalidTx 416fbc952b2 [refactor] move new orphan handling to ProcessInvalidTx c8e67b9169b [refactor] move ProcessInvalidTx and ProcessValidTx definitions down 3a41926d1b5 [refactor] move notfound processing to txdownload 042a97ce7fc [refactor] move tx inv/getdata handling to txdownload 58e09f244b4 [p2p] don't log tx invs when in IBD 288865338f5 [refactor] rename maybe_add_extra_compact_tx to first_time_failure f48d36cd97e [refactor] move peer (dis)connection logic to TxDownload f61d9e4b4b8 [refactor] move AlreadyHaveTx to TxDownload 84e4ef843db [txdownload] add read-only reference to mempool af918349de5 [refactor] move ValidationInterface functions to TxDownloadManager f6c860efb12 [doc] fix typo in m_lazy_recent_confirmed_transactions doc 5f9004e1550 [refactor] add TxDownloadManager wrapping TxOrphanage, TxRequestTracker, and bloom filters 947f2925d55 Merge bitcoin/bitcoin#31124: util: Remove RandAddSeedPerfmon 7640cfdd624 Merge bitcoin/bitcoin#31118: doc: replace `-?` with `-h` and `-help` 74fb19317ae Merge bitcoin/bitcoin#30849: refactor: migrate `bool GetCoin` to return `optional<Coin>` c16e909b3e2 Merge bitcoin/bitcoin#28574: wallet: optimize migration process, batch db transactions a9598e5eaab build: drop miniupnpc dependency a5fcfb7385c interfaces: remove now unused 'use_upnp' arg from 'mapPort' 038bbe7b200 daemon: remove UPnP support 844770b05eb qt: remove UPnP settings dd92911732d Merge bitcoin/bitcoin#31148: ci: display logs of failed unit tests automatically fa69a5f4b76 util: Treat Assume as Assert when evaluating at compile-time 0c79c343a9f Merge bitcoin/bitcoin#31147: cmake, qt, test: Remove problematic code 8523d8c0fc8 ci: display logs of failed tests automatically 2f40e453ccd Merge bitcoin/bitcoin#29450: build: replace custom `MAC_OSX` macro with existing `__APPLE__` cb7c5ca824e Add gdb and lldb links to debugging troubleshooting 6c6b2442eda build: Replace MAC_OSX macro with existing __APPLE__ fb46d57d4e7 cmake, qt, test: Remove problematic code fa9747a8961 ci: Temporary workaround for old CCACHE_DIR cirrus env 6c9fe7b73ea test: Prevent connection attempts to random IPs in p2p_seednodes.py bb97b1ffa9f test: fix intermittent timeout in p2p_seednodes.py 57529ac4dbb test: set P2PConnection.p2p_connected_to_node in peer_connect_helper() 22cd0e888c7 test: support WTX INVs from P2PDataStore and fix a comment ebe42c00aa4 test: extend the SOCKS5 Python proxy to actually connect to a destination 9bb92c0e7ff util: Remove RandAddSeedPerfmon c98fc36d094 wallet: migration, consolidate external wallets db writes 7c9076a2d2e wallet: migration, consolidate main wallet db writes 9ef20e86d7f wallet: provide WalletBatch to 'SetupDescriptorScriptPubKeyMans' 34bf0795fc0 wallet: refactor ApplyMigrationData to return util::Result<void> aacaaaa0d3a wallet: provide WalletBatch to 'RemoveTxs' 57249ff6697 wallet: introduce active db txn listeners 91e065ec175 wallet: remove post-migration signals connection 055c0532fc8 wallet: provide WalletBatch to 'DeleteRecords' 122d103ca22 wallet: introduce 'SetWalletFlagWithDB' 6052c7891dc wallet: decouple default descriptors creation from external signer setup f2541d09e13 wallet: batch MigrateToDescriptor() db transactions 66c9936455f bench: add coverage for wallet migration process 33a28e252a7 Change default help arg to `-help` and mention `-h` and `-?` as alternatives f0130ab1a1e doc: replace `-?` with `-h` for bench_bitcoin help 681ebcceca7 netinfo: rename and hoist max level constant to use in top-level help e7d307ce8cf netinfo: clarify relaytxes and addr_relay_enabled help docs eef2a9d4062 netinfo: add peer services column 3a4a788ee0d init: Correct coins db cache size setting 2957ca96119 build: have "make test" depend on "make all" bbbbaa0d9ac Fix unsigned integer overflows in interpreter c4dc81f9c69 test: Remove dead code from interface_zmq c495731a316 fuzz: wallet: add target for `CreateTransaction` 3db68e29ec6 wallet: move `ImportDescriptors`/`FuzzedWallet` to util 552cae243a1 fuzz: cover `ASMapHealthCheck` in connman target 33b0f3ae966 fuzz: use `ConsumeNetGroupManager` in connman target 18c8a0945bd fuzz: move `ConsumeNetGroupManager` to util fe624631aeb fuzz: fuzz `connman` with a non-empty addrman 0a12cff2a8e fuzz: move `AddrManDeterministic` to util 4feaa287284 refactor: Rely on returned value of GetCoin instead of parameter 46dfbf169b4 refactor: Return optional of Coin in GetCoin e31bfb26c21 refactor: Remove unrealistic simulation state ba621ffb9cb test: improve debug log message from P2PConnection::connection_made() def6dd0c597 depends: sqlite 3.46.1 66082ca3488 Preallocate addresses in GetAddr based on nNodes REVERT: 1047757ea3b kernel: Add pure kernel bitcoin-chainstate REVERT: c568fdf75fd kernel: Add block index utility functions to C header REVERT: 0f1da1dcba5 kernel: Add function to read block undo data from disk to C header REVERT: 45af559c9f6 kernel: Add functions to read block from disk to C header REVERT: 2a7f8a8240c kernel: Add function for copying block data to C header REVERT: b19f5336c03 kernel: Add functions for the block validation state to C header REVERT: 9c0ffa913f4 kernel: Add validation interface to C header REVERT: a93318c6152 kernel: Add interrupt function to C header REVERT: 51053f33720 kernel: Add import blocks function to C header REVERT: 6b0ada2af42 kernel: Add chainstate load options for in-memory dbs in C header REVERT: 34427bfa9c7 kernel: Add options for reindexing in C header REVERT: ca57311c969 kernel: Add block validation to C header REVERT: 44156d84838 Kernel: Add chainstate loading to kernel C header REVERT: 2cee46cdcc1 kernel: Add chainstate manager object to C header REVERT: 7102c7ae45e kernel: Add notifications context option to C header REVERT: ed628a2a3c4 kerenl: Add chain params context option to C header REVERT: 27643297ff7 kernel: Add kernel library context object REVERT: 2ba22cf3f90 kernel: Add logging to kernel library C header REVERT: 873874c03e9 kernel: Introduce initial kernel C header API git-subtree-dir: libbitcoinkernel-sys/bitcoin git-subtree-split: 48158303fe276cb2f8fbc53ff31a4162d8f55c84

…y Defined Tracing (USDT) - backported from bitcoin/bitcoin#19866, bitcoin/bitcoin#22006, bitcoin/bitcoin#26593

Since bitcoin/bitcoin#26593 a PID is required, so have the ebpf-extractor require one.

0xB10C force-pushed the 2022-11-only-prepare-tracepoint-arguments-when-tracing branch from 27bd196 to 94331ba Compare November 28, 2022 20:54

luke-jr reviewed Nov 28, 2022

View reviewed changes

src/util/trace.h Outdated Show resolved Hide resolved

0xB10C commented Nov 29, 2022

View reviewed changes

src/util/trace.h Outdated Show resolved Hide resolved

0xB10C force-pushed the 2022-11-only-prepare-tracepoint-arguments-when-tracing branch from 94331ba to be0a187 Compare November 29, 2022 15:44

0xB10C force-pushed the 2022-11-only-prepare-tracepoint-arguments-when-tracing branch from be0a187 to 61ca0f9 Compare November 29, 2022 16:29

0xB10C mentioned this pull request Dec 1, 2022

mempool: Add mempool tracepoints #26531

Merged

DrahtBot mentioned this pull request Dec 8, 2022

wallet: Coin Selection, return accurate error messages #26661

Merged

This was referenced Dec 16, 2022

BIP324: Enable v2 P2P encrypted transport #24545

Closed

test/BIP324: functional tests for v2 P2P encryption #24748

Merged

0xB10C force-pushed the 2022-11-only-prepare-tracepoint-arguments-when-tracing branch 2 times, most recently from 636d0b8 to 6ac946d Compare December 20, 2022 12:39

0xB10C force-pushed the 2022-11-only-prepare-tracepoint-arguments-when-tracing branch from 6ac946d to c4db6fb Compare December 20, 2022 12:44

DrahtBot added the Needs rebase label Jan 4, 2023

0xB10C force-pushed the 2022-11-only-prepare-tracepoint-arguments-when-tracing branch from c4db6fb to e35f0b5 Compare January 4, 2023 10:32

DrahtBot added the Needs rebase label Oct 24, 2024

0xB10C and others added 3 commits October 28, 2024 14:23

tracing: use bitcoind pid in bcc tracing examples

0de3e96

BCC needs the PID of a bitcoind process to attach to the tracepoints (instead of the binary path used before) when the tracepoints have a semaphore. For reference, we already use the PID in our tracepoint interface tests. See 220a5a2.

0xB10C force-pushed the 2022-11-only-prepare-tracepoint-arguments-when-tracing branch from 8a5b224 to 0de3e96 Compare October 28, 2024 13:28

DrahtBot removed the Needs rebase label Oct 28, 2024

DrahtBot mentioned this pull request Oct 28, 2024

cmake: Set top-level target output locations #31161

Merged

DrahtBot requested a review from vasild October 30, 2024 10:51

vasild approved these changes Nov 6, 2024

View reviewed changes

0xB10C mentioned this pull request Nov 6, 2024

Support PID / PID file to attach tracepoints to bitcoind 0xB10C/peer-observer#58

Closed

fanquake added this to the 29.0 milestone Nov 7, 2024

fanquake merged commit 19f2777 into bitcoin:master Nov 11, 2024
16 checks passed

0xB10C deleted the 2022-11-only-prepare-tracepoint-arguments-when-tracing branch November 11, 2024 11:57

This was referenced Nov 11, 2024

Tracepoint Interface Tracking Issue #31274

Open

add: use bitcoind PID in peer-observer 0xB10C/nix#75

Merged

ismaelsadeeq mentioned this pull request Nov 14, 2024

Fee Estimation via Fee rate Forecasters #30157

Draft

HiHat added a commit to HiHat/pocketnet.core that referenced this pull request Dec 15, 2024

tracing: first tracepoints and documentation on User-Space, Staticall…

0e181f3

…y Defined Tracing (USDT) - backported from bitcoin/bitcoin#19866, bitcoin/bitcoin#22006, bitcoin/bitcoin#26593

0xB10C added a commit to 0xB10C/peer-observer that referenced this pull request Aug 7, 2025

clean: either pid or pid file need to be given

aa2afda

Since bitcoin/bitcoin#26593 a PID is required, so have the ebpf-extractor require one.

0xB10C added a commit to 0xB10C/peer-observer that referenced this pull request Aug 8, 2025

clean: either pid or pid file need to be given

549e40b

Since bitcoin/bitcoin#26593 a PID is required, so have the ebpf-extractor require one.

0xB10C mentioned this pull request Aug 8, 2025

Various ebpf-extractor cleanups 0xB10C/peer-observer#214

Merged

0xB10C added a commit to 0xB10C/peer-observer that referenced this pull request Aug 11, 2025

clean: either pid or pid file need to be given

0aa0ea2

Since bitcoin/bitcoin#26593 a PID is required, so have the ebpf-extractor require one.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

tracing: Only prepare tracepoint arguments when actually tracing #26593

tracing: Only prepare tracepoint arguments when actually tracing #26593

Uh oh!

0xB10C commented Nov 28, 2022 •

edited by fanquake

Loading

Uh oh!

DrahtBot commented Nov 28, 2022 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

0xB10C commented Nov 29, 2022 •

edited

Loading

Uh oh!

jb55 commented Nov 29, 2022

Uh oh!

vasild commented Dec 2, 2022

Uh oh!

virtu commented Dec 12, 2022

Uh oh!

0xB10C commented Dec 13, 2022

Uh oh!

kouloumos commented Dec 15, 2022

Uh oh!

0xB10C commented Dec 20, 2022

Uh oh!

willcl-ark commented Oct 30, 2024

Uh oh!

vasild left a comment

Uh oh!

laanwj commented Nov 8, 2024

Uh oh!

jb55 commented Nov 8, 2024

Uh oh!

Uh oh!

0xB10C commented Nov 11, 2024 •

edited

Loading

Uh oh!

Uh oh!

tracing: Only prepare tracepoint arguments when actually tracing #26593

tracing: Only prepare tracepoint arguments when actually tracing #26593

Uh oh!

Conversation

0xB10C commented Nov 28, 2022 • edited by fanquake Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DrahtBot commented Nov 28, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Coverage & Benchmarks

Reviews

Conflicts

Uh oh!

Uh oh!

Uh oh!

0xB10C commented Nov 29, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jb55 commented Nov 29, 2022

Uh oh!

vasild commented Dec 2, 2022

Uh oh!

virtu commented Dec 12, 2022

Uh oh!

0xB10C commented Dec 13, 2022

Uh oh!

kouloumos commented Dec 15, 2022

Uh oh!

0xB10C commented Dec 20, 2022

Uh oh!

willcl-ark commented Oct 30, 2024

Uh oh!

vasild left a comment

Choose a reason for hiding this comment

Uh oh!

laanwj commented Nov 8, 2024

Uh oh!

jb55 commented Nov 8, 2024

Uh oh!

Uh oh!

0xB10C commented Nov 11, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

0xB10C commented Nov 28, 2022 •

edited by fanquake

Loading

DrahtBot commented Nov 28, 2022 •

edited

Loading

0xB10C commented Nov 29, 2022 •

edited

Loading

0xB10C commented Nov 11, 2024 •

edited

Loading