Skip to content

Conversation

glozow
Copy link
Member

@glozow glozow commented Nov 27, 2023

See #27463 for overall package relay tracking.

Delving Bitcoin discussion thread: https://delvingbitcoin.org/t/v3-transaction-policy-for-anti-pinning/340
Delving Bitcoin discussion for LN usage: https://delvingbitcoin.org/t/lightning-transactions-with-v3-and-ephemeral-anchors/418

Rationale:

  • There are various pinning problems with RBF and our general ancestor/descendant limits. These policies help mitigate many pinning attacks and make package RBF feasible (see Cluster size 2 package rbf #28984 which implements package RBF on top of this). I would focus the most here on Rule 3 pinning. [1][2]
  • Switching to a cluster-based mempool (see Proposal for a new mempool design #27677 and Cluster mempool implementation #28676) requires the removal of CPFP carve out, which applications depend on. V3 + package RBF + ephemeral anchors + 1-parent-1-child package relay provides an intermediate solution.

V3 policy is for "Priority Transactions." [3][4] It allows users to opt in to more restrictive topological limits for shared transactions, in exchange for the more robust fee-bumping abilities that offers. Even though we don't have cluster limits, we are able to treat these transactions as having as having a maximum cluster size of 2.

Immediate benefits:

  • You can presign a transaction with 0 fees (not just 1sat/vB!) and add a fee-bump later.
  • Rule 3 pinning is reduced by a significant amount, since the attacker can only attach a maximum of 1000vB to your shared transaction.

This also enables some other cool things (again see #27463 for overall roadmap):

  • Ephemeral Anchors
  • Package RBF for these 1-parent-1-child packages. That means e.g. a commitment tx + child can replace another commitment tx using the child's fees.
  • We can transition to a "single anchor" universe without worrying about package limit pinning. So current users of CPFP carve out would have something else to use.
  • We can switch to a cluster-based mempool [5] (Proposal for a new mempool design #27677 Cluster mempool implementation #28676), which removes CPFP carve out [6].

[1]: Original mailing list post and discussion about RBF pinning problems https://gist.github.com/glozow/25d9662c52453bd08b4b4b1d3783b9ff, https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-January/019817.html
[2]: A FAQ is "we need this for cluster mempool, but is this still necessary afterwards?" There are some pinning issues that are fixed here and not fully fixed in cluster mempool, so we will still want this or something similar afterward.
[3]: Mailing list post for v3 https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-September/020937.html
[4]: Original PR #25038 also contains a lot of the discussion
[5]: https://delvingbitcoin.org/t/an-overview-of-the-cluster-mempool-proposal/393/7
[6]: https://delvingbitcoin.org/t/an-overview-of-the-cluster-mempool-proposal/393#the-cpfp-carveout-rule-can-no-longer-be-supported-12

@DrahtBot
Copy link
Contributor

DrahtBot commented Nov 27, 2023

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage

For detailed information about the code coverage, see the test coverage report.

Reviews

See the guideline for information on the review process.

Type Reviewers
ACK sdaftuar, instagibbs, achow101
Concept NACK petertodd
Concept ACK ismaelsadeeq

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #29325 (consensus: Store transaction nVersion as uint32_t by achow101)
  • #29306 (policy: enable sibling eviction for v3 transactions by glozow)
  • #29242 (Mempool util: Add RBF diagram checks for single chunks against clusters of size 2 by instagibbs)
  • #28984 (Cluster size 2 package rbf by instagibbs)
  • #28960 (kernel: Remove dependency on CScheduler by TheCharlatan)
  • #28886 (refactor: Replace sets of txiter with CTxMemPoolEntryRefs by TheCharlatan)
  • #28710 (Remove the legacy wallet and BDB dependency by achow101)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

Copy link
Member

@dergoegge dergoegge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be good to have some fuzzing for the new policy rules. Maybe amending the existing fuzz targets (tx_pool, tx_package_eval) to include v3 txs is enough? A standalone harness for the new rules would probably also be good regardless.

@glozow
Copy link
Member Author

glozow commented Nov 27, 2023

Changed tx_pool and tx_package_eval fuzzer to sometimes create v3 transactions, and check v3 invariants at the end of each iteration.

@dergoegge
Copy link
Member

$ echo "ACVL/gA+Pj4+Pj4+3D4AAAAAAAAAAAAAAAAAAAAAAAAAAAAAPT09//8AAP/8AAAAigAAAAAAAAAAAAAAAAAAAAAAAAD6AAAAAAAIAQk9AAAAAAAAAAAAAAAAAAAAABMA9wAAAAYiACkAAAAAAAAAAAD///8AAPkAAAAkAAAAAACiAGxpbWl0YW739/f3Y2VzdG9yc2lSemUAAAAmAKMAAP9kZQBi4nUxZ2xvZ2ZpbGUAJgD/AA==" | base64 --decode > tx_pool-bbe7bc3c2a6d0cb1a21c48506cd16fdb807ae3cf.crash
$ FUZZ=tx_pool ./src/test/fuzz/fuzz tx_pool-bbe7bc3c2a6d0cb1a21c48506cd16fdb807ae3cf.crash
INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 1136977254
INFO: Loaded 1 modules   (572973 inline 8-bit counters): 572973 [0x563877bfae70, 0x563877c86c9d), 
INFO: Loaded 1 PC tables (572973 PCs): 572973 [0x563877c86ca0,0x563878544f70), 
/workdir/fuzz_bins/fuzz_libfuzzer: Running 1 inputs 1 time(s) each.
Running: /workdir/crashes/crash-bbe7bc3c2a6d0cb1a21c48506cd16fdb807ae3cf
test/fuzz/tx_pool.cpp:114 Finish: Assertion `entry.GetModFeesWithDescendants() > 0' failed.
==1877== ERROR: libFuzzer: deadly signal
    #0 0x5638764b4c88 in __sanitizer_print_stack_trace (/workdir/fuzz_bins/fuzz_libfuzzer+0x14b4c88) (BuildId: 7fbfcc32a58adde3cb3dcfe8229731b5bb30d71d)
    #1 0x56387648c04c in fuzzer::PrintStackTrace() crtstuff.c
    #2 0x563876471e67 in fuzzer::Fuzzer::CrashCallback() crtstuff.c
    #3 0x7f168cc0850f  (/lib/x86_64-linux-gnu/libc.so.6+0x3c50f) (BuildId: 8a1bf172e710f8ca0c1576912c057b45f90d90d8)
    #4 0x7f168cc560fb  (/lib/x86_64-linux-gnu/libc.so.6+0x8a0fb) (BuildId: 8a1bf172e710f8ca0c1576912c057b45f90d90d8)
    #5 0x7f168cc08471 in raise (/lib/x86_64-linux-gnu/libc.so.6+0x3c471) (BuildId: 8a1bf172e710f8ca0c1576912c057b45f90d90d8)
    #6 0x7f168cbf24b1 in abort (/lib/x86_64-linux-gnu/libc.so.6+0x264b1) (BuildId: 8a1bf172e710f8ca0c1576912c057b45f90d90d8)
    #7 0x5638778c6167 in assertion_fail(std::basic_string_view<char, std::char_traits<char>>, int, std::basic_string_view<char, std::char_traits<char>>, std::basic_string_view<char, std::char_traits<char>>) check.cpp
    #8 0x56387682def1 in (anonymous namespace)::Finish(FuzzedDataProvider&, (anonymous namespace)::MockedTxPool&, Chainstate&) tx_pool.cpp
    #9 0x563876832e79 in (anonymous namespace)::tx_pool_fuzz_target(Span<unsigned char const>) tx_pool.cpp
    #10 0x5638768972c7 in LLVMFuzzerTestOneInput fuzz.cpp
    #11 0x563876473334 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) crtstuff.c
    #12 0x56387645c263 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) crtstuff.c
    #13 0x563876461e86 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) crtstuff.c
    #14 0x56387648c9d6 in main crtstuff.c
    #15 0x7f168cbf36c9  (/lib/x86_64-linux-gnu/libc.so.6+0x276c9) (BuildId: 8a1bf172e710f8ca0c1576912c057b45f90d90d8)
    #16 0x7f168cbf3784 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x27784) (BuildId: 8a1bf172e710f8ca0c1576912c057b45f90d90d8)
    #17 0x563876456cd0 in _start (/workdir/fuzz_bins/fuzz_libfuzzer+0x1456cd0) (BuildId: 7fbfcc32a58adde3cb3dcfe8229731b5bb30d71d)

NOTE: libFuzzer has rudimentary signal handlers.
      Combine libFuzzer with AddressSanitizer or similar for better crash reports.
SUMMARY: libFuzzer: deadly signal

@ariard
Copy link

ariard commented Nov 28, 2023

Even though we don't have cluster limits, we are able to treat these transactions as having as having a maximum cluster size of 2.

I don’t think adopting cluster limits instead of max ancestors / descendants limits change anything for V3 packages. What matters is the overall weight limit of the package (4000 WU for a child), as this limit draws an anti-pinning security bound on the lowest off-chain payment that one can afford to burn as an absolute fee . Limit to be evaluated in function of network mempools congestion. Removing the absolute fee replacement rules I think would simplify current pinning analysis - though sounds this is beyond the scope of this proposal.

@glozow
Copy link
Member Author

glozow commented Nov 29, 2023

CI is green. I've added a rule for maximum sigops and updated the doc.
(cc @sdaftuar)

@sdaftuar
Copy link
Member

sdaftuar commented Feb 9, 2024

ACK 29029df

In addition to my code review and running a slightly earlier version of this branch on a year's worth of historical data, I also did a bunch of mutation testing to confirm code coverage for every component of the new logic (found one block that I think is unreachable, which I commented on above, but doesn't seem like an issue either way).

Copy link
Member

@instagibbs instagibbs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like if I revert TX_MAX_STANDARD_VERSION to 3, all unit/functional tests pass?

ACK 29029df modulo that

@glozow
Copy link
Member Author

glozow commented Feb 9, 2024

looks like if I revert TX_MAX_STANDARD_VERSION to 3, all unit/functional tests pass?

I suppose my change at https://github.com/bitcoin/bitcoin/pull/28948/files#diff-cc0c6a9039a1c9fe38b8a21fe28391fffbac9b8531dfda0f658919a9f74b46baR793 is partially to blame. I made it TX_MAX_STANDARD_VERSION + 1 instead of 3, thinking it might be easier to flip in the future.

I'm working on a followup to address the comments, and can incorporate if I need to retouch.

@achow101
Copy link
Member

ACK 29029df

@achow101 achow101 merged commit 7143d43 into bitcoin:master Feb 10, 2024
Copy link
Member

@ismaelsadeeq ismaelsadeeq left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Post merge ACK

@ariard
Copy link

ariard commented Feb 11, 2024

@sdaftuar @achow101 @instagibbs

Congrats guys to ACK and merge a +1100 LoC PR a Friday evening with still standing objections on the very heart of the PR.
What a lack of professionalism…Still my pleasure to break v3 anti-pinning mitigations in the future, with a pinning toolkit.
If script kiddies don’t make it before me and go to savage the LN ecosystem, who would have naively thought they’re safe.

@glozow

I think there is medium difficulty follow-up to this PR to make pinning far harder to exploit.
Namely a) moving 1000 kb limit on the client-side and b) making it cover the whole package, not the child only.
This can be embedded in bip331 changes or in any future p2p extensions.
That way LN channels could adjust their pinning exposure in the level of trust they have towards their counterparties.
And L2s won’t have to bargain the 1000 kb limit that doesn’t fit their use-case.

@glozow glozow deleted the v3-policy branch February 12, 2024 12:02
@glozow glozow mentioned this pull request Feb 12, 2024
@instagibbs
Copy link
Member

@ariard

I'll let people decide for themselves our professionalism or lack thereof.

I asked hundreds of comments ago to stop
using this PR as a venue for debating the finer points of the LN spec, obviously unheeded.
There's a BOLT repo, which is an appropriate place on github to discuss these things.

You are also free to lobby different projects to not use these primitives moving forward and push
whatever solutions you think are necessary, even consensus changes. Meanwhile people
who find this useful and want to, they can use it, including use-cases which fall
outside of your sphere of interest.

As was noted already: We're nearing feature freeze, which means if we miss the window the rebasing would continue
on for yet another release cycle. It's left as non-standard which leaves some room for
finding a slightly better local optimum, perhaps informed by @sdaftuar's investigating work
and more feedback.

To end a positive note, I'm looking forward to iterative improvements,
hoping to make things more generally useful for systems deployed today,
improve incentive compatibility, without policing their use-cases:

  1. 1p1c relay: p2p: opportunistically accept 1-parent-1-child packages #28970
  2. sibling eviction for v3: policy: enable sibling eviction for v3 transactions #29306
  3. limited package rbf: Cluster size 2 package rbf #28984
  4. cluster mempool: Cluster mempool implementation #28676

and leveraging a totally ordered mempool in interesting ways beyond.

fanquake added a commit that referenced this pull request Feb 13, 2024
6b161cb [test] second child of a v3 tx can be replaced individually (glozow)
5c998a6 [refactor] use MAX_PUBKEYS_PER_MULTISIG instead of magic numbers in test (glozow)
a934642 [test] PackageV3Checks with inheritance violation in mempool ancestor (glozow)
63b62e1 [doc] fix docs and comments from v3 (glozow)

Pull request description:

  Addresses final comments from #28948:
  - thread at #28948 (comment), using ismaelsadeeq@87fc7f0 with some modifications
  - #28948 (comment)
  - #28948 (comment)
  - #28948 (comment)
  - #28948 (comment)
  - #28948 (comment)
  - #28948 (comment)
  - #28948 (comment)
  - #28948 (comment)

ACKs for top commit:
  instagibbs:
    ACK 6b161cb
  sdaftuar:
    utACK 6b161cb

Tree-SHA512: 584fce7810f4d704ee6ab51fdc7d42bab342140cae3d076f89b5e1966dd1dd8293cb25b3121e41a4dcd65f9d4a735102b9ab2e90f98aa770b84e21f4d35d63d3
@ariard
Copy link

ariard commented Feb 15, 2024

@instagibbs

I maintain your review and the ones of @sdaftuar and @achow101 are lacking professionalism.

For a PR which is called "anti-pinning" (it's all in the title), there has been no substantial answer on my concerns that it does not address neither rbf rule3 pinning (the 1000 vbyte rule limit offers to give a way, see "loophole" scenario exposed above), neither topological-based scenario at the tx-relay network level (NTA one), and that actually "hard" limits are a source of "no-one-fits-all" bargainging for use-cases operations (as we have seen with mempoolfullrbf).

On the review process itself and calling to stop using this PR to debate the robustness of the mitigation, this is insulting and a flagrant proof of your lack of system engineering culture. When software engineers are designing new cryptopgraphic primtives or kernel mechanism (e.g linux cgroups), they are of course considering how those mechanisms are used by downstream use-cases.

The current design and review mentality you're exposing and I understanding you're advocating has been proven multiple times a source of potential security failures (e.g CVE-2021-31876 or the 1 CSV added on non-anchor outputs commitment transactions as original carve out mechanism semantics were unclear). Beyond, you're falling in the trap of the protocol design process communication silos I've been pointong out before and I aimed to address in the past with the "L2s Onchain Support IRC Workshop"

On the remark that people who find this useful and want to, they can use it, which I understanding that LN maintainers or anyone else (e.g exchange batching) are free to level up their bitcoin use-cases with nversion=3 semantics. Sadly one has to be very realistic and observe that LN security track record is not great. When you're doing adversarial reviews on financial software you cannot trust the incentives of the developers "in defense", as they have always a moral interest to claim attack X or attack Y is "mitigated", playing some kind of security theater.

Finally, on the call of approaching feature freeze, this is neither a convicining argument in my eyes. In critical system design, development and maintenance there is a social phenomena called process fatigue. Namely, when engineers are getting in front of some deadlines after lengthy work cycles, they are rushing to deliver a technical project, provoking downstream major failure (e.g known historic example is Space Shuttle Challenger disaster).

There is no magical reason that Bitcoin Core review process is not affected by such phenomena of "process fatigue" and as such when there is no review consensus on security-first contribution in one of the part of the codebaee like the mempool, practical wisdom would recommend to delay it to the next release cycle. This is better than yet-another "poor hot fixes" in production were we have to mobilize half-of the engineering, vendors and users ecosystem time, and jeopardize their funds.

To conclude your answer is a poor one sustaining my precedent accusation of lacking professionalism. You're still free to communicate any example or demonstration infrastructure running with v3 semantics that I can target with demo pinning attacks. And as such back up your Bitcoin Core technical review with your credibility and reputation, in a skin-in-the-game approach.

@ariard
Copy link

ariard commented Feb 15, 2024

@glozow are you still interested if I do a proof-of-concept bitcoin core branch or idea draft of a) moving the “package limit” on the use-case side and b) make it dynamic (in the limit of absolute core DoS robustness) ? I think it can be backup in future bip331 changes. v3 is your work and this is your PR making anti-pinning claims after-all.

@ariard
Copy link

ariard commented Feb 19, 2024

are you still interested if I do a proof-of-concept bitcoin core branch or idea draft of a) moving the “package limit” on the use-case side and b) make it dynamic (in the limit of absolute core DoS robustness) ? I think it can be backup in future bip331 changes. v3 is your work and this is your PR making anti-pinning claims after-all.

did it with #29448

@@ -694,6 +696,7 @@ libbitcoin_common_a_SOURCES = \
netbase.cpp \
net_permissions.cpp \
outputtype.cpp \
policy/v3_policy.cpp \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Considering the current code organisation, I can't see any reasons to include policy/v3_policy.cpp into the libbitcoin_common library. Both its symbols, i.e., SingleV3Checks and PackageV3Checks are referenced in the validation.cpp only. As such, policy/v3_policy.cpp inclusion into the libbitcoin_node library is enough.

Comment on lines +17 to +56
/** Helper for PackageV3Checks: Returns a vector containing the indices of transactions (within
* package) that are direct parents of ptx. */
std::vector<size_t> FindInPackageParents(const Package& package, const CTransactionRef& ptx)
{
std::vector<size_t> in_package_parents;

std::set<Txid> possible_parents;
for (auto &input : ptx->vin) {
possible_parents.insert(input.prevout.hash);
}

for (size_t i{0}; i < package.size(); ++i) {
const auto& tx = package.at(i);
// We assume the package is sorted, so that we don't need to continue
// looking past the transaction itself.
if (&(*tx) == &(*ptx)) break;
if (possible_parents.count(tx->GetHash())) {
in_package_parents.push_back(i);
}
}
return in_package_parents;
}

/** Helper for PackageV3Checks, storing info for a mempool or package parent. */
struct ParentInfo {
/** Txid used to identify this parent by prevout */
const Txid& m_txid;
/** Wtxid used for debug string */
const Wtxid& m_wtxid;
/** nVersion used to check inheritance of v3 and non-v3 */
decltype(CTransaction::nVersion) m_version;
/** If parent is in mempool, whether it has any descendants in mempool. */
bool m_has_mempool_descendant;

ParentInfo() = delete;
ParentInfo(const Txid& txid, const Wtxid& wtxid, decltype(CTransaction::nVersion) version, bool has_mempool_descendant) :
m_txid{txid}, m_wtxid{wtxid}, m_version{version},
m_has_mempool_descendant{has_mempool_descendant}
{}
};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: This stuff can be placed into a anonymous namespace.

See SF.22: Use an unnamed (anonymous) namespace for all internal/non-exported entities.

achow101 added a commit that referenced this pull request Jun 7, 2024
30a0113 [doc] update bips.md for 431 (glozow)
9dbe6a0 [test] wallet uses CURRENT_VERSION which is 2 (glozow)
539404f [policy] make v3 transactions standard (glozow)
052ede7 [refactor] use TRUC_VERSION in place of 3 (glozow)

Pull request description:

  Make `nVersion=3` (which is currently nonstandard on mainnet) standard.

  Note that we will treat these transactions as Topologically Restricted Until Confirmation (TRUC). Spec is in BIP 431 and implementation is in #28948, #29306, and #29873

  See #27463 for overall project tracking, and #29319 for information about relevance to cluster mempool.

ACKs for top commit:
  sdaftuar:
    utACK 30a0113
  achow101:
    ACK 30a0113
  instagibbs:
    utACK 30a0113
  murchandamus:
    ACK 30a0113
  ismaelsadeeq:
    ACK 30a0113 🛰️

Tree-SHA512: 2a4aec0442c860e792a061d83e36483c1f1b426f946efbdf664c8db97a596e498b535707e1d3a900218429486ea69fd4552e3d476526a6883cbd5556c6534b48
@bitcoin bitcoin locked and limited conversation to collaborators Mar 1, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.