Skip to content

Conversation

glozow
Copy link
Member

@glozow glozow commented Aug 3, 2022

We have pretty thorough documentation of our RBF policy in doc/policy/mempool-replacements.md. It enumerates each rule with several sentences of rationale. Also, each rule pretty much has its own function (3 and 4 share one), with extensive comments. The doc states explicitly that our rules are similar but differ from BIP125, and contains a record of historical changes to RBF policy.

We should not use "BIP125" as synonymous with our RBF policy because:

This PR removes all non-signaling mentions of BIP125 (if people feel strongly, we can remove all mentions of BIP125 period). It may be useful to refer to the concept of "tx opts in to RBF if it has at least one nSequence less than (0xffffffff - 1)" as "BIP125 signaling" because:

  • It is succint.
  • It has already been widely marketed as BIP125 opt-in signaling.
  • Our API uses it when referring to signaling (e.g. getmempoolentry["bip125-replaceable"] and wallet error message "not BIP 125 replaceable"). Changing those is more invasive.
  • If/when we have other ways to signal in the future, we can disambiguate them this way. See policy: nVersion=3 and Package RBF #25038 which proposes another way of signaling, and where I pulled these commits from.

Alternatives:

  • Changing our policy to match BIP125. This doesn't make sense as, for example, we would have to remove the requirement that a replacement tx has a higher feerate (Rule 6).
  • Changing BIP125 to match what we have. This doesn't make sense as it would be a significant change to a BIP years after it was finalized and already used as a spec to implement RBF in other places.
  • Document our policy as a new BIP and give it a number. This might make sense if we don't expect things to change a lot, and can be done as a next step.

Our RBF policy is different from the rules specified in BIP125 (refer to
doc/policy/mempool-replacements.md instead), and will continue to
diverge with package RBF.  Keep references to BIP125 sequence number,
since that is still useful and correct.

-BEGIN VERIFY SCRIPT-
ren() { sed -i "s:\<$1\>:$2:g" $(git grep -l "\<$1\>" ./src ./test); }

ren m_allow_bip125_replacement m_allow_replacement
ren allow_bip125_replacement allow_replacement
ren MAX_BIP125_REPLACEMENT_CANDIDATES MAX_REPLACEMENT_CANDIDATES
-END VERIFY SCRIPT-
@fanquake fanquake added the Docs label Aug 3, 2022
@glozow glozow force-pushed the 2022-08-bip125-signal-only branch from f969ca9 to fbdbff9 Compare August 3, 2022 14:48
@petertodd
Copy link
Contributor

  • Changing BIP125 to match what we have.

Another alternative: delete the details from BIP125. The only interesting thing in it is the opt-in part. The rest was just suggestions for how to implement RBF that have turned out to not be ideal. It's not consensus after all.

@glozow
Copy link
Member Author

glozow commented Aug 3, 2022

Another alternative: delete the details from BIP125. The only interesting thing in it is the opt-in part. The rest was just suggestions for how to implement RBF that have turned out to not be ideal. It's not consensus after all.

Sure, that makes sense. Though we would still want these changes, because then saying "BIP125 Rule 5" would be quite confusing.

@michaelfolkson
Copy link

I think it makes sense to remove references to BIP125 in the code (e.g. variable names). But I don't know why references to BIP125 have to be removed from the comments. The comments can (and should) refer to BIP125 e.g. BIP125 rule X and explain what differences there are between a BIP125 rule and how it is implemented in Core.

BIP wise, ideally we'd have a replacement BIP or a BIP125 version 2 at some point in future and then the code comments could refer to that. I think the maintaining a historical record argument will prevent widespread changes to BIP125 itself.

@ariard
Copy link

ariard commented Aug 3, 2022

Concept ACK

Changing BIP125 to match what we have. This doesn't make sense as it would be a significant change to a BIP years after it was finalized and already used as a spec to implement RBF in other places.

I think I would still recommend to update BIP125 with a historical note linking to this PR if it's accepted to provide awareness to future readers that the replacement policy described by BIP125 is not the one effectively implemented by Core.

Document our policy as a new BIP and give it a number. This might make sense if we don't expect things to change a lot, and can be done as a next step.

I don't know about documenting the Core mempool policy as a BIP. I'm quite sure I've argued for that practice in the past though it was at a time we didn't have doc/policy consistently and rigorously describing policy, in an optic to be consumed by wallet or second-layers developers. Now, we would have two sets of document to keep in-sync. There is also the fact that raising our policy documentation as a BIP "best" standard could give an inaccurate perception across the ecosystem that Core policy is "authoritative" and discourage experimentation or divergent policy deployment by other Bitcoin softwares. Of course, I think P2P extensions still deserve a BIP to achieve an unified P2P network. No strong opinions here.

@michaelfolkson
Copy link

michaelfolkson commented Aug 3, 2022

I'm quite sure I've argued for that practice in the past though it was at a time we didn't have doc/policy consistently and rigorously describing policy, in an optic to be consumed by wallet or second-layers developers.

Documentation in this repo is often an afterthought despite best intentions. My concern would be that without a (relatively) static formal document outlining what rules are being followed and why the rationale for policy changes won't be clear. Regular unscrutinized changes and tweaks will become the norm and make them much harder to monitor and analyze.

There is also the fact that raising our policy documentation as a BIP "best" standard could give an inaccurate perception across the ecosystem that Core policy is "authoritative" and discourage experimentation or divergent policy deployment by other Bitcoin softwares.

Other than on consensus no BIP is attempting to be authoritative. As the dominant implementation on the network Core is setting the default policy rules for the vast majority of nodes and attempting to maintain compatibility with a BIP means more scrutiny and an alternative point of reference to compare to. (The recent exercise of comparing current Core code to BIP125 for example was much more rigorous than it would have been if comparing Core code to an internal Core doc that may or may not be up to date.)

edit: Maybe it is acceptable (or even desirable) for the RBF rules to be in a constant state of flux. There have been multiple ideas/proposals on changing the RBF rules in recent months. In that case a new BIP clearly does not make sense. But if the aspiration is to have a set of RBF rules with strong rationales that don't regularly change, moving from having a BIP to internal Core documentation seems like an erosion of scrutiny to me.

@murchandamus
Copy link
Contributor

Concept ACK

It would feel a bit strange to refer to the rules by № if there was no documentation that comprehensively enumerates them as such anymore. If e.g. the details were deleted from BIP125, should the rules perhaps be listed in code documentation? Otherwise, a new BIP to supersede BIP125 and document the actual behavior would sound useful.

@glozow
Copy link
Member Author

glozow commented Aug 3, 2022

I don't know why references to BIP125 have to be removed from the comments. The comments can (and should) refer to BIP125 e.g. BIP125 rule X and explain what differences there are between a BIP125 rule and how it is implemented in Core.

It would feel a bit strange to refer to the rules by № if there was no documentation that comprehensively enumerates them as such anymore. If e.g. the details were deleted from BIP125, should the rules perhaps be listed in code documentation? Otherwise, a new BIP to supersede BIP125 and document the actual behavior would sound useful.

Not sure if this is widely known, but doc/policy/mempool-replacements.md lists each rule with several sentences of rationale. Each rule pretty much has its own function (3 and 4 share one), with extensive comments. The doc states explicitly that our rules are similar but differ from BIP125, and contains a record of historical changes to RBF policy. Comments about what code doesn't do and referencing BIPs it doesn't implement is just clutter and confusing.

@michaelfolkson
Copy link

Maybe it is acceptable (or even desirable) for the RBF rules to be in a constant state of flux. There have been multiple ideas/proposals on changing the RBF rules in recent months. In that case a new BIP clearly does not make sense. But if the aspiration is to have a set of RBF rules with strong rationales that don't regularly change, moving from having a BIP to internal Core documentation seems like an erosion of scrutiny to me.

@glozow: Perhaps you can comment on the above then. There is a difference between cross checking code with a formal static doc like a BIP and updating regularly changing documentation (assuming one remembers to). Otherwise there is no argument to BIP or spec anything in Core whatsoever (except maybe consensus changes).

@murchandamus
Copy link
Contributor

Not sure if this is widely known, but doc/policy/mempool-replacements.md lists each rule with several sentences of rationale.

Oh great! It wasn't clear to me from the first post's description that this already existed—you could perhaps highlight that more in the first post. Given that most of the nodes on the network are Bitcoin Core, I'm not sure whether a BIP is strictly needed in that case then, but a BIP might still be a bit easier to discover. Since it's bad to have two diverging authoritative sources, I would agree that a BIP should wait until the rules aren't expected to change much anymore.

@darosior
Copy link
Member

darosior commented Aug 4, 2022

Concept ACK

@glozow
Copy link
Member Author

glozow commented Aug 4, 2022

Oh great! It wasn't clear to me from the first post's description that this already existed—you could perhaps highlight that more in the first post.

Thanks, done!

if the aspiration is to have a set of RBF rules with strong rationales that don't regularly change, moving from having a BIP to internal Core documentation seems like an erosion of scrutiny to me. There is a difference between cross checking code with a formal static doc like a BIP and updating regularly changing documentation (assuming one remembers to). Otherwise there is no argument to BIP or spec anything in Core whatsoever (except maybe consensus changes).

Putting code documentation in a BIP does not inherently make it more correct or up-to-date. There is a pretty large logical leap here from "docs should change when code changes" to "there's no point in writing a spec." Feel free to open a PR if you see a divergence between code and documentation, which is what this PR is doing.

@michaelfolkson
Copy link

@glozow: You're not answering the question on whether the aim is RBF rules that are constantly in flux which lends itself to internal Core documentation or whether the aim is to have a set of static RBF rules with clear rationales that are universally well understood by the ecosystem which lends itself to a BIP.

I'll write something up for the mailing list as this desire to move from BIPs to internal Core documentation has concerned me for a while and it won't be resolved on this PR.

If there is a purge of all references to RBF rules being BIPed in this repo then there are additional references to BIP125 in the wallet and test code.

I will probably draft a BIP assuming I can work out if and when we have a set of replacement RBF rules for BIP 125 that are intended to be static. This is not a criticism of that work, it is important, challenging and difficult to follow. But when security of L2 protocols are depending on it (whether we like it or not) and it is possible to achieve I do think a set of static RBF rules with clear rationales is what we should be aiming for.

Our RBF policy is different from the rules specified in BIP125. For
example, the BIP does not mention Rule 6, and our Rule 4 uses the
(configurable) incremental relay feerate (distinct from the
minimum relay feerate). Those interested in our policy should refer to
doc/policy/mempool-replacements.md instead. These rules may also
continue to diverge with package RBF and other RBF improvements. Keep
references to the BIP125 signaling wrt sequence numbers, since that is
still correct and widely used. It is helpful to refer to this as "BIP125
signaling" since it is unambiguous and succint, especially if we have
multiple ways to signal replaceability in the future.

The rule numbers in doc/policy/mempool-replacements.md correspond
largely to those of BIP 125, so we can still refer to them like "Rule 5."
@glozow glozow force-pushed the 2022-08-bip125-signal-only branch from fbdbff9 to 1dc03dd Compare August 4, 2022 15:56
@DrahtBot
Copy link
Contributor

DrahtBot commented Aug 5, 2022

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #25718 (net: Add -allowinbound config option by fjahr)
  • #25038 (policy: nVersion=3 and Package RBF by glozow)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@instagibbs
Copy link
Member

Given the document doc/policy/mempool-replacements.md already exists(and explains the relationship to bip125), concept ACK

@michaelfolkson
Copy link

Concept ACK. I have reservations but seems there are good reasons not to BIP RBF rules anymore and seems I was in the minority anyway.

Copy link
Contributor

@t-bast t-bast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concept ACK

@darosior
Copy link
Member

ACK 1dc03dd

This PR removes all non-signaling mentions of BIP125 (if people feel strongly, we can remove all mentions of BIP125 period).

I'd be in favour of this. BIP125 caused too much confusion. But i don't feel strongly, the signaling part will hopefully become less important as we go and could be slowly phased out.

Copy link

@ariard ariard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 1dc03dd

This PR removes all non-signaling mentions of BIP125 (if people feel strongly, we can remove all mentions of BIP125 period).

I would favor too slow phase out of even the signaling part of BIP125.

@@ -333,7 +333,7 @@ RPCHelpMan sendmany()
{"address", RPCArg::Type::STR, RPCArg::Optional::OMITTED, "Subtract fee from this address"},
},
},
{"replaceable", RPCArg::Type::BOOL, RPCArg::DefaultHint{"wallet default"}, "Allow this transaction to be replaced by a transaction with higher fees via BIP 125"},
{"replaceable", RPCArg::Type::BOOL, RPCArg::DefaultHint{"wallet default"}, "Signal that this transaction can replaced by a transaction (BIP 125)"},
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if you should update the argument descriptions similarly in TransactionDescriptionString() (L420 in src/wallet/rpc/transactions.cpp) and in MempoolEntryDescription() (L258 in src/rpc/mempool.cpp)

If yes, can be done as a follow-up.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, done in #25902

Copy link
Contributor

@t-bast t-bast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 1dc03dd

@fanquake fanquake merged commit c5f0cbe into bitcoin:master Aug 22, 2022
Copy link
Member

@maflcko maflcko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the grammar right, as I can't parse some sentences?

@glozow glozow deleted the 2022-08-bip125-signal-only branch August 22, 2022 13:46
maflcko pushed a commit to bitcoin-core/gui that referenced this pull request Aug 22, 2022
375ebad fixups for BIP125 doc cleanup (glozow)

Pull request description:

  Followups from #25775:

  - bitcoin/bitcoin#25775 (comment)
  - bitcoin/bitcoin#25775 (comment)
  - bitcoin/bitcoin#25775 (comment)
  - bitcoin/bitcoin#25775 (comment)

ACKs for top commit:
  t-bast:
    ACK bitcoin/bitcoin@375ebad
  ariard:
    ACK 375ebad

Tree-SHA512: 66f240a020a2f6994a3b3bcce446bc07655b5714eab18aac4a3223d35226e57902c475fc60a57e3511b2f176741d6c93d7ebfe90574793813f90ceca33d44669
sidhujag pushed a commit to syscoin/syscoin that referenced this pull request Aug 22, 2022
@fanquake fanquake added this to the 24.0 milestone Sep 15, 2022
@bitcoin bitcoin locked and limited conversation to collaborators Sep 15, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.