-
Notifications
You must be signed in to change notification settings - Fork 37.7k
p2p: Remove BIP61 reject messages #15437
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Not for 0.18.0, obviously |
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. ConflictsReviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first. |
fa68bce
to
fa1fb01
Compare
Concept ACK -134 lines: nice! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Mild concept ACK.
However, I think this PR should be preceded by discussion on the dev mailing list. It's possible that organizations are using bitcoind as their border node, and broadcast transactions from internal applications via bitcoind. Perhaps they rely on REJECT messages? @laanwj has also commented that he found the messages indispensable when writing client software: #13134 (comment).
Mild concept ACK-- the loc reduction makes a case for it. If someone were relying on this, I'd really encourage them to not do so, unrelated to if this PR were going in or not. :) The ability to use it during testing/debugging, makes for a case against it. |
Indeed, if someone relies on this in production, I'd rather have them to switch to safer ways of achieving their goal. So imo another reason to remove the "feature" |
I think there's a difference between encouraging users to change behaviour and pulling a feature out from under them. We really don't have a good picture of how users use our code, so I'd always err on the side of caution when deprecating features. I'm going to modify my mild concept ACK with a merge-NACK-until-this-has-been-more-widely-discussed. We're probably fine, but it seems to me that any P2P change that was added to Bitcoin Core with a BIP should be removed after public discussion. Here's how I expect it'll go: someone will post a mail to the bitcoin-core mailing list saying "I propose to remove REJECT messages in Bitcoin Core v0.19", no-one will respond, and then we'll remove them. I'm happy to send that post if you'd prefer not to. |
I am happy to write release notes and a mailing list post, I just didn't come around to do this yet. The previous release 0.18.0 already disabled BIP61 by default, so this shouldn't come off as a massive surprise. Also, I wouldn't be surprised if the entity that uses the feature isn't subscribed to the mailing list. Could Bitcoin Optech send out a mail as well? |
Thanks Marco. Sounds good. I'm very happy to review the post/release notes if you want.
Agree. 👍 for staged deprecation of features!
We'd certainly include it in our next newsletter if there was a post on the mailing list. |
Added a commit with the draft for the mailing list post as requested by @jnewbery |
1bbf4ca
to
91d1930
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Email draft looks good Marco.
doc/release-notes-15437.md
Outdated
relies on this feature and you can not use any of the recommended alternatives: | ||
|
||
* Testing or debugging of implementations of the Bitcoin P2P network protocol | ||
should be done by inspecting the reject messages that are produced by a |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should this say "inspecting the logs ..."?
Might be useful to have an example log here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess an example log depends on the message type that was rejected "version", "tx" or "block" and the reason for rejection. I'd rather not add a specific example, since the logs are not guaranteed to be stable (only used for testing and debugging)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it'd still be helpful, even with a caveat that log messages are not guaranteed to be stable. Obviously up to you!
f22e760
to
3458603
Compare
Moved to 0.20.0 milestone for now |
fa00728
to
fab8741
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A few minor comments inline. Otherwise ACK fab8741
(for merging after the 0.19 branch)
Thanks for the clarifications. ACK once the test comment is corrected. |
fab8741
to
fa25f43
Compare
utACK fa25f43 |
I'm still not 100% convinced that I like getting rid of BIP61 conceptually, but apparently everyone wants it, code review ACK fa25f43. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code review ACK fa25f43
I don't know much about the BIP process, but should https://github.com/bitcoin/bips/blob/master/bip-0061.mediawiki have some kind of "Status: Obsolete" tag or a separate warning that it shouldn't be relied on?
@@ -179,10 +179,6 @@ Allow DNS lookups for \fB\-addnode\fR, \fB\-seednode\fR and \fB\-connect\fR (def | |||
Query for peer addresses via DNS lookup, if low on addresses (default: 1 | |||
unless \fB\-connect\fR used) | |||
.HP | |||
\fB\-enablebip61\fR |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think these man files get generated from -help, so there's no need to make the same updates repeatedly
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think generally it is fine to update them either in the same commit, or wait for the script to bump them later.
Here, I made the changes directly, so that git grep enablebip61
doesn't return these.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes it's okay to do this (but also to not do this, for ex. because it can cause easily avoidable merge conflicts, as these are would-be hotspots), similar for updating the bitcoin_en
translation
fa25f43 p2p: Remove BIP61 reject messages (MarcoFalke) Pull request description: Reject messages (BIP 61) appear in the following settings: * Parsing of reject messages (in case `-debug=net` is set, off by default). This has only been used for a single `LogPrint` call for several releases now. Such logging is completely meaningless to us and should thus be removed. * The sending of reject messages (in case `-enablebip61` is set, off by default). This can be used to debug a node that is under our control. Instead of hacking this debugging into the p2p protocol, it could be more easily achieved by parsing the debug log. (Use `-printtoconsole` to have it as stream, or read from the `debug.log` file like our python function `assert_debug_log` in the test framework does) Having to maintain all of this logic and code to accommodate debugging, which can be achieved by other means a lot easier, is a burden. It makes review on net processing changes a lot harder, since the reject message logic has to be carried around without introducing any errors or DOS vectors. ACKs for top commit: jnewbery: utACK fa25f43 laanwj: I'm still not 100% convinced that I like getting rid of BIP61 conceptually, but apparently everyone wants it, code review ACK fa25f43. ryanofsky: Code review ACK fa25f43 Tree-SHA512: daf55254202925e56be3d6cfb3c1c804e7a82cecb1dd1e5bd7b472bae989fd68ac4f21ec53fc46751353056fd645f7f877bebcb0b40920257991423a3d99e0be
fa25f43 p2p: Remove BIP61 reject messages (MarcoFalke) Pull request description: Reject messages (BIP 61) appear in the following settings: * Parsing of reject messages (in case `-debug=net` is set, off by default). This has only been used for a single `LogPrint` call for several releases now. Such logging is completely meaningless to us and should thus be removed. * The sending of reject messages (in case `-enablebip61` is set, off by default). This can be used to debug a node that is under our control. Instead of hacking this debugging into the p2p protocol, it could be more easily achieved by parsing the debug log. (Use `-printtoconsole` to have it as stream, or read from the `debug.log` file like our python function `assert_debug_log` in the test framework does) Having to maintain all of this logic and code to accommodate debugging, which can be achieved by other means a lot easier, is a burden. It makes review on net processing changes a lot harder, since the reject message logic has to be carried around without introducing any errors or DOS vectors. ACKs for top commit: jnewbery: utACK fa25f43 laanwj: I'm still not 100% convinced that I like getting rid of BIP61 conceptually, but apparently everyone wants it, code review ACK fa25f43. ryanofsky: Code review ACK fa25f43 Tree-SHA512: daf55254202925e56be3d6cfb3c1c804e7a82cecb1dd1e5bd7b472bae989fd68ac4f21ec53fc46751353056fd645f7f877bebcb0b40920257991423a3d99e0be
We should announce that this has been merged on the bitcoin-dev mailing list, to close the https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2019-March/016701.html thread. |
b1b0cfe test: Remove REJECT message code (Hennadii Stepanov) Pull request description: We no longer use REJECT p2p message: - bitcoin#15437 - bitcoin#17004 ACKs for top commit: theStack: ACK bitcoin@b1b0cfe (nice dead code find) Tree-SHA512: 0a662b282e921c3991aeb15f54d077837f1ef20bc2e3b0b35117bb97a21d1bd1c3e21458e5c18ba0ca02030d559e3e8e74dbd3d3e2b46dbe7bede550948c3b55
Summary: >Reject messages (BIP 61) appear in the following settings: > - Parsing of reject messages (in case -debug=net is set, off by default). This has only been used for a single LogPrint call for several releases now. Such logging is completely meaningless to us and should thus be removed. > - The sending of reject messages (in case -enablebip61 is set, off by default). This can be used to debug a node that is under our control. Instead of hacking this debugging into the p2p protocol, it could be more easily achieved by parsing the debug log. (Use -printtoconsole to have it as stream, or read from the debug.log file like our python function assert_debug_log in the test framework does) > > Having to maintain all of this logic and code to accommodate debugging, which can be achieved by other means a lot easier, is a burden. It makes review on net processing changes a lot harder, since the reject message logic has to be carried around without introducing any errors or DOS vectors. Backport of Core [[bitcoin/bitcoin#15437 | PR15437]] Test Plan: `ninja && ninja check-all` Reviewers: O1 Bitcoin ABC, #bitcoin_abc, deadalnix Reviewed By: O1 Bitcoin ABC, #bitcoin_abc, deadalnix Subscribers: deadalnix Differential Revision: https://reviews.bitcoinabc.org/D7981
…ic + BIP61 removal. 8e9f2bc Net_processing: Minimize tier two messages search time and improve code. (furszy) 56a78fc Clean few compiler warnings in blockassembler, validation, rpcwallet and evo_deterministicmns_tests files. (furszy) c398797 Refactor: Name GetDataMsg enum and replace ppszTypeName array for direct GetCommand call. (furszy) f99c5ed Move-only: IsTransactionInChain functions moved inside zpivchain.cpp and made static as them are only used there. (furszy) 2e1af2b Remove IsBlockHashInChain function. Only used inside IsTransactionInChain. (furszy) 38d49ea Document 'DisconnectOldProtocol' call rationale in block processing. (furszy) 4f62c04 Remove now unneeded `fAccepted` flag from ProcessNewBlock. (furszy) 0d4ba29 Move block spam filter check to BlockChecked (furszy) bd85440 Refactor: get rid off the error prone CValidationState argument in ProcessNewBlock. (furszy) 22b1757 validation: Proper validation state set for "out of order" check in CheckBlock. (furszy) 9606474 net_processing: Clean CheckBlockSpam code and add proper cs_main locks in State() call and mapBlockIndex access. (furszy) e6d2bd6 net_processing: Remove duplicate, and wrong, peer misbehaving due an invalid arriving block. (furszy) 6a5df01 p2p: Remove BIP61 reject messages (furszy) Pull request description: Continuation of #2463. Another error prone topic. Essentially focused in the following points: * Remove duplicate and incorrect peer misbehaving: -1) The peer misbehaving score set for an invalid arriving block is being performed inside `PeerLogicValidation::BlockChecked`. -2) The `CValidationState` param in `ProcessNewBlock` does not return the error nor the invalidity reason if the block is marked invalid during the activate best chain process (block connection), an empty `CValidationState` is set. The `BlockChecked` signal is used instead to notify the block invalidity reason. * Get rid off `CValidationState` param in `ProcessNewBlock` and use only `BlockChecked` (own version of ae22357) * Removal of BIP61 (bitcoin#15437 and bitcoin#18609). -- BIP61 could be disabled first by default before its removal like upstream did (or moved to its own standalone PR), but.. i don't think that worth to continue maintaining it, most likely no one is using it and if someone is actually using it, isn't good to have any piece of software depending on it. -- * Move block spam filter check to `BlockChecked`. Following the same peer-processing-logic / block-connection-logic division. Improves the spam protection as now blocks rejected during the connection process are going to pass through the spam filter check as well (before, only blocks rejected during `AcceptBlock` were passing through it). Plus, this improvement let me do some further cleanup and remove the extra `fAccepted` flag from `ProcessNewBlock`. * Create `GetDataMsg` enum and replace `ppszTypeName` array for direct `GetCommand` call. The array was duplicated with the `NetMsgType` constants ACKs for top commit: random-zebra: ACK 8e9f2bc Tree-SHA512: 362219e995c857ac9ad45119d0cad24b1afcb94c807d5af60d7008dd985c3b324b5f2938d60890f89d610d214f69f6a6d5ef12d2ce4e6016480ad984d00706c4
Reject messages (BIP 61) appear in the following settings:
Parsing of reject messages (in case
-debug=net
is set, off by default). This has only been used for a singleLogPrint
call for several releases now. Such logging is completely meaningless to us and should thus be removed.The sending of reject messages (in case
-enablebip61
is set, off by default). This can be used to debug a node that is under our control. Instead of hacking this debugging into the p2p protocol, it could be more easily achieved by parsing the debug log. (Use-printtoconsole
to have it as stream, or read from thedebug.log
file like our python functionassert_debug_log
in the test framework does)Having to maintain all of this logic and code to accommodate debugging, which can be achieved by other means a lot easier, is a burden. It makes review on net processing changes a lot harder, since the reject message logic has to be carried around without introducing any errors or DOS vectors.