-
Notifications
You must be signed in to change notification settings - Fork 37.7k
validation: Tidy up ValidationState interface #15921
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
bca1b5f
to
edcbe6c
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Concept ACK, looks like a good refactor and improvement to add ValidationState subclasses.
Concept ACK. I'd like to see I also somewhat think that ideally there should be a third state class which handles Corruption cases for 'system state' or something. This covers the notion that the issue is not the fault of the block or the transaction, it is an issue with our local state or implementation. Not sure it's worth the churn though, two is already a huge improvement. |
Concept ACK |
The current split is more about "what was being tested" not "what was at fault" -- that's easy to deal with via types, because you know what's being tested at compile time; but you don't know what's going to have been at fault at compile time, so I think dealing with that via the type system (ie as a third state class) wouldn't work. |
edcbe6c
to
4a51148
Compare
2b794dc
to
4b4d081
Compare
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. ConflictsReviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first. |
4b4d081
to
8d88c86
Compare
Handle this failure in the same way as all other failures: call Invalid() with the reasons for the failure.
92584fe
to
3004d5a
Compare
squashed fixup commits. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code review ACK 3004d5a. Just whitespace change and pure virtual destructor added since last review.
code review ACK 3004d5a. Also built & ran tests locally. |
Code review ACK 3004d5a . Only nit style change and pure virtual destructor added since my last review. |
3004d5a [validation] Remove fMissingInputs from AcceptToMemoryPool() (John Newbery) c428622 [validation] Remove unused first_invalid parameter from ProcessNewBlockHeaders() (John Newbery) 7204c64 [validation] Remove useless ret parameter from Invalid() (John Newbery) 1a37de4 [validation] Remove error() calls from Invalid() calls (John Newbery) 067981e [validation] Tidy Up ValidationResult class (John Newbery) a27a295 [validation] Add CValidationState subclasses (John Newbery) Pull request description: Carries out some remaining tidy-ups remaining after PR 15141: - split ValidationState into TxValidationState and BlockValidationState (commit from ajtowns) - various minor code style tidy-ups to the ValidationState class - remove the useless `ret` parameter from `ValidationState::Invalid()` - remove the now unused `first_invalid` parameter from `ProcessNewBlockHeaders()` - remove the `fMissingInputs` parameter from `AcceptToMemoryPool()`, and deal with missing inputs the same way as other errors by using the `TxValidationState` object. Tip for reviewers (thanks ryanofsky!): The first commit ("[validation] Add CValidationState subclasses" ) is huge and can be easier to start reviewing if you revert the rote, mechanical changes: Substitute the commit hash of commit "[validation] Add CValidationState subclasses" for <CommitHash> in the commands below. ```sh git checkout <CommitHash> git grep -l ValidationState | xargs sed -i 's/BlockValidationState\|TxValidationState/CValidationState/g' git grep -l ValidationResult | xargs sed -i 's/BlockValidationResult\|TxValidationResult/ValidationInvalidReason/g' git grep -l MaybePunish | xargs sed -i 's/MaybePunishNode\(ForBlock\|ForTx\)/MaybePunishNode/g' git diff HEAD^ ``` After that it's possible to easily see the mechanical changes with: ```sh git log -p -n1 -U0 --word-diff-regex=. <CommitHash> ``` ACKs for top commit: laanwj: ACK 3004d5a amitiuttarwar: code review ACK 3004d5a. Also built & ran tests locally. fjahr: Code review ACK 3004d5a . Only nit style change and pure virtual destructor added since my last review. ryanofsky: Code review ACK 3004d5a. Just whitespace change and pure virtual destructor added since last review. Tree-SHA512: 511de1fb380a18bec1944ea82b513b6192df632ee08bb16344a2df3c40811a88f3872f04df24bc93a41643c96c48f376a04551840fd804a961490d6c702c3d36
There oughta be a rule about merging review club PRs right before the meeting :) |
@jonatack whoops … |
Ok, will still mention my code-review ACK for 3004d5a that I was just about to write when this got merged :-) |
Agree. The only downside is that review comments (with the reviewer's name) can not be included in the merge commit. |
Could possibly use a |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Built at 3004d5a, ran tests, running bitcoind tailing the log.
Code review progress:
- [validation] Add CValidationState subclasses
- [validation] Tidy Up ValidationResult class
- [validation] Remove error() calls from Invalid() calls
- [validation] Remove useless ret parameter from Invalid()
- [validation] Remove unused first_invalid parameter from ProcessNewBlo…
- [validation] Remove fMissingInputs from AcceptToMemoryPool()
Interesting reviewer tip in the PR description; result here (without the right colors).
Done |
Thanks! |
Reviewers of this PR are encouraged to review PR #17624 ("net: Fix an uninitialized read in ProcessMessage(…, "tx", …) when receiving a transaction we already have") which fixes a quite serious bug introduced in this PR. Luckily it was caught before being part of any release :) |
…", …) when receiving a transaction we already have 73b96c9 net: Fix uninitialized read in ProcessMessage(...) (practicalswift) Pull request description: Fix an uninitialized read in `ProcessMessage(…, "tx", …)` when receiving a transaction we already have. The uninitialized value is read and used on [L2526 in the case of `AlreadyHave(inv) == true`](https://github.com/bitcoin/bitcoin/blob/d8a66626d63135fd245d5afc524b88b9a94d208b/src/net_processing.cpp#L2494-L2526). Proof of concept being run against a `bitcoind` built with MemorySanitizer (`-fsanitize=memory`): ``` $ ./p2p-uninit-read-in-conditional-poc.py Usage: ./p2p-uninit-read-in-conditional-poc.py <dstaddr> <dstport> <net> $ bitcoind -regtest & $ ./p2p-uninit-read-in-conditional-poc.py 127.0.0.1 18444 regtest SUMMARY: MemorySanitizer: use-of-uninitialized-value [1]+ Exit 77 bitcoind -regtest $ ``` Proof of concept being run against a `bitcoind` running under Valgrind (`valgrind --exit-on-first-error`): ``` $ valgrind -q --exit-on-first-error=yes --error-exitcode=1 bitcoind -regtest & $ ./p2p-uninit-read-in-conditional-poc.py 127.0.0.1 18444 regtest ==27351== Conditional jump or move depends on uninitialised value(s) [1]+ Exit 1 valgrind -q --exit-on-first-error=yes --error-exitcode=1 bitcoind -regtest $ ``` Proof of concept script: ``` #!/usr/bin/env python3 import sys from test_framework.mininode import NetworkThread from test_framework.mininode import P2PDataStore from test_framework.messages import CTransaction, CTxIn, CTxOut, msg_tx def send_duplicate_tx(dstaddr="127.0.0.1", dstport=18444, net="regtest"): network_thread = NetworkThread() network_thread.start() node = P2PDataStore() node.peer_connect(dstaddr=dstaddr, dstport=dstport, net=net)() node.wait_for_verack() tx = CTransaction() tx.vin.append(CTxIn()) tx.vout.append(CTxOut()) node.send_message(msg_tx(tx)) node.send_message(msg_tx(tx)) node.peer_disconnect() network_thread.close() if __name__ == "__main__": if len(sys.argv) != 4: print("Usage: {} <dstaddr> <dstport> <net>".format(sys.argv[0])) sys.exit(0) send_duplicate_tx(sys.argv[1], int(sys.argv[2]), sys.argv[3]) ``` Note that the transaction in the proof of concept is the simplest possible, but really any transaction can be used. It does not have to be a valid transaction. This bug was introduced in #15921 ("validation: Tidy up ValidationState interface") which was merged in to `master` 28 days ago. Luckily this bug was caught before being part of any Bitcoin Core release :) ACKs for top commit: jnewbery: utACK 73b96c9 laanwj: ACK 73b96c9, thanks for discovering and reporting this before it ended up in a release. Tree-SHA512: 7ce6b8f260bcdd9b2ec4ff4b941a891bbef578acf4456df33b7a8d42b248237ec4949e65e2445b24851d1639b10681c701ad500b1c0b776ff050ef8c3812c795
Summary: This is in preparation for the next commit, which removes the useless `ret` parameter from ValidationState::Invalid(). error() is simply a convenience wrapper that calls LogPrintf and returns false. Call LogPrintf explicitly and substitute the error() call for a false bool literal. Partial backport of Core [[bitcoin/bitcoin#15921 | PR15921]] : bitcoin/bitcoin@1a37de4 Test Plan: `ninja check check-functional` Reviewers: #bitcoin_abc, Fabien Reviewed By: #bitcoin_abc, Fabien Differential Revision: https://reviews.bitcoinabc.org/D6876
…ckHeaders() Summary: No callers use the returned value in first_invalid. Remove it from the function signature and don't set it in the function. Partial backport of Core [[bitcoin/bitcoin#15921 | PR15921]] : bitcoin/bitcoin@c428622 Depends on D6873 only to be lint-free. Test Plan: `ninja check` Reviewers: #bitcoin_abc, Fabien Reviewed By: #bitcoin_abc, Fabien Differential Revision: https://reviews.bitcoinabc.org/D6874
Summary: ValidationState::Invalid() takes a parameter `ret` which is returned to the caller. All call sites set this to false. Remove the `ret` parameter and just return false always. Partial backport of Core [[bitcoin/bitcoin#15921 | PR15921]] : bitcoin/bitcoin@7204c64 Test Plan: `ninja check check-functional` Reviewers: #bitcoin_abc, Fabien Reviewed By: #bitcoin_abc, Fabien Differential Revision: https://reviews.bitcoinabc.org/D6885
Summary: Handle this failure in the same way as all other failures: call Invalid() with the reasons for the failure. Partial backport of Core [[bitcoin/bitcoin#15921 | PR15921]] : bitcoin/bitcoin@3004d5a Test Plan: `ninja check check-functional` Reviewers: #bitcoin_abc, Fabien Reviewed By: #bitcoin_abc, Fabien Differential Revision: https://reviews.bitcoinabc.org/D6923
Summary: 3004d5a12d09d94bfc4dee2a8e8f2291996a4aaf [validation] Remove fMissingInputs from AcceptToMemoryPool() (John Newbery) c428622a5bb1e37b2e6ab2c52791ac05d9271238 [validation] Remove unused first_invalid parameter from ProcessNewBlockHeaders() (John Newbery) 7204c6434b944f6ad51b3c895837729d3aa56eea [validation] Remove useless ret parameter from Invalid() (John Newbery) 1a37de4b3174d19a6d8691ae07e92b32fdfaef11 [validation] Remove error() calls from Invalid() calls (John Newbery) 067981e49246822421a7bcc720491427e1dba8a3 [validation] Tidy Up ValidationResult class (John Newbery) a27a2957ed9afbe5a96caa5f0f4cbec730d27460 [validation] Add CValidationState subclasses (John Newbery) Pull request description: Carries out some remaining tidy-ups remaining after PR 15141: - split ValidationState into TxValidationState and BlockValidationState (commit from ajtowns) - various minor code style tidy-ups to the ValidationState class - remove the useless `ret` parameter from `ValidationState::Invalid()` - remove the now unused `first_invalid` parameter from `ProcessNewBlockHeaders()` - remove the `fMissingInputs` parameter from `AcceptToMemoryPool()`, and deal with missing inputs the same way as other errors by using the `TxValidationState` object. Tip for reviewers (thanks ryanofsky!): The first commit ("[validation] Add CValidationState subclasses" ) is huge and can be easier to start reviewing if you revert the rote, mechanical changes: Substitute the commit hash of commit "[validation] Add CValidationState subclasses" for <CommitHash> in the commands below. ```sh git checkout <CommitHash> git grep -l ValidationState | xargs sed -i 's/BlockValidationState\|TxValidationState/CValidationState/g' git grep -l ValidationResult | xargs sed -i 's/BlockValidationResult\|TxValidationResult/ValidationInvalidReason/g' git grep -l MaybePunish | xargs sed -i 's/MaybePunishNode\(ForBlock\|ForTx\)/MaybePunishNode/g' git diff HEAD^ ``` After that it's possible to easily see the mechanical changes with: ```sh git log -p -n1 -U0 --word-diff-regex=. <CommitHash> ``` ACKs for top commit: laanwj: ACK 3004d5a12d09d94bfc4dee2a8e8f2291996a4aaf amitiuttarwar: code review ACK 3004d5a12d09d94bfc4dee2a8e8f2291996a4aaf. Also built & ran tests locally. fjahr: Code review ACK 3004d5a12d09d94bfc4dee2a8e8f2291996a4aaf . Only nit style change and pure virtual destructor added since my last review. ryanofsky: Code review ACK 3004d5a12d09d94bfc4dee2a8e8f2291996a4aaf. Just whitespace change and pure virtual destructor added since last review. Tree-SHA512: 511de1fb380a18bec1944ea82b513b6192df632ee08bb16344a2df3c40811a88f3872f04df24bc93a41643c96c48f376a04551840fd804a961490d6c702c3d36 Backport of Core [[bitcoin/bitcoin#15921 | PR15921]] and [[bitcoin/bitcoin#17624 | PR17624]] (small fix to 15921) Followup to [[bitcoin/bitcoin#15141 | PR15141]] Some differences reviewers will encounter: 1. RejectCodes (such as REJECT_INVALID) do not appear in the original PR due to an out-of-order backport, but we currently still support them, so it was retained. 2. Some files (notably tests) contain refactors not present in the original PR. This is due to a few out-of-order backports but otherwise harmless. Depends on D6923, D6929, D6930 Test Plan: `ninja check check-functional-extended` Reviewers: #bitcoin_abc, deadalnix Reviewed By: #bitcoin_abc, deadalnix Subscribers: deadalnix Differential Revision: https://reviews.bitcoinabc.org/D6860
Carries out some remaining tidy-ups remaining after PR 15141:
ret
parameter fromValidationState::Invalid()
first_invalid
parameter fromProcessNewBlockHeaders()
fMissingInputs
parameter fromAcceptToMemoryPool()
, and deal with missing inputs the same way as other errors by using theTxValidationState
object.Tip for reviewers (thanks ryanofsky!): The first commit ("[validation] Add CValidationState subclasses" ) is huge and can be easier to start reviewing if you revert the rote, mechanical changes:
Substitute the commit hash of commit "[validation] Add CValidationState subclasses" for in the commands below.
After that it's possible to easily see the mechanical changes with: