Skip to content

Conversation

theStack
Copy link
Contributor

In the test p2p_segwit.py, there are many instances where we send a segwit block to a node with the expectation that it is rejected. For this purpose, the helper function test_witness_block exists which allows also to check for a specific reject reason that is asserted in the debug log:

def test_witness_block(node, p2p, block, accepted, with_witness=True, reason=None):
"""Send a block to the node and check that it's accepted

This PR aims to increase the precision of the tests by adding the expected reject reasons to all test_witness_block call instances (found via greping after test_witness_block(.*accepted=False). For some blocks that are rejected due to failed script verification, the exact reason is only shown in the debug log if parallel script verification is disabled. For this reason, the addition of the reasons is split up in two commits:

  • first, all instances are tackled that are not subject to script verification, i.e. it doesn't matter whether parallel script verification is enabled or not (e.g. bad-blk-weight, bad-witness-merkle-match...)
  • then, we explicitely set -par=1 to only use one script thread, and add the remaining reasons (non-mandatory-script-verify-flag with the more specific reason in parantheses)

@theStack theStack force-pushed the 202108-test-check_specific_segwit_reject_reasons branch from 5ca9b69 to d742b1c Compare August 15, 2021 21:16
@DrahtBot DrahtBot added the Tests label Aug 15, 2021
Copy link
Member

@josibake josibake left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK d742b1c

this is a really important change: without it, the test could be failing for an entirely different reason and you would assume it was failing for the reason you expected. i tested this by adding the following lines to test_v0_outputs_arent_spendable:

from copy import deepcopy
fake_block = deepcopy(block)
# use a hardcoded hash from a previous test run
fake_block.hashPrevBlock = 45514493863593995281992143828725175249324913083213145988561565159796949646003
self.update_witness_block_with_transactions(block, [tx])
# Verify that segwit isn't activated. A block serialized with witness
# should be rejected prior to activation.
test_witness_block(self.nodes[0], self.test_node, fake_block, accepted=False, with_witness=True)

without a reason specified, this test passed! 😱

after adding reason='unexpected-witness', the test failed with the following error:

AssertionError: [node 0] Expected messages "['unexpected-witness']" does not partially match log:

 - 2021-08-17T11:29:45.259050Z [msghand] [net_processing.cpp:2478] [ProcessMessage] received: block (144 bytes) peer=1
 - 2021-08-17T11:29:45.259090Z [msghand] [net_processing.cpp:3719] [ProcessMessage] received block 5be173022096f0ddfa40c27d8b1c6b8da20d2c2ede788c69f49c1c174559db08 peer=1
 - 2021-08-17T11:29:45.259114Z [msghand] [validation.cpp:3298] [AcceptBlockHeader] ERROR: AcceptBlockHeader: prev block not found

based on this, we should have assert_debug_log fail if both expected_msgs and unexpected_msgs are empty arrays. ill open a follow up PR for this.

Copy link
Member

@jonatack jonatack left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting. Are these error cases already covered by ./src/test/test_bitcoin -t script_tests?

@@ -869,7 +875,7 @@ def test_block_malleability(self):
@subtest # type: ignore
def test_witness_block_size(self):
# TODO: Test that non-witness carrying blocks can't exceed 1MB
# Skipping this test for now; this is covered in p2p-fullblocktest.py
# Skipping this test for now; this is covered in feature_block.py
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Per git show ca6523d0c8 (thanks for the commit reference!) this was changed in January 2018, is it still accurate, covered in feature_block.py?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think the relevant part is here:

self.log.info("Accept a block of weight MAX_BLOCK_WEIGHT")
self.move_tip(15)
b23 = self.next_block(23, spend=out[6])
tx = CTransaction()
script_length = (MAX_BLOCK_WEIGHT - b23.get_weight() - 276) // 4
script_output = CScript([b'\x00' * script_length])
tx.vout.append(CTxOut(0, script_output))
tx.vin.append(CTxIn(COutPoint(b23.vtx[1].sha256, 0)))
b23 = self.update_block(23, [tx])
# Make sure the math above worked out to produce a max-weighted block
assert_equal(b23.get_weight(), MAX_BLOCK_WEIGHT)
self.send_blocks([b23], True)
self.save_spendable_output()
self.log.info("Reject a block of weight MAX_BLOCK_WEIGHT + 4")
self.move_tip(15)
b24 = self.next_block(24, spend=out[6])
script_length = (MAX_BLOCK_WEIGHT - b24.get_weight() - 276) // 4
script_output = CScript([b'\x00' * (script_length + 1)])
tx.vout = [CTxOut(0, script_output)]
b24 = self.update_block(24, [tx])
assert_equal(b24.get_weight(), MAX_BLOCK_WEIGHT + 1 * 4)
self.send_blocks([b24], success=False, reject_reason='bad-blk-length', reconnect=True)

These blocks (as all others in feature_block.py, IIRC) don't have a witness commitment, so they are "non-witness carrying blocks". For those the serialized size equals to the block weight divided by WITNESS_SCALE_FACTOR (4), i.e. with MAX_BLOCK_WEIGHT (4000000) this results in the 1MB size limit being checked. Note that these test parts were changed recently in #22378 to switch to weight-based accounting, the constant used was formerly named MAX_BLOCK_BASE_SIZE.

@theStack
Copy link
Contributor Author

Interesting. Are these error cases already covered by ./src/test/test_bitcoin -t script_tests?

It seems like each one of the following script interpreter errors (see src/script/script_error.cpp):

  • ERR_WITNESS_PROGRAM_WITNESS_EMPTY ("Witness program was passed an empty witness")
  • ERR_WITNESS_UNEXPECTED ("Witness provided for non-witness script")
  • ERR_CLEANSTACK ("Stack size must be exactly one after execution")
  • ERR_WITNESS_MALLEATED ("Witness requires empty scriptSig")
  • ERR_PUSH_SIZE ("Push value size limit exceeded")
  • ERR_SCRIPT_SIZE ("Script is too big")
  • ERR_INVALID_STACK_OPERATION ("Operation not valid with the current stack size")
  • ERR_EVAL_FALSE ("Script evaluated without error but finished with a false/empty top stack element")

can at least be found once in src/test/data/script_tests.json (without the ERR_ prefix), which is in turn used as input for the script tests in src/test/script_tests.cpp. I'm thinking that maybe a good follow-up PR idea would be to als introduce these script error constants in the functional test framework too, to not have those strings duplicated in the tests; it would be also easier to find which script errors are covered in unit and functional tests with a single grep. The test feature_taproot.py already does this locally:

# Expected error strings
ERR_SIG_SIZE = {"err_msg": "Invalid Schnorr signature size"}
ERR_SIG_HASHTYPE = {"err_msg": "Invalid Schnorr signature hash type"}
ERR_SIG_SCHNORR = {"err_msg": "Invalid Schnorr signature"}
ERR_OP_RETURN = {"err_msg": "OP_RETURN was encountered"}
ERR_CONTROLBLOCK_SIZE = {"err_msg": "Invalid Taproot control block size"}
ERR_WITNESS_PROGRAM_MISMATCH = {"err_msg": "Witness program hash mismatch"}
ERR_PUSH_LIMIT = {"err_msg": "Push value size limit exceeded"}
ERR_DISABLED_OPCODE = {"err_msg": "Attempted to use a disabled opcode"}
ERR_TAPSCRIPT_CHECKMULTISIG = {"err_msg": "OP_CHECKMULTISIG(VERIFY) is not available in tapscript"}
ERR_MINIMALIF = {"err_msg": "OP_IF/NOTIF argument must be minimal in tapscript"}
ERR_UNKNOWN_PUBKEY = {"err_msg": "Public key is neither compressed or uncompressed"}
ERR_STACK_SIZE = {"err_msg": "Stack size limit exceeded"}
ERR_CLEANSTACK = {"err_msg": "Stack size must be exactly one after execution"}
ERR_STACK_EMPTY = {"err_msg": "Operation not valid with the current stack size"}
ERR_SIGOPS_RATIO = {"err_msg": "Too much signature validation relative to witness weight"}
ERR_UNDECODABLE = {"err_msg": "Opcode missing or not understood"}
ERR_NO_SUCCESS = {"err_msg": "Script evaluated without error but finished with a false/empty top stack element"}
ERR_EMPTY_WITNESS = {"err_msg": "Witness program was passed an empty witness"}
ERR_CHECKSIGVERIFY = {"err_msg": "Script failed an OP_CHECKSIGVERIFY operation"}

maflcko pushed a commit that referenced this pull request Aug 26, 2021
…e machines

7720d4f test: fix failure in feature_nulldummy.py on single-core machines (Sebastian Falbesoner)
646b388 test: refactor: use named args for block_submit in feature_nulldummy.py (Sebastian Falbesoner)

Pull request description:

  On single-core machines, executing the test `feature_nulldummy.py` results in the following assertion error:

  ```
      ...
      2021-08-18T15:37:58.805000Z TestFramework (INFO): Test 4: Non-NULLDUMMY base multisig transaction is invalid after activation
      2021-08-18T15:37:58.814000Z TestFramework (ERROR): Assertion failed
      Traceback (most recent call last):
        File "[...]/test/functional/test_framework/test_framework.py", line 131, in main
          self.run_test()
        File "[...]/test/functional/feature_nulldummy.py", line 107, in run_test
          self.block_submit(self.nodes[0], [test4tx], accept=False)
        File "[...]/test/functional/feature_nulldummy.py", line 134, in block_submit
          assert_equal(None if accept else 'block-validation-failed', node.submitblock(block.serialize().hex()))
        File "[...]/test/functional/test_framework/util.py", line 49, in assert_equal
          raise AssertionError("not(%s)" % " == ".join(str(arg) for arg in (thing1, thing2) + args))
      AssertionError: not(block-validation-failed == non-mandatory-script-verify-flag (Dummy CHECKMULTISIG argument must be zero))
      2021-08-18T15:37:58.866000Z TestFramework (INFO): Stopping nodes
      ...
  ```

  There are hardly any single-core machines around anymore, but the behaviour can be reproduced on a multi-core machine by patching the function `GetNumCores()` to return 1 on the master branch and running `feature_nulldummy.py`:

  ```diff
  diff --git a/src/util/system.cpp b/src/util/system.cpp
  index 30d4103..149b512fc 100644
  --- a/src/util/system.cpp
  +++ b/src/util/system.cpp
  @@ -1338,7 +1338,7 @@ bool SetupNetworking()

   int GetNumCores()
   {
  -    return std::thread::hardware_concurrency();
  +    return 1;
   }
  ```
  As solution, parallel script verification is disabled (`-par=1`) and the exact reject reason is checked, which also increases the precision of the test (the possibility that the block is rejected because of another unintended reason is ruled out). See also related PR #22711 which applies the same approach for the p2p segwit test. The PR also includes a refactoring commit which changes the calls to `self.block_submit()` to use named arguments and removes the default value for parameter `accept` (i.e. explicitely passing `accept=...` is mandatory), with the aim to increase the test readability.

ACKs for top commit:
  josibake:
    ACK 7720d4f
  Saviour1001:
    Tested ACK <code>[7720d4f](https://github.com/bitcoin/bitcoin/commit/7720d4f650015272dc7109238230520f71858c6c)</code>

Tree-SHA512: 8a31ebab3e2ab38e555d7a23139b3324a134a0dedc5b879a2419348ae858323882dbbfcbbf88b68e4f8d7eea8cfe43ee19da1d0d2a36c93ae7878c4980cac31d
@theStack theStack force-pushed the 202108-test-check_specific_segwit_reject_reasons branch from d742b1c to 00b93e5 Compare August 26, 2021 18:06
@DrahtBot
Copy link
Contributor

DrahtBot commented Aug 30, 2021

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

No conflicts as of last run.

@josibake
Copy link
Member

reACK: 00b93e5

The block test was renamed from `p2p-fullblocks.py` to
`feature_block.py` in commit ca6523d (PR bitcoin#11774).
This commit adds specific expected reject reasons for segwit blocks
sent to the node, that are independent of whether multiple script threads
are activated or not.
This commit adds specific expected reject reasons for segwit blocks
sent to the node, that are only showing up if one script threads
is used. For this reason, the node is started with the parameter
`-par=1`.
@theStack theStack force-pushed the 202108-test-check_specific_segwit_reject_reasons branch from 00b93e5 to 45827fd Compare September 24, 2021 16:14
@theStack
Copy link
Contributor Author

Rebased on master.

@pg156
Copy link

pg156 commented Sep 29, 2021

Concept and approach ACK

Copy link
Contributor

@stratospher stratospher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tested ACK 45827fd.

Checking the reject reasons in test_witness_block is a great addition to the test.

@maflcko maflcko merged commit 49e40f5 into bitcoin:master Oct 25, 2021
sidhujag pushed a commit to syscoin/syscoin that referenced this pull request Oct 25, 2021
… p2p_segwit.py

45827fd test: check for block reject reasons in p2p_segwit.py [2/2] (Sebastian Falbesoner)
4eb532f test: check for block reject reasons in p2p_segwit.py [1/2] (Sebastian Falbesoner)
b1488c4 test: fix reference to block processing test in p2p_segwit.py (Sebastian Falbesoner)

Pull request description:

  In the test `p2p_segwit.py`, there are many instances where we send a segwit block to a node with the expectation that it is rejected. For this purpose, the helper function `test_witness_block` exists which allows also to check for a specific reject `reason` that is asserted in the debug log:
  https://github.com/bitcoin/bitcoin/blob/502d22ceed1f90ed41336260f8eb428d3acaf514/test/functional/p2p_segwit.py#L119-L120

  This PR aims to increase the precision of the tests by adding the expected reject reasons to all `test_witness_block` call instances (found via `grep`ing after `test_witness_block(.*accepted=False`). For some blocks that are rejected due to failed script verification, the exact reason is only shown in the debug log if parallel script verification is disabled. For this reason, the addition of the reasons is split up in two commits:
  * first, all instances are tackled that are not subject to script verification, i.e. it doesn't matter whether parallel script verification is enabled or not (e.g. `bad-blk-weight`, `bad-witness-merkle-match`...)
  * then, we explicitely set `-par=1` to only use one script thread, and add the remaining reasons (`non-mandatory-script-verify-flag` with the more specific reason in parantheses)

ACKs for top commit:
  stratospher:
    tested ACK 45827fd.

Tree-SHA512: 00f31867f11d48b38a42b1e79a1303bda1c797ccf3d8c73e6107d70b062604d51ee2a3f2251e7f068dfafdaf09469d27dfee438d9bc9ebaef7febc4b6ef90a95
@bitcoin bitcoin locked and limited conversation to collaborators Oct 30, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants