[rpc] fix verbose argument for getblock in bitcoin-cli #10747

jnewbery · 2017-07-05T11:23:23Z

Using the verbose option with getblock in bitcoin-cli has been broken since #8704:

→ bitcoin-cli -named getblock blockhash=0f9188f13cb7b2c71f2a335e3a4fc328bf5beb436012afca590b1a11466e2206 verbose=true
error code: -1
error message:
JSON value is not a boolean as expected

→ bitcoin-cli -named getblock blockhash=0f9188f13cb7b2c71f2a335e3a4fc328bf5beb436012afca590b1a11466e2206 verbosity=true
{
  "hash": "0f9188f13cb7b2c71f2a335e3a4fc328bf5beb436012afca590b1a11466e2206",
...

In general, I think that adding aliases because some people find argument names distasteful is a bad idea since it leads to subtle bugs like this.

However, that functionality has already been merged in so I'm not going to try to undo it. This is the simplest fix for restoring the previous behavior.

@achow101 @luke-jr

jonasschnelli · 2017-07-05T16:18:21Z

I agree, we should not allow aliases (If possible, something API compatibility could require it) . Should we then get completely rid of the alias here: https://github.com/bitcoin/bitcoin/blob/master/src/rpc/blockchain.cpp#L1542

The alias in general works, it's just the bitcoin-cli's JSON string-to-bool conversion that failed.

achow101 · 2017-07-05T16:32:45Z

utACK 58e9864

TheBlueMatt · 2017-07-05T18:29:13Z

Not to blow a small fix into a big change, but is it possible to more easily ensure we dont break this stuff in the future (also generally issues in having typos between the server registration and client registration here scares me cause most of it is untested). Maybe the easiest way is to just add a python or bash script that checks for consistency ala our documentation-required-for-options checks?

jnewbery · 2017-07-05T18:45:39Z

is it possible to more easily ensure we dont break this stuff in the future

Yes, absolutely agree that there should be better testing (bitcoin-cli seems to be entirely untested). Can we open an issue to track that and treat this as a quick fix for the recent regression?

TheBlueMatt · 2017-07-05T18:53:05Z

@jnewbery fair enough. utACK 58e9864

luke-jr · 2017-07-05T19:39:40Z

The alias isn't about taste, but compatibility. bitcoin-cli is essentially just a testing tool, so I don't think it needs to itself be compatible (but I don't mind either).

laanwj · 2017-07-10T15:50:13Z

utACK 58e9864

58e9864 [rpc] fix verbose argument for getblock in bitcoin-cli (John Newbery) Tree-SHA512: 7f176e1ddd9f3f7722ea0d268549629567ecf0c30bebf368824997566df0bfa01d31cf761abc9ca355e48c0bf0cb06d49d15a02b858999fcb7472dc7df2fbbf2

c6ec435 [tests] Add bitcoin_cli.py test script (John Newbery) b23549f [tests] add TestNodeCLI class for calling bitcoin-cli for a node (John Newbery) Pull request description: We don't test bitcoin-cli at all. That means that we can miss inconsistencies between the bitcoin-cli client and the RPC interface, such as #10698 and #10747. It also means that the various bitcoin-cli options and features are untested and regressions could be silently introduced. Let's fix that. This PR adds bitcoin-cli testing in the python functional test_framework: 1. Add a bitcoin_cli.py test script that tests bitcoin-cli. At the moment it only tests that the result of `getinfo` is the same if you run it as an RPC or through bitcoin-cli, but can easily be extended to test additional bitcoin-cli features **EDIT: `--usecli` option is moved to a separate PR. This PR now only covers the bitcoin_cli.py test.** 2. ~Add a `--usecli` option to the test framework. This changes the test to use bitcoin-cli for all RPC calls instead of using direct HTTP requests. This is somewhat experimental. It works for most tests, but there are some cases where it can't work transparently because:~ - ~the testcase is asserting on a specific error code, and bitcoin-cli returns a different error code from the direct RPC~ - ~we're sending a very large RPC request (eg `submitblock`) and it can't be serialized into a shell bitcoin-cli call.~ ~I think that even though `--usecli` doesn't work on all tests, it's still a useful experimental feature. Future potential enhancements:~ - ~enhance the framework to automatically skip tests that are known to fail with bitcoin-cli if the `--usecli` option is used.~ - ~run a subset of tests in Travis with `-usecli`~ This builds on and requires the `TestNode` PR #10711 . As an aside, this is a good demonstration of how tidy it is to add additional features/interfaces now that test node logic/state is encapsulated in a TestNode class. Addresses #10791 Tree-SHA512: a1e6be12e8e007f6f67b3d3bbcd142d835787300831eb38e6027a1ad25ca9d79c4bc99a41b19e31ee95205cba1b3b2d21a688b5909316aad70bfc2b4eb6d8a52

77aa9e5 test: Check RPC argument mapping (Wladimir J. van der Laan) Pull request description: Parse the dispatch tables from the server implementation files, and the conversion table from the client (see #10751). Perform the following consistency checks: - Arguments defined in conversion table, must be present in dispatch table. If not, it was probably forgotten to add them to the dispatch table, and they will not work. - Arguments defined in conversion table must have the same names as in the dispatch table. If not, they will not work. - All aliases for an argument must either be present in the conversion table, or not. Anything in between means an oversight and some aliases won't work. Any of these results in an error. It also performs a consistency check to see if the same named argument is sometimes converted, and sometimes not. E.g. one RPC call might have a 'verbose' argument that is converted, another RPC call might have one that is not converted. This is not necessarily wrong, but points at a possible error (as well as makes the API harder to memorize) - so it is emitted as a warning (could upgrade this to error). This test is added to travis and run when `CHECK_DOC`. Currently fails with the following output: ``` * Checking consistency between dispatch tables and vRPCConvertParams ERROR: createrawtransaction argument 3 (named optintorbf in vRPCConvertParams) is not defined in dispatch table ERROR: getblock argument ['verbosity', 'verbose'] has conflicts in vRPCConvertParams conversion specifier [True, False] WARNING: conversion mismatch for argument named verbose ([('getblock', False), ('getblockheader', True), ('getmempoolancestors', True), ('getmempooldescendants', True), ('getrawmempool', True), ('getrawtransaction', True)]) ``` - ~#10698 fixes the first ERROR~ - #10747 fixes the second ERROR, as well as the WARNING Update: #10698 was merged, leaving: ``` * Checking consistency between dispatch tables and vRPCConvertParams ERROR: getblock argument ['verbosity', 'verbose'] has conflicts in vRPCConvertParams conversion specifier [True, False] WARNING: conversion mismatch for argument named verbose ([('getblock', False), ('getblockheader', True), ('getmempoolancestors', True), ('getmempooldescendants', True), ('getrawmempool', True), ('getrawtransaction', True)]) ``` Tree-SHA512: feabebfbeda5d4613b2b9d5265aa6bde4e1a0235297ffd48fa415ad7edc531d9ed7913fe76d191ac60d481a915a326f216bc93de3c671e45e1d14e97d07dea7a

…in-cli 58e9864 [rpc] fix verbose argument for getblock in bitcoin-cli (John Newbery) Tree-SHA512: 7f176e1ddd9f3f7722ea0d268549629567ecf0c30bebf368824997566df0bfa01d31cf761abc9ca355e48c0bf0cb06d49d15a02b858999fcb7472dc7df2fbbf2

c6ec435 [tests] Add bitcoin_cli.py test script (John Newbery) b23549f [tests] add TestNodeCLI class for calling bitcoin-cli for a node (John Newbery) Pull request description: We don't test bitcoin-cli at all. That means that we can miss inconsistencies between the bitcoin-cli client and the RPC interface, such as bitcoin#10698 and bitcoin#10747. It also means that the various bitcoin-cli options and features are untested and regressions could be silently introduced. Let's fix that. This PR adds bitcoin-cli testing in the python functional test_framework: 1. Add a bitcoin_cli.py test script that tests bitcoin-cli. At the moment it only tests that the result of `getinfo` is the same if you run it as an RPC or through bitcoin-cli, but can easily be extended to test additional bitcoin-cli features **EDIT: `--usecli` option is moved to a separate PR. This PR now only covers the bitcoin_cli.py test.** 2. ~Add a `--usecli` option to the test framework. This changes the test to use bitcoin-cli for all RPC calls instead of using direct HTTP requests. This is somewhat experimental. It works for most tests, but there are some cases where it can't work transparently because:~ - ~the testcase is asserting on a specific error code, and bitcoin-cli returns a different error code from the direct RPC~ - ~we're sending a very large RPC request (eg `submitblock`) and it can't be serialized into a shell bitcoin-cli call.~ ~I think that even though `--usecli` doesn't work on all tests, it's still a useful experimental feature. Future potential enhancements:~ - ~enhance the framework to automatically skip tests that are known to fail with bitcoin-cli if the `--usecli` option is used.~ - ~run a subset of tests in Travis with `-usecli`~ This builds on and requires the `TestNode` PR bitcoin#10711 . As an aside, this is a good demonstration of how tidy it is to add additional features/interfaces now that test node logic/state is encapsulated in a TestNode class. Addresses bitcoin#10791 Tree-SHA512: a1e6be12e8e007f6f67b3d3bbcd142d835787300831eb38e6027a1ad25ca9d79c4bc99a41b19e31ee95205cba1b3b2d21a688b5909316aad70bfc2b4eb6d8a52

*DASH* DOES NOT IMPLEMENT RUNNING THIS IN TRAVIS 77aa9e5 test: Check RPC argument mapping (Wladimir J. van der Laan) Pull request description: Parse the dispatch tables from the server implementation files, and the conversion table from the client (see bitcoin#10751). Perform the following consistency checks: - Arguments defined in conversion table, must be present in dispatch table. If not, it was probably forgotten to add them to the dispatch table, and they will not work. - Arguments defined in conversion table must have the same names as in the dispatch table. If not, they will not work. - All aliases for an argument must either be present in the conversion table, or not. Anything in between means an oversight and some aliases won't work. Any of these results in an error. It also performs a consistency check to see if the same named argument is sometimes converted, and sometimes not. E.g. one RPC call might have a 'verbose' argument that is converted, another RPC call might have one that is not converted. This is not necessarily wrong, but points at a possible error (as well as makes the API harder to memorize) - so it is emitted as a warning (could upgrade this to error). This test is added to travis and run when `CHECK_DOC`. Currently fails with the following output: ``` * Checking consistency between dispatch tables and vRPCConvertParams ERROR: createrawtransaction argument 3 (named optintorbf in vRPCConvertParams) is not defined in dispatch table ERROR: getblock argument ['verbosity', 'verbose'] has conflicts in vRPCConvertParams conversion specifier [True, False] WARNING: conversion mismatch for argument named verbose ([('getblock', False), ('getblockheader', True), ('getmempoolancestors', True), ('getmempooldescendants', True), ('getrawmempool', True), ('getrawtransaction', True)]) ``` - ~bitcoin#10698 fixes the first ERROR~ - bitcoin#10747 fixes the second ERROR, as well as the WARNING Update: bitcoin#10698 was merged, leaving: ``` * Checking consistency between dispatch tables and vRPCConvertParams ERROR: getblock argument ['verbosity', 'verbose'] has conflicts in vRPCConvertParams conversion specifier [True, False] WARNING: conversion mismatch for argument named verbose ([('getblock', False), ('getblockheader', True), ('getmempoolancestors', True), ('getmempooldescendants', True), ('getrawmempool', True), ('getrawtransaction', True)]) ``` Tree-SHA512: feabebfbeda5d4613b2b9d5265aa6bde4e1a0235297ffd48fa415ad7edc531d9ed7913fe76d191ac60d481a915a326f216bc93de3c671e45e1d14e97d07dea7a

c6ec435 [tests] Add bitcoin_cli.py test script (John Newbery) b23549f [tests] add TestNodeCLI class for calling bitcoin-cli for a node (John Newbery) Pull request description: We don't test bitcoin-cli at all. That means that we can miss inconsistencies between the bitcoin-cli client and the RPC interface, such as bitcoin#10698 and bitcoin#10747. It also means that the various bitcoin-cli options and features are untested and regressions could be silently introduced. Let's fix that. This PR adds bitcoin-cli testing in the python functional test_framework: 1. Add a bitcoin_cli.py test script that tests bitcoin-cli. At the moment it only tests that the result of `getinfo` is the same if you run it as an RPC or through bitcoin-cli, but can easily be extended to test additional bitcoin-cli features **EDIT: `--usecli` option is moved to a separate PR. This PR now only covers the bitcoin_cli.py test.** 2. ~Add a `--usecli` option to the test framework. This changes the test to use bitcoin-cli for all RPC calls instead of using direct HTTP requests. This is somewhat experimental. It works for most tests, but there are some cases where it can't work transparently because:~ - ~the testcase is asserting on a specific error code, and bitcoin-cli returns a different error code from the direct RPC~ - ~we're sending a very large RPC request (eg `submitblock`) and it can't be serialized into a shell bitcoin-cli call.~ ~I think that even though `--usecli` doesn't work on all tests, it's still a useful experimental feature. Future potential enhancements:~ - ~enhance the framework to automatically skip tests that are known to fail with bitcoin-cli if the `--usecli` option is used.~ - ~run a subset of tests in Travis with `-usecli`~ This builds on and requires the `TestNode` PR bitcoin#10711 . As an aside, this is a good demonstration of how tidy it is to add additional features/interfaces now that test node logic/state is encapsulated in a TestNode class. Addresses bitcoin#10791 Tree-SHA512: a1e6be12e8e007f6f67b3d3bbcd142d835787300831eb38e6027a1ad25ca9d79c4bc99a41b19e31ee95205cba1b3b2d21a688b5909316aad70bfc2b4eb6d8a52

*DASH* DOES NOT IMPLEMENT RUNNING THIS IN TRAVIS 77aa9e5 test: Check RPC argument mapping (Wladimir J. van der Laan) Pull request description: Parse the dispatch tables from the server implementation files, and the conversion table from the client (see bitcoin#10751). Perform the following consistency checks: - Arguments defined in conversion table, must be present in dispatch table. If not, it was probably forgotten to add them to the dispatch table, and they will not work. - Arguments defined in conversion table must have the same names as in the dispatch table. If not, they will not work. - All aliases for an argument must either be present in the conversion table, or not. Anything in between means an oversight and some aliases won't work. Any of these results in an error. It also performs a consistency check to see if the same named argument is sometimes converted, and sometimes not. E.g. one RPC call might have a 'verbose' argument that is converted, another RPC call might have one that is not converted. This is not necessarily wrong, but points at a possible error (as well as makes the API harder to memorize) - so it is emitted as a warning (could upgrade this to error). This test is added to travis and run when `CHECK_DOC`. Currently fails with the following output: ``` * Checking consistency between dispatch tables and vRPCConvertParams ERROR: createrawtransaction argument 3 (named optintorbf in vRPCConvertParams) is not defined in dispatch table ERROR: getblock argument ['verbosity', 'verbose'] has conflicts in vRPCConvertParams conversion specifier [True, False] WARNING: conversion mismatch for argument named verbose ([('getblock', False), ('getblockheader', True), ('getmempoolancestors', True), ('getmempooldescendants', True), ('getrawmempool', True), ('getrawtransaction', True)]) ``` - ~bitcoin#10698 fixes the first ERROR~ - bitcoin#10747 fixes the second ERROR, as well as the WARNING Update: bitcoin#10698 was merged, leaving: ``` * Checking consistency between dispatch tables and vRPCConvertParams ERROR: getblock argument ['verbosity', 'verbose'] has conflicts in vRPCConvertParams conversion specifier [True, False] WARNING: conversion mismatch for argument named verbose ([('getblock', False), ('getblockheader', True), ('getmempoolancestors', True), ('getmempooldescendants', True), ('getrawmempool', True), ('getrawtransaction', True)]) ``` Tree-SHA512: feabebfbeda5d4613b2b9d5265aa6bde4e1a0235297ffd48fa415ad7edc531d9ed7913fe76d191ac60d481a915a326f216bc93de3c671e45e1d14e97d07dea7a

…in-cli 58e9864 [rpc] fix verbose argument for getblock in bitcoin-cli (John Newbery) Tree-SHA512: 7f176e1ddd9f3f7722ea0d268549629567ecf0c30bebf368824997566df0bfa01d31cf761abc9ca355e48c0bf0cb06d49d15a02b858999fcb7472dc7df2fbbf2

c6ec435 [tests] Add bitcoin_cli.py test script (John Newbery) b23549f [tests] add TestNodeCLI class for calling bitcoin-cli for a node (John Newbery) Pull request description: We don't test bitcoin-cli at all. That means that we can miss inconsistencies between the bitcoin-cli client and the RPC interface, such as bitcoin#10698 and bitcoin#10747. It also means that the various bitcoin-cli options and features are untested and regressions could be silently introduced. Let's fix that. This PR adds bitcoin-cli testing in the python functional test_framework: 1. Add a bitcoin_cli.py test script that tests bitcoin-cli. At the moment it only tests that the result of `getinfo` is the same if you run it as an RPC or through bitcoin-cli, but can easily be extended to test additional bitcoin-cli features **EDIT: `--usecli` option is moved to a separate PR. This PR now only covers the bitcoin_cli.py test.** 2. ~Add a `--usecli` option to the test framework. This changes the test to use bitcoin-cli for all RPC calls instead of using direct HTTP requests. This is somewhat experimental. It works for most tests, but there are some cases where it can't work transparently because:~ - ~the testcase is asserting on a specific error code, and bitcoin-cli returns a different error code from the direct RPC~ - ~we're sending a very large RPC request (eg `submitblock`) and it can't be serialized into a shell bitcoin-cli call.~ ~I think that even though `--usecli` doesn't work on all tests, it's still a useful experimental feature. Future potential enhancements:~ - ~enhance the framework to automatically skip tests that are known to fail with bitcoin-cli if the `--usecli` option is used.~ - ~run a subset of tests in Travis with `-usecli`~ This builds on and requires the `TestNode` PR bitcoin#10711 . As an aside, this is a good demonstration of how tidy it is to add additional features/interfaces now that test node logic/state is encapsulated in a TestNode class. Addresses bitcoin#10791 Tree-SHA512: a1e6be12e8e007f6f67b3d3bbcd142d835787300831eb38e6027a1ad25ca9d79c4bc99a41b19e31ee95205cba1b3b2d21a688b5909316aad70bfc2b4eb6d8a52

[rpc] fix verbose argument for getblock in bitcoin-cli

58e9864

fanquake added the RPC/REST/ZMQ label Jul 5, 2017

TheBlueMatt mentioned this pull request Jul 5, 2017

Automated testing of mappings in RPC server registration and client arg conversion #10751

Closed

laanwj mentioned this pull request Jul 6, 2017

test: Check RPC argument mapping #10753

Merged

laanwj merged commit 58e9864 into bitcoin:master Jul 10, 2017

jnewbery mentioned this pull request Jul 11, 2017

[tests] [utils] test bitcoin-cli #10798

Merged

bitcoin locked as resolved and limited conversation to collaborators Sep 8, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[rpc] fix verbose argument for getblock in bitcoin-cli #10747

[rpc] fix verbose argument for getblock in bitcoin-cli #10747

Uh oh!

jnewbery commented Jul 5, 2017

Uh oh!

jonasschnelli commented Jul 5, 2017

Uh oh!

achow101 commented Jul 5, 2017

Uh oh!

TheBlueMatt commented Jul 5, 2017

Uh oh!

jnewbery commented Jul 5, 2017

Uh oh!

TheBlueMatt commented Jul 5, 2017

Uh oh!

luke-jr commented Jul 5, 2017

Uh oh!

laanwj commented Jul 10, 2017

Uh oh!

Uh oh!

[rpc] fix verbose argument for getblock in bitcoin-cli #10747

[rpc] fix verbose argument for getblock in bitcoin-cli #10747

Uh oh!

Conversation

jnewbery commented Jul 5, 2017

Uh oh!

jonasschnelli commented Jul 5, 2017

Uh oh!

achow101 commented Jul 5, 2017

Uh oh!

TheBlueMatt commented Jul 5, 2017

Uh oh!

jnewbery commented Jul 5, 2017

Uh oh!

TheBlueMatt commented Jul 5, 2017

Uh oh!

luke-jr commented Jul 5, 2017

Uh oh!

laanwj commented Jul 10, 2017

Uh oh!

Uh oh!