rpc, test: Improve getblockstats for unspendables #19888

fjahr · 2020-09-05T22:25:38Z

Fixes #19885

The genesis block does not have undo data saved to disk so the RPC errored because of that.

test/functional/rpc_getblockstats.py

n-thumann

tACK bb608b7 on master at 3ba25e3, ran functional & unit tests, confirmed that bug is fixed now :)

src/rpc/blockchain.cpp

maflcko · 2020-09-06T12:26:00Z

The genesis block has no effect on the utxo set, so the returned result is wrong

fjahr · 2020-09-06T23:44:03Z

The genesis block has no effect on the utxo set, so the returned result is wrong

Heh, yeah, the whole RPC actually didn't deal with unspendables at all so far. I am fixing this with my latest changes, fixing the Genesis block but also BIP30 and unspendable outputs. I think the numbers make much more sense this way. I also needed to regenerate the test data because the witness commitment is now excluded from the stats.

theStack · 2020-09-12T14:43:26Z

Concept ACK
Both enabling getblockstats to work with the genesis block and improving the accuracy of the utxo set statistics make big sense to me.

DrahtBot · 2020-10-02T01:13:41Z

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#26308 (rpc/rest/zmq: reduce LOCK(cs_main) scope: ~6 times as many requests per second by andrewtoth)
#25740 (assumeutxo: background validation completion by jamesob)
#24008 (assumeutxo: net_processing changes by jamesob)
#15606 (assumeutxo by jamesob)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

luke-jr · 2020-10-24T15:01:56Z

src/rpc/blockchain.cpp

@@ -1743,7 +1747,7 @@ static UniValue getblockstats(const JSONRPCRequest& request)
                {RPCResult::Type::NUM, "totalfee", "The fee total"},
                {RPCResult::Type::NUM, "txs", "The number of transactions (including coinbase)"},
                {RPCResult::Type::NUM, "utxo_increase", "The increase/decrease in the number of unspent outputs"},
-                {RPCResult::Type::NUM, "utxo_size_inc", "The increase/decrease in size for the utxo index (not discounting op_return and similar)"},
+                {RPCResult::Type::NUM, "utxo_size_inc", "The increase/decrease in size for the utxo index"},


With this change to the documented API, I think you'd better add a new field instead...

Slightly would have preferred to just fix the existing values but I have now introduced new values with a *_actual naming scheme.

luke-jr · 2020-10-24T15:03:53Z

src/rpc/blockchain.cpp

+    // set, so they can be excluded from the statistics
+    const bool is_genesis = pindex->nHeight == 0;
+    const bool is_bip30_repeat = (pindex->nHeight == 91880 && pindex->GetBlockHash() == uint256S("0x00000000000743f190a18c5577a3c2d2a1f610ae9601ac046a38084ccb7cd721")) ||
+                                 (pindex->nHeight == 91842 && pindex->GetBlockHash() == uint256S("0x00000000000a4d0a398161ffc163c503763b1f4360639393e0e4c8e300e0caec"));


It would be nice to abstract this to a block-aware IsUnspendable instead.

+1, it would be better not to repeat this in multiple places

Right, but it's a bit more complicated because these blocks' coinbases are not actually unspendable, they are the ones that overwrote the other previous coinbases aifaik. But I have refactored both of these out into helper functions that would also be helpful to me for Coinstatsindex. Let me know what you think about the naming, 'repeat' was the best I came up with.

luke-jr · 2020-10-24T15:05:06Z

test/functional/rpc_getblockstats.py

@@ -160,6 +160,9 @@ def run_test(self):
        assert_raises_rpc_error(-1, 'getblockstats hash_or_height ( stats )', self.nodes[0].getblockstats, '00', 1, 2)
        assert_raises_rpc_error(-1, 'getblockstats hash_or_height ( stats )', self.nodes[0].getblockstats)

+        self.log.info('Test block height 0')
+        genesis_stats = self.nodes[0].getblockstats(0)
+        assert_equal(genesis_stats["blockhash"], "0f9188f13cb7b2c71f2a335e3a4fc328bf5beb436012afca590b1a11466e2206")


Would be nice to test the other corner-case result values

I am not sure if you had more values in mind (let me know) but I have added the old and new values that this PR touches.

Added a test with an OP_RETURN output.

re: #19888 (comment)

Added a test with an OP_RETURN output.

Note: This is referring to commit f87d4b9 "Added a test with an OP_RETURN output" I think

fjahr · 2020-12-24T13:35:21Z

Addressed feedback comments. Thanks for reviewing and sorry for the long wait!

luke-jr · 2021-02-28T04:20:56Z

test/functional/data/rpc_getblockstats.json

@@ -102,8 +102,8 @@
    "00000020f44e7a48b9f221af95f3295c8dcefc5358934a68dc79e2933dc0794b350cad0a90fad2cd50b41d4ef45e76c2a456b98c180632bb4b44e0cd18ce90679fe54e552b4ae75affff7f200000000001020000000001010000000000000000000000000000000000000000000000000000000000000000ffffffff0401630101ffffffff0200f2052a010000001976a9142b4569203694fc997e13f2c0a1383b9e16c77a0d88ac0000000000000000266a24aa21a9ede2f61c3f71d1defd3fa999dfa36953755c690689799962b48bebd836974e8cf90120000000000000000000000000000000000000000000000000000000000000000000000000",
    "0000002087454276cce83f4d19e0120f6e9728ac5905f7adaf6b27e3f5bbe43ab823f85db7d1f44666531483df3d67c15f2c231718ad93b63b851dce5ff4c4a67f524ffa2b4ae75affff7f200100000001020000000001010000000000000000000000000000000000000000000000000000000000000000ffffffff0401640101ffffffff0200f2052a010000001976a9142b4569203694fc997e13f2c0a1383b9e16c77a0d88ac0000000000000000266a24aa21a9ede2f61c3f71d1defd3fa999dfa36953755c690689799962b48bebd836974e8cf90120000000000000000000000000000000000000000000000000000000000000000000000000",
    "000000202cdc3e99f07a80252dd6097faa0eddf3f2dde5ae390610e0bca94ecc25931551d31fceb8fe0a682f6017ca3dbb582f3a2f06e5d99ec99c42c8a744dd4c9216b82b4ae75affff7f200300000001020000000001010000000000000000000000000000000000000000000000000000000000000000ffffffff0401650101ffffffff0200f2052a010000001976a9142b4569203694fc997e13f2c0a1383b9e16c77a0d88ac0000000000000000266a24aa21a9ede2f61c3f71d1defd3fa999dfa36953755c690689799962b48bebd836974e8cf90120000000000000000000000000000000000000000000000000000000000000000000000000",
-    "000000209b3ace9bd510918d20e87518c0cf5976cab3e28cc7af41259a89c6dd7668a32922808b8a082be71bcd6152cb8fd223650b5579a41344ba749e4d17b9bf211a9e2b4ae75affff7f200000000002020000000001010000000000000000000000000000000000000000000000000000000000000000ffffffff0401660101ffffffff026c03062a010000001976a9142b4569203694fc997e13f2c0a1383b9e16c77a0d88ac0000000000000000266a24aa21a9edb85d8f3c122c43a72f1e0dd122c8f7af040aa0b0a46001621110fb37818021510120000000000000000000000000000000000000000000000000000000000000000000000000020000000128394022bf44bff30d7399cb5a16e3b94fed67dc174c2e1d77df91bad5a51cb3000000006a47304402201c16d06a5c4353168b3881071aea7d1eb4d88eedfea53a9d6af9abb56da9060002205abf3ae535f1f1b5cfe8ba955535c2b20ac003e7d7720c5b7d2640ac2a04d19001210227d85ba011276cf25b51df6a188b75e604b38770a462b2d0e9fb2fc839ef5d3ffeffffff0294b89a3b000000001976a9142b4569203694fc997e13f2c0a1383b9e16c77a0d88ac00286bee0000000017a91452bab4f229415d0dc5c6d30b162f93a1a0cac5958765000000",


Can you retain the old test?

Done now in this particular commit but now need to regenerate it in another commit :-/

fjahr · 2021-04-11T19:12:24Z

Finally got back to give this the attention it deserves. First of all, it is rebased. Then I followed some recommendation from @luke-jr which also let me to find some other issue.

Here is a recap per commit to make it easier to understand what changed:

c7ccc07: (unchanged) Refactors BIP30 logic into helper functions. This will also be helpful for #19521 where the logic is currently duplicated as well.
5d0649c: Fixes getblockstats behavior for unspendable statistics (genesis, BIP30, OP_RETURN). Here I now followed Luke's suggestion and retain the old test.
50c956f: While making changes and testing I noticed that the data generator is currently broken since the default wallet was removed, this is fixed here. I initially used MiniWallet but it would have been a much larger change (mostly due to the next commit) and since this is generator code it's not critical I think.
1c18449: Following Luke's request to test more corner cases I added a test with an OP_RETURN output. But this made it impractical to keep the current test, so the test data is now regenerated in this commit.

fjahr · 2022-05-21T21:43:46Z

Rebased and fixed the silent merge conflict noted by @ryanofsky . Thanks!

Making the checks to identify BIP30 available outside of validation.cpp is needed for reporting and tracking statistics on specific blocks and the UTXO set correctly. Github-Pull: bitcoin#19888 Rebased-From: 1effbfe

- Fix getblockstats for block height 0 which previously returned an error. - Introduce alternative utxo_*_actual statistics which exclude unspendables: Genesis block, BIP30, unspendable outputs - Update test data - Explicitly test Genesis block results Github-Pull: bitcoin#19888 Rebased-From: 0fec5c2

The getblockstats RPC functional test is using previously generated test data that is part of the repository. That test data can be regenerated by running the test file with `--gen-test-data` which invokes the `generate_test_data()` function. That function still relied on the old wallet behavior of having a default wallet to work. Because of this the function was broken and this change fixes this. The fact that this was broken did was not noticed previously because the function is not used by the automated test suite by default. Github-Pull: bitcoin#19888 Rebased-From: 2d7975c

Github-Pull: bitcoin#19888 Rebased-From: ff16851

fjahr · 2022-07-20T22:04:05Z

rebased

Making the checks to identify BIP30 available outside of validation.cpp is needed for reporting and tracking statistics on specific blocks and the UTXO set correctly.

- Fix getblockstats for block height 0 which previously returned an error. - Introduce alternative utxo_*_actual statistics which exclude unspendables: Genesis block, BIP30, unspendable outputs - Update test data - Explicitly test Genesis block results

The getblockstats RPC functional test is using previously generated test data that is part of the repository. That test data can be regenerated by running the test file with `--gen-test-data` which invokes the `generate_test_data()` function. That function still relied on the old wallet behavior of having a default wallet to work. Because of this the function was broken and this change fixes this. The fact that this was broken did was not noticed previously because the function is not used by the automated test suite by default.

fjahr · 2022-10-23T12:55:31Z

rebased

aureleoules

ACK d885bb2

Some end results I tried:

$ diff ./stats_master ./stats_pr # testnet block 000000007be904ff76a13eb6f0a10d945dbdc7a8ce18d66dd62af0d7e31d69fb 
36c36,38
<   "utxo_size_inc": 614
---
>   "utxo_size_inc": 614,
>   "utxo_increase_actual": 6,
>   "utxo_size_inc_actual": 393

Master

Details

./src/bitcoin-cli getblockstats '"000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f"' # genesis
error code: -1
error message:
Can't read undo data from disk

./src/bitcoin-cli getblockstats '"00000000000a4d0a398161ffc163c503763b1f4360639393e0e4c8e300e0caec"'
{
  "avgfee": 0,
  "avgfeerate": 0,
  "avgtxsize": 0,
  "blockhash": "00000000000a4d0a398161ffc163c503763b1f4360639393e0e4c8e300e0caec",
  "feerate_percentiles": [
    0,
    0,
    0,
    0,
    0
  ],
  "height": 91842,
  "ins": 0,
  "maxfee": 0,
  "maxfeerate": 0,
  "maxtxsize": 0,
  "medianfee": 0,
  "mediantime": 1289767893,
  "mediantxsize": 0,
  "minfee": 0,
  "minfeerate": 0,
  "mintxsize": 0,
  "outs": 1,
  "subsidy": 5000000000,
  "swtotal_size": 0,
  "swtotal_weight": 0,
  "swtxs": 0,
  "time": 1289768691,
  "total_out": 0,
  "total_size": 0,
  "total_weight": 0,
  "totalfee": 0,
  "txs": 1,
  "utxo_increase": 1,
  "utxo_size_inc": 117
}

PR

Details

./src/bitcoin-cli getblockstats '"000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f"' # genesis
{
  "avgfee": 0,
  "avgfeerate": 0,
  "avgtxsize": 0,
  "blockhash": "000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f",
  "feerate_percentiles": [
    0,
    0,
    0,
    0,
    0
  ],
  "height": 0,
  "ins": 0,
  "maxfee": 0,
  "maxfeerate": 0,
  "maxtxsize": 0,
  "medianfee": 0,
  "mediantime": 1231006505,
  "mediantxsize": 0,
  "minfee": 0,
  "minfeerate": 0,
  "mintxsize": 0,
  "outs": 1,
  "subsidy": 5000000000,
  "swtotal_size": 0,
  "swtotal_weight": 0,
  "swtxs": 0,
  "time": 1231006505,
  "total_out": 0,
  "total_size": 0,
  "total_weight": 0,
  "totalfee": 0,
  "txs": 1,
  "utxo_increase": 1,
  "utxo_size_inc": 117,
  "utxo_increase_actual": 0,
  "utxo_size_inc_actual": 0
}

./src/bitcoin-cli getblockstats '"00000000000a4d0a398161ffc163c503763b1f4360639393e0e4c8e300e0caec"'
{
  "avgfee": 0,
  "avgfeerate": 0,
  "avgtxsize": 0,
  "blockhash": "00000000000a4d0a398161ffc163c503763b1f4360639393e0e4c8e300e0caec",
  "feerate_percentiles": [
    0,
    0,
    0,
    0,
    0
  ],
  "height": 91842,
  "ins": 0,
  "maxfee": 0,
  "maxfeerate": 0,
  "maxtxsize": 0,
  "medianfee": 0,
  "mediantime": 1289767893,
  "mediantxsize": 0,
  "minfee": 0,
  "minfeerate": 0,
  "mintxsize": 0,
  "outs": 1,
  "subsidy": 5000000000,
  "swtotal_size": 0,
  "swtotal_weight": 0,
  "swtxs": 0,
  "time": 1289768691,
  "total_out": 0,
  "total_size": 0,
  "total_weight": 0,
  "totalfee": 0,
  "txs": 1,
  "utxo_increase": 1,
  "utxo_size_inc": 117,
  "utxo_increase_actual": 0,
  "utxo_size_inc_actual": 0
}

stickies-v

ACK d885bb2

No blocking comments/okay in follow-up. However, since it's difficult to change RPC variable names after they're part of the public API, might be best to consider that first.

stickies-v · 2022-10-27T11:30:12Z

src/validation.h

+/** Identifies blocks that overwrote an existing coinbase output in the UTXO set (see BIP30) */
+bool IsBIP30Repeat(const CBlockIndex& block_index);
+
+/** Identifies blocks which coinbase output was subsequently overwritten in the UTXO set (see BIP30) */


nit (see e.g. this)

Suggested change

/** Identifies blocks which coinbase output was subsequently overwritten in the UTXO set (see BIP30) */

/** Identifies blocks whose coinbase output was subsequently overwritten in the UTXO set (see BIP30) */

stickies-v · 2022-10-27T11:34:21Z

src/validation.h

@@ -1082,4 +1082,10 @@ bool DeploymentEnabled(const ChainstateManager& chainman, DEP dep)
 */
 const AssumeutxoData* ExpectedAssumeutxo(const int height, const CChainParams& params);

+/** Identifies blocks that overwrote an existing coinbase output in the UTXO set (see BIP30) */


nit:

Suggested change

/** Identifies blocks that overwrote an existing coinbase output in the UTXO set (see BIP30) */

/** Did this block overwrite an existing coinbase output in the UTXO set? (see BIP30) */

stickies-v · 2022-10-27T11:35:44Z

src/validation.h

+/** Identifies blocks that overwrote an existing coinbase output in the UTXO set (see BIP30) */
+bool IsBIP30Repeat(const CBlockIndex& block_index);
+
+/** Identifies blocks which coinbase output was subsequently overwritten in the UTXO set (see BIP30) */


nit

Suggested change

/** Identifies blocks which coinbase output was subsequently overwritten in the UTXO set (see BIP30) */

/** Was this block's coinbase output overwritten in the UTXO set by a later block? (see BIP30) */

stickies-v · 2022-10-27T11:44:17Z

src/rpc/blockchain.cpp

+                {RPCResult::Type::NUM, "utxo_increase_actual", /*optional=*/true, "The increase/decrease in the number of unspent outputs, not counting unspendables"},
+                {RPCResult::Type::NUM, "utxo_size_inc_actual", /*optional=*/true, "The increase/decrease in size for the utxo index, not counting unspendables"},


I think "actual" is quite vague, "spendable" would be more immediately obvious imo?

Suggested change

{RPCResult::Type::NUM, "utxo_increase_actual", /*optional=*/true, "The increase/decrease in the number of unspent outputs, not counting unspendables"},

{RPCResult::Type::NUM, "utxo_size_inc_actual", /*optional=*/true, "The increase/decrease in size for the utxo index, not counting unspendables"},

{RPCResult::Type::NUM, "utxo_increase_spendable", /*optional=*/true, "The increase/decrease in the number of unspent outputs, not counting unspendables"},

{RPCResult::Type::NUM, "utxo_size_inc_spendable", /*optional=*/true, "The increase/decrease in size for the utxo index, not counting unspendables"},

stickies-v · 2022-10-27T11:47:36Z

src/rpc/blockchain.cpp

@@ -1771,8 +1775,10 @@ static RPCHelpMan getblockstats()
                {RPCResult::Type::NUM, "total_weight", /*optional=*/true, "Total weight of all non-coinbase transactions"},
                {RPCResult::Type::NUM, "totalfee", /*optional=*/true, "The fee total"},
                {RPCResult::Type::NUM, "txs", /*optional=*/true, "The number of transactions (including coinbase)"},
-                {RPCResult::Type::NUM, "utxo_increase", /*optional=*/true, "The increase/decrease in the number of unspent outputs"},
+                {RPCResult::Type::NUM, "utxo_increase", /*optional=*/true, "The increase/decrease in the number of unspent outputs (not discounting op_return and similar)"},


I think using different terminology here compared to in the 2 new lines makes it less clear what the difference in functionality is. Realigning the phrasing makes sense for readability imo.

"including unspendables such as op_return" and "excluding unspendables such as op_return" makes the difference more immediately obvious I think.

stickies-v · 2022-10-27T13:17:47Z

src/rpc/blockchain.cpp

                {RPCResult::Type::NUM, "utxo_size_inc", /*optional=*/true, "The increase/decrease in size for the utxo index (not discounting op_return and similar)"},
+                {RPCResult::Type::NUM, "utxo_increase_actual", /*optional=*/true, "The increase/decrease in the number of unspent outputs, not counting unspendables"},


nit: every other line is alphabetically sorted, this one isn't

stickies-v · 2022-10-27T13:43:50Z

src/rpc/blockchain.cpp

@@ -1825,7 +1831,9 @@ static RPCHelpMan getblockstats()
    int64_t swtxs = 0;
    int64_t total_size = 0;
    int64_t total_weight = 0;
+    int64_t utxos = 0;


nit: spendable_utxo_count would be my preferred, but that doesn't seem to be in line with naming convention here, so I'm happy with:

Suggested change

int64_t utxos = 0;

int64_t utxos_spendable = 0;

stickies-v · 2022-10-27T14:14:49Z

test/functional/rpc_getblockstats.py

+        tip_stats = self.nodes[0].getblockstats(tip)
+        assert_equal(tip_stats["utxo_increase"], 6)
+        assert_equal(tip_stats["utxo_size_inc"], 441)
+        assert_equal(tip_stats["utxo_increase_actual"], 4)


Would you mind adding a docstring that explains why 4 (i.e. 2 unspendable UTXOs)? From generate_test_data it would seem there is only 1 OP_RETURN, and genesis/bip30 wouldn't apply here either?

stickies-v · 2022-10-27T15:06:30Z

test/functional/rpc_getblockstats.py

+        assert_equal(tip_stats["utxo_size_inc"], 441)
+        assert_equal(tip_stats["utxo_increase_actual"], 4)
+        assert_equal(tip_stats["utxo_size_inc_actual"], 300)


When I run --gen-test-data, I get slightly different sizes: 441->438 and 300->297. Not sure how local ./configure options would change that, so... wondering if it's just me or leftover from using an older wallet version in earlier versions of the PR?

achow101 · 2022-12-05T22:27:58Z

ACK d885bb2

fanquake added the RPC/REST/ZMQ label Sep 5, 2020

fjahr mentioned this pull request Sep 5, 2020

getblockstats RPC command has issues #19885

Closed

n-thumann reviewed Sep 5, 2020

View reviewed changes

test/functional/rpc_getblockstats.py Outdated Show resolved Hide resolved

fjahr force-pushed the genesisblockstats branch from 3b40c6f to bb608b7 Compare September 5, 2020 22:59

n-thumann approved these changes Sep 5, 2020

View reviewed changes

promag reviewed Sep 5, 2020

View reviewed changes

src/rpc/blockchain.cpp Outdated Show resolved Hide resolved

fjahr force-pushed the genesisblockstats branch from bb608b7 to defaac4 Compare September 6, 2020 11:19

fjahr force-pushed the genesisblockstats branch from defaac4 to c266571 Compare September 6, 2020 23:39

fjahr changed the title ~~rpc: Fix getblockstats for block height 0~~ rpc: Fix getblockstats issues Sep 7, 2020

DrahtBot mentioned this pull request Oct 2, 2020

rpc: calculate fees in getblock using BlockUndo data #18772

Merged

luke-jr suggested changes Oct 24, 2020

View reviewed changes

fjahr force-pushed the genesisblockstats branch from c266571 to 37bbfe9 Compare December 24, 2020 13:23

DrahtBot mentioned this pull request Dec 24, 2020

validation: UTXO snapshot activation #19806

Merged

DrahtBot added the Needs rebase label Feb 16, 2021

luke-jr reviewed Feb 28, 2021

View reviewed changes

fjahr mentioned this pull request Mar 2, 2021

Coinstats Index #19521

Merged

fjahr force-pushed the genesisblockstats branch from 37bbfe9 to 1c18449 Compare April 11, 2021 18:56

fjahr changed the title ~~rpc: Fix getblockstats issues~~ rpc, test: Improve getblockstats for unspendables Apr 11, 2021

DrahtBot removed the Needs rebase label Apr 11, 2021

DrahtBot mentioned this pull request May 18, 2021

test: Use COINBASE_MATURITY in functional tests #21989

Merged

DrahtBot mentioned this pull request May 28, 2021

assumeutxo #15606

Closed

18 tasks

DrahtBot added the Needs rebase label May 31, 2021

fjahr force-pushed the genesisblockstats branch from 1c18449 to 0af88a8 Compare May 31, 2021 21:26

DrahtBot removed the Needs rebase label May 31, 2021

fjahr force-pushed the genesisblockstats branch from d41254a to 884e7e1 Compare May 21, 2022 21:35

luke-jr pushed a commit to luke-jr/bitcoin that referenced this pull request May 21, 2022

test: Test exclusion of OP_RETURN from getblockstats

5c57371

Github-Pull: bitcoin#19888 Rebased-From: ff16851

DrahtBot mentioned this pull request Jun 29, 2022

indexes: Stop using node internal types #25494

Merged

DrahtBot added the Needs rebase label Jul 19, 2022

fjahr force-pushed the genesisblockstats branch from 884e7e1 to 7232bd2 Compare July 20, 2022 22:03

DrahtBot removed the Needs rebase label Jul 20, 2022

DrahtBot mentioned this pull request Jul 21, 2022

assumeutxo: snapshot initialization #25667

Merged

DrahtBot mentioned this pull request Jul 29, 2022

assumeutxo: background validation completion #25740

Merged

DrahtBot added the Needs rebase label Oct 13, 2022

fanquake mentioned this pull request Oct 22, 2022

rpc: fix getblockstats for genesis block #26369

Closed

fjahr added 4 commits October 23, 2022 01:33

validation, index: Add unspendable coinbase helper functions

cb94db1

Making the checks to identify BIP30 available outside of validation.cpp is needed for reporting and tracking statistics on specific blocks and the UTXO set correctly.

rpc: Improve getblockstats

2ca5a49

- Fix getblockstats for block height 0 which previously returned an error. - Introduce alternative utxo_*_actual statistics which exclude unspendables: Genesis block, BIP30, unspendable outputs - Update test data - Explicitly test Genesis block results

test: Test exclusion of OP_RETURN from getblockstats

d885bb2

theStack mentioned this pull request Oct 22, 2022

rpc: Return accurate results for scanblocks #26325

Merged

fjahr force-pushed the genesisblockstats branch from 7232bd2 to d885bb2 Compare October 23, 2022 11:21

DrahtBot removed the Needs rebase label Oct 23, 2022

DrahtBot mentioned this pull request Oct 23, 2022

rpc/rest/zmq: reduce LOCK(cs_main) scope: ~6 times as many requests per second #26308

Merged

aureleoules approved these changes Oct 27, 2022

View reviewed changes

stickies-v approved these changes Oct 27, 2022

View reviewed changes

achow101 merged commit 5d9b530 into bitcoin:master Dec 5, 2022

sidhujag pushed a commit to syscoin/syscoin that referenced this pull request Dec 6, 2022

Merge bitcoin#19888: rpc, test: Improve getblockstats for unspendables

477381a

bitcoin locked and limited conversation to collaborators Dec 5, 2023

	/** Identifies blocks which coinbase output was subsequently overwritten in the UTXO set (see BIP30) */
	/** Identifies blocks whose coinbase output was subsequently overwritten in the UTXO set (see BIP30) */

	/** Identifies blocks that overwrote an existing coinbase output in the UTXO set (see BIP30) */
	/** Did this block overwrite an existing coinbase output in the UTXO set? (see BIP30) */

	/** Identifies blocks which coinbase output was subsequently overwritten in the UTXO set (see BIP30) */
	/** Was this block's coinbase output overwritten in the UTXO set by a later block? (see BIP30) */

		{RPCResult::Type::NUM, "utxo_increase_actual", /optional=/true, "The increase/decrease in the number of unspent outputs, not counting unspendables"},
		{RPCResult::Type::NUM, "utxo_size_inc_actual", /optional=/true, "The increase/decrease in size for the utxo index, not counting unspendables"},

		{RPCResult::Type::NUM, "utxo_size_inc", /optional=/true, "The increase/decrease in size for the utxo index (not discounting op_return and similar)"},
		{RPCResult::Type::NUM, "utxo_increase_actual", /optional=/true, "The increase/decrease in the number of unspent outputs, not counting unspendables"},

rpc, test: Improve getblockstats for unspendables #19888

rpc, test: Improve getblockstats for unspendables #19888

Uh oh!

Conversation

fjahr commented Sep 5, 2020

Uh oh!

Uh oh!

n-thumann left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

maflcko commented Sep 6, 2020

Uh oh!

fjahr commented Sep 6, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

theStack commented Sep 12, 2020

Uh oh!

DrahtBot commented Oct 2, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Conflicts

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fjahr commented Dec 24, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fjahr commented Apr 11, 2021

Uh oh!

fjahr commented May 21, 2022

Uh oh!

fjahr commented Jul 20, 2022

Uh oh!

fjahr commented Oct 23, 2022

Uh oh!

aureleoules left a comment

Choose a reason for hiding this comment

Master

PR

Uh oh!

stickies-v left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

fjahr commented Sep 6, 2020 •

edited

Loading

DrahtBot commented Oct 2, 2020 •

edited

Loading