RPC: Introduce getblockstats to plot things #10757

jtimon · 2017-07-06T23:21:21Z

It returns per block statistics about several things. It should be easy to add more if people think of other things to add or remove some if I went too far (but once written, why not keep it? EDIT: answer: not to test or maintain them).

The currently available options are: minfee,maxfee,totalfee,minfeerate,maxfeerate,avgfee,avgfeerate,txs,ins,outs (EDIT: see updated list in the rpc call documentation)

For the x axis, one can use height or block.nTime (I guess I could add mediantime if there's interest [EDIT: nobody showed interest but I implemented mediantime nonetheless, in fact there's no distinction between x or y axis anymore, that's for the caller to judge]).

To calculate fees, -txindex is required.

clarkmoody · 2017-07-07T15:03:31Z

Some ideas for additions:

Non-fee total output amount
Coinbase reward
Money supply including this block
Transaction weight txweight (it can be derived from existing fields, however)

I would prefer to see both time and mediantime returned, since they are available.

Should we return non-independent fields, such as avgfee when also including totalfee and txs?

I find that for bitcoin-related data, the median is often more useful than the average of a distribution. Including medianweight, medianfee, medianfeerate, medianoutput etc would expose these useful quantities to the user.

jnewbery · 2017-07-07T15:28:30Z

but once written, why not keep it?

Because more code => more bugs and more maintenance effort. I prefer:

If it's not really needed, why add it?

This is perhaps a nice-to-have, but since #8704, getblock can return all transactions in a block (without requiring txindex). Those can then be parsed and analysed offline.

Is there a compelling use-case I'm missing here? This seems like a feature only a small subset of users would be interested in, in which case an offline tools seems more appropriate.

Sorry - not meaning to be negative, but my default reaction to new RPCs/arguments tends towards NACK unless I can see a compelling and widespread use-case.

clarkmoody · 2017-07-07T15:50:38Z

This is perhaps a nice-to-have, but since #8704, getblock can return all transactions in a block (without requiring txindex). Those can then be parsed and analysed offline.

This code pulls each transaction input's previous outpoint in order to compute transaction fees. Replicating that in RPC would require thousands of calls for most blocks.

jnewbery · 2017-07-07T15:53:50Z

This code pulls each transaction input's previous outpoint

Ah yes, of course. ~~Concept ACK~~ in that case. Doing this with getblock / getrawtransaction is infeasible.

EDIT: I'm going to reverse myself again: I don't think +700 lines is worthwhile for something with limited usage for most users. I'm -0 on this.

jtimon · 2017-07-07T16:29:24Z

Because more code => more bugs and more maintenance effort. I prefer:

Sure, but I mean, removing for example the avgfee or avgfeerate won't safe much code or testing code, just a few lines. Forget I said this, if there's specific functions to remove because nobody will want them, let's remove those and focus on the ones people want. Adding specific things only a few people want can also happen in their own branches, so it's no big deal.

The only use case is gather statistics, presumably to plot things, create charts. That is, at least, compelling to me, but I don't think that will have widespread usage. I also don't think all rpc calls have it. Is getchaintxstats, for example, a widespread use case?

If that's enough reason not to merge this, it's fine, I can maintain it as a separate branch that I periodically rebase, it is simple enough, so that won't be a big deal. On the other hand, if I can get it reviewed and merged it'll be less work for me in the long run and I also get the review.

Non-fee total output amount
Coinbase reward

Sounds good.

Money supply including this block

Mhmm, it would be simpler to calculate here from start to end here than from genesis. But it's pretty trivial to write a function in any language that returns the total supply for a given height without access to any historic data. Unless you are talking about discounting op_return outputs or something like that. I don't think this is very interesting here. Perhaps that can be done in getchaintxstats ?

Transaction weight txweight (it can be derived from existing fields, however)

In fact I'm using weight for everything. I should s/size/weight/ and probably also show size separately.
Maybe separate feerates in by weight and serialize size? I don't know...

I would prefer to see both time and mediantime returned, since they are available.

Yeah, the mediantime takes a little bit longer to be calculated but not much and one can always disable anything. In fact, the height and time shouldn't be treated in any special way for being "the x axis" and should be allowed to be disabled like the rest.

Should we return non-independent fields, such as avgfee when also including totalfee and txs?

This is a good question. This is mostly what I meant by "why not if it's this easy?".
But yeah, I guess non-independent are good candidates to be removed.

re median: yeah, that sounds interesting too, good idea!

clarkmoody · 2017-07-07T16:37:46Z

Mhmm, it would be simpler to calculate here from start to end here than from genesis. But it's pretty trivial to write a function in any language that returns the total supply for a given height without access to any historic data. Unless you are talking about discounting op_return outputs or something like that. I don't think this is very interesting here. Perhaps that can be done in getchaintxstats ?

I was thinking of the more trivial version, rather than the supply - provably_unspendable version, so keeping that as external code makes more sense. Maintaining the sum of spendable outputs against block height is a much more ambitious idea, and it may make sense in the future. However, it is probably out of scope of this PR.

promag

Partial review, suggestion to use std::set.

Nit, rename allowed_plot_values to valid_plot_values.
Nit, rename getperblockstats to just getblockstats?

promag · 2017-07-07T22:37:58Z

src/rpc/blockchain.cpp

+
+UniValue getperblockstats(const JSONRPCRequest& request)
+{
+    std::string str_allowed_plot_values = "minfee,maxfee,totalfee,minfeerate,maxfeerate,avgfee,avgfeerate,txs,ins,outs";


promag · 2017-07-07T22:52:25Z

src/rpc/blockchain.cpp

+UniValue getperblockstats(const JSONRPCRequest& request)
+{
+    std::string str_allowed_plot_values = "minfee,maxfee,totalfee,minfeerate,maxfeerate,avgfee,avgfeerate,txs,ins,outs";
+    std::vector<std::string> allowed_plot_values;


std::set<std::string> allowed_plot_values = {"minfee", "maxfee", "..."};

promag · 2017-07-07T22:53:29Z

src/rpc/blockchain.cpp

+{
+    std::string str_allowed_plot_values = "minfee,maxfee,totalfee,minfeerate,maxfeerate,avgfee,avgfeerate,txs,ins,outs";
+    std::vector<std::string> allowed_plot_values;
+    boost::split(allowed_plot_values, str_allowed_plot_values, boost::is_any_of(","));


promag · 2017-07-07T22:54:32Z

src/rpc/blockchain.cpp

+            "\nArguments:\n"
+            "1. \"start\"      (numeric, required) The height of the block that starts the window.\n"
+            "2. \"end\"        (numeric, optional) The height of the block that ends the window (default: current tip).\n"
+            "3. \"plotvalues\"  (string,  optional) Values to plot (comma separated), default(all): " + str_allowed_plot_values +


"..." + boost::join(allowed_plot_values, ",")

promag · 2017-07-07T22:55:17Z

src/rpc/blockchain.cpp

+            "3. \"plotvalues\"  (string,  optional) Values to plot (comma separated), default(all): " + str_allowed_plot_values +
+            "\nResult:\n"
+            "{\n"
+            "}\n"


Missing result example.

promag · 2017-07-07T22:55:25Z

src/rpc/blockchain.cpp

+            "}\n"
+            "\nExamples:\n"
+            + HelpExampleCli("getperblockstats", "1000 1000 \"minfeerate,avgfeerate\"")
+            + HelpExampleRpc("getperblockstats", "1000 1000 \"maxfeerate,avgfeerate\"")


One is HelpExampleCli and the other is HelpExampleRpc

Ops sorry, overlooked it.

promag · 2017-07-07T23:00:26Z

src/rpc/blockchain.cpp

+    if (request.params.size() > 2) {
+        str_plot_values = request.params[2].get_str();
+    }
+    std::vector<std::string> plot_values;


std::set<std::string> plot_values; if (request.params.size() > 2) { boost::split(plot_values, request.params[2].get_str(), boost::is_any_of(",")); // only validate in this case // ... } else { plot_values = allowed_plot_values; }

promag · 2017-07-07T23:03:29Z

src/rpc/blockchain.cpp

+    }
+}
+
+static bool IsAllowedPlotValue(const std::string& plot_value, std::vector<std::string>& allowed_plot_values)


promag · 2017-07-07T23:07:37Z

src/rpc/blockchain.cpp

+    std::vector<std::string> plot_values;
+    boost::split(plot_values, str_plot_values, boost::is_any_of(","));
+    for (const std::string plot_value : plot_values) {
+        if (!IsAllowedPlotValue(plot_value, allowed_plot_values)) {


if (allowed_plot_values.count(plot_value) == 0) {

duh, I was so much over-complicating things so much for no good reason...thank you!

jtimon · 2017-07-08T07:27:36Z

Thanks again for the great feedback!

@promag I think I solved all your nits except for #10757 (comment)

@clarkmoody I think I added most of your suggestions, explicitly excluding anything that involved accumulations neither from height=1 nor from height=start.
The former potentially implies a world of complexity and the latter can be trivially calculated on the visual side: I would completely discard any accumulator redundancy in this rpc beforehand.

And for the rest of the redundancies, @jnewbery and @clarkmoody - thanks again for pointing it out -, it's never too late to remove them before merging like a trivial squash and it's never too soon to start saying which ones you would bikesay* out first. Also bikesay the names for the curves and even the order in the list (duplicated for c++ and python).

In the meantime, I embraced redundancy since, as said, it will be trivial for me to remove later. And also the pertinent optimizations to skip calculations when plot_values.count("minfee") == 0 or actually only when the extra calculation is more expensive than the searching in plot_values which is a set of strings.

For example, we have blockfees, reward, subsidy, complying with consensus rule reward == blockfees + subsidy. Only 2 of the 3 are necessary, at least one is redundant. My personal preference is removing either subsidy or reward or subsidy, but not blockfees. But at said once written there's no problem with me in just making sure their tests don't surprise me until we decide which ones didn't deserve it.

Which one seems bikesaying in principle. But not in this case.
blockfees/total_fees serves for other calculations like avgfeerate. Let's not remove that one, just rename it.

But it is more interesting to propose new ones than to rename or vote for removal IMO. I believe the most interesting addition to this point was utxo_size_inc, which would welcomed some review from people who measures sizes more carefully like @sipa , since this doesn't use GetSerializeSize for Coin intentionally, independently of the optimization to read Coin if available in the utxo before calling RpcGetTx. I'm still not sure what to do with pre/post segwit feerates, does anybody care about the pre ones? which one needs the scale factor? none?

REM CalculateTruncatedMedian doesn't need to be a template at this point, but there's no harm being static IMO

EDIT: still some TODOs, mostly documentation and pending decisions

promag · 2017-07-08T07:36:50Z

@jtimon no problem. There are some nits to fix but I'll review more in depth later.

promag · 2017-07-09T00:11:49Z

src/policy/feerate.cpp

@@ -20,20 +20,22 @@ CFeeRate::CFeeRate(const CAmount& nFeePaid, size_t nBytes_)
        nSatoshisPerK = 0;
 }

-CAmount CFeeRate::GetFee(size_t nBytes_) const
+CAmount CFeeRate::GetTruncatedFee(size_t nBytes_) const


Drop _? Same below.

Well, yes, conserving the old name only saves 1 line of extra disruption. But I guess if we're touching the variable name we should use the new style. just bytes?

promag · 2017-07-09T00:13:04Z

src/rpc/blockchain.cpp

@@ -685,6 +686,22 @@ UniValue getblockheader(const JSONRPCRequest& request)
    return blockheaderToJSON(pblockindex);
 }

+static void ReadBlockCheckPruned(const CBlockIndex* pblockindex, CBlock& block)


Keep argument order as ReadBlockFromDisk? Is there a convention for where the output arguments should be?

Not that I know of, but your proposed change sounds good to me.

promag · 2017-07-09T00:19:36Z

src/rpc/blockchain.cpp

+{
+    T median;
+    size_t size = scores.size();
+    std::sort(scores.begin(), scores.end());


Nit, could sort only after size == 1 case.

promag · 2017-07-10T15:34:29Z

src/rpc/blockchain.cpp

+}
+
+// outpoint (needed for the utxo index) + nHeight + fCoinBase
+static const size_t PER_UTXO_OVERHEAD = sizeof(COutPoint) + sizeof(uint32_t) + sizeof(bool);


static constexpr ...

what's the gain? https://stackoverflow.com/a/41132221/935325 says it's the same...

what's the gain? https://stackoverflow.com/a/41132221/935325 says it's the same...

Gain is just that constexpr is more descriptive, and that stackoverflow answer isn't really correct. const and constexpr aren't identical even for integers, for example const is valid here:

static const int X = rand();

where constexpr would not be:

static constexpr int X = rand();

Also that stackoverflow answer is relying on special case treatment for integers that doesn't apply to other types like floats. constexpr is just better for declaring compile time constants that const, so we should prefer it.

promag · 2017-07-10T15:34:43Z

src/rpc/blockchain.cpp

+    ReadBlockCheckPruned(pindex, block);
+
+    for (const auto& tx : block.vtx) {
+


Remove empty line.

promag · 2017-07-10T15:51:49Z

src/rpc/blockchain.cpp

+        map_stats[plot_value] = UniValue(UniValue::VARR);
+    }
+
+    CBlockIndex* pindex = chainActive[end];


promag · 2017-07-10T15:52:22Z

src/rpc/blockchain.cpp

+
+    CBlockIndex* pindex = chainActive[end];
+    for (int i = end; i >= start; i--) {
+        UpdateBlockStats(pindex, plot_values, map_stats);


UpdateBlockStats(chainActive[i], ...);

This will be slightly less efficient, no?

I guess it takes few more cycles but non critical code should be cleaner?

promag · 2017-07-10T15:54:58Z

src/rpc/blockchain.cpp

+            "\nArguments:\n"
+            "1. \"start\"      (numeric, required) The height of the block that starts the window.\n"
+            "2. \"end\"        (numeric, optional) The height of the block that ends the window (default: current tip).\n"
+            "3. \"plotvalues\"  (string,  optional) Values to plot (comma separated), default(all): " + boost::join(valid_plot_values, ",") +


Replace plotvalues with stats? Also, 3rd argument could be object options?

Isn't the string simpler?

Ignore options suggestion.

promag · 2017-07-10T15:56:31Z

src/rpc/blockchain.cpp

+    int end;
+    if (request.params.size() > 1) {
+        end = request.params[1].get_int();
+        if (end < 0 || end > chainActive.Height()) {


Nit, negative block could mean end = height - end to avoid early blocks (not new concept here I believe)?

Mhmm, interesting. To be clear you mean start=-10 end=200 would be equivalent to start=190 end=200, right?

No, I meant negative values are relative to the tip. To get the stats for the last 10 blocks you would pass start = -10 without querying the current block height.

jtimon · 2017-07-11T21:47:09Z

Here are some images generated using this branch in combination with (WIP): https://github.com/jtimon/rpc-explorer

GUI detail:

GUI detail zoom:

Hide some:

Hide more:

Fees:

Utxo size increase:

jtimon · 2017-07-12T05:54:14Z

Without the documentation for the result it was impossible to distinguish a weird choice to spring discussion from an implementation mistake. Removed the other TODO comments.
Coded more pending suggestions by @promag (hopefully all pending ones? if not, please insist) with some extra bikeshedding derived from s/plotvalues/stats/ and adapt tests to start and end being allowed to be negative.

More cleanups can be done, specially in the tests if we go further with #10757 (comment) and not calculate in inverse order (there's no point if we don't get the slight optimization).

jtimon · 2017-08-24T20:27:13Z

Needed rebase.
If somebody made a web for it, it may be interesting to show number of segwit txs too http://segwit.5gbfree.com/countsegwit

forklol · 2017-08-29T08:35:27Z

Just wanted to say that this would be massively helpful to track statistics. I hope this finds it's way into a release soon.

jtimon · 2017-08-31T06:57:27Z

Reversed the order of the values to the natural one, since as discussed the optimization of doing fetching the blocks in reverse order is not worth the loss in clarity of the code.
Added segwit tx counter stat, and also the total size and weight for those txs (txs that at least have one sw input, txs sending to sw outputs don't count).

trippysalmon · 2017-09-01T15:29:09Z

Perhaps a better name for <stat>_old is <stat>_virtual, _virt or _v. Or perhaps prepend it with "v" just like the tx size in the output of getrawtransaction (vsize).

For example:

avgfeerate_old becomes vavgfeerate
maxfeerate_old becomes vmaxfeerate
medianfeerate_old becomes vmedianfeerate

etc.

trippysalmon · 2017-09-03T17:34:35Z

I just finished calling getblockstats on every block in the chain and saving it into a database. I didn't encounter any issues and the performance is quite good (100-1000ms per "full" block on an i7 6900k /w 32gb ram + nvme ssd).

Btw, if anyone is interested in the dataset I can share it. Just convo me at freenode irc (nick: "trippysalmon"). It includes some other stats as well, like rolling average hashrates.

promag · 2018-05-07T13:04:12Z

test/functional/rpc_getblockstats.py

+        assert_raises_rpc_error(-5, 'Block not found', self.nodes[0].getblockstats,
+                                hash_or_height='000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f')
+
+


Linter error:

W293 blank line contains whitespace.

promag · 2018-05-15T13:21:53Z

Lightly tested ACK 563eee9.

kallewoof

utACK 563eee92c9ddc5b537fca1ce08bce811d97aed97 w/ minor nits

kallewoof · 2018-05-17T07:59:42Z

src/rpc/blockchain.cpp

+            "      ,...\n"
+            "    ]\n"
+            "\nResult: (all values are in reverse order height-wise)\n"
+            "{                             (json object)\n"


Due to two escapes in below lines, this needs to remove two spaces for alignment.

kallewoof · 2018-05-17T08:05:38Z

src/rpc/blockchain.cpp

+        const uint256 hash(uint256S(strHash));
+        pindex = LookupBlockIndex(hash);
+        if (!pindex) {
+            throw JSONRPCError(RPC_INVALID_ADDRESS_OR_KEY, "Block not found");


Just a note here. In

bitcoin/src/rpc/blockchain.cpp

Lines 809 to 818 in 4cfe17c

if (fHavePruned && !(pblockindex->nStatus & BLOCK_HAVE_DATA) && pblockindex->nTx > 0)

throw JSONRPCError(RPC_MISC_ERROR, "Block not available (pruned data)");

if (!ReadBlockFromDisk(block, pblockindex, Params().GetConsensus()))

// Block not found on disk. This could be because we have the block

// header in our index but don't have the block (for example if a

// non-whitelisted node sends us an unrequested long chain of valid

// blocks, we add the headers to our index, but don't accept the

// block).

throw JSONRPCError(RPC_MISC_ERROR, "Block not found on disk");

RPC_MISC_ERROR is used, while RPC_INVALID_ADDRESS_OR_KEY is used here. Not sure which is better, but I think they should both use the same error type. (This PR touches the other one so could easily tweak, I think.)

I disagree. If the header is not found, that means the request was invalid, hence RPC_INVALID ADDRESS_OR KEY (key loosely taken to mean request argument, I take it). Whereas if the header is known but the block is not known, I do consider that to be a different error.

In #13144, I introduce RPC_DATA_UNAVAILABLE, which may be more appropriate than RPC_MISC in the latter case.

kallewoof · 2018-05-17T08:13:39Z

src/rpc/blockchain.cpp

+            totalfee += txfee;
+
+            // New feerate uses satoshis per virtual byte instead of per serialized byte
+            CAmount feerate = weight ? (txfee * WITNESS_SCALE_FACTOR) / weight : 0;


May be ok, but I think (txfee * WITNESS_SCALE_FACTOR) / (weight + WITNESS_SCALE_FACTOR - 1) is precisely accurate while the above will have truncation errors sometimes.

That equation has truncated errors. Consider a tx with weight 3 and fee 100. The vsize is 1, so the fee rate should be 1, but your equation would give (100 * 4) / 6 = 66.

You point out correctly though that vsize is computed incorrectly here by rounding down rather than up. This should use GetVirtualTransactionSize().

Sorry, I don't follow. This is using GetTransactionWeight() which does ::GetSerializeSize(tx, SER_NETWORK, PROTOCOL_VERSION | SERIALIZE_TRANSACTION_NO_WITNESS) * (WITNESS_SCALE_FACTOR - 1) + ::GetSerializeSize(tx, SER_NETWORK, PROTOCOL_VERSION)

GetVirtualTransactionSize does (std::max(nWeight, nSigOpCost * nBytesPerSigOp) + WITNESS_SCALE_FACTOR - 1) / WITNESS_SCALE_FACTOR

But I'm not sure why would we prefer that here.

Sorry, I think I got things mixed up.

The fee rate for a tx with weight 3 and fee 100 is the same as if there were 100 such tx's. That would have weight 300 or vsize 75, and fee 10,000, for a fee rate of 10,000/75 or 133.3, not 1, 100, or 66. :) Formula looks fine to me, though I guess estimatefee uses fee/vkbyte rather than fee/byte, which would be (txfee * WITNESS_SCALE_FACTOR * 1000) / weight.

Well, we can move from sat/vbyte to sat/vkbyte, sure. I t should be a simple change if people prefer that.

ajtowns

ACK 563eee92c9ddc5b537fca1ce08bce811d97aed97 modulo nits

ajtowns · 2018-05-18T06:55:11Z

src/rpc/blockchain.cpp

+            "\nIt won't work for some heights with pruning.\n"
+            "\nIt won't work without -txindex for utxo_size_inc, *fee or *feerate stats.\n"
+            "\nArguments:\n"
+            "1. \"hash_or_height\"     (string or numeric, required) The block hash or height of the target block. If height, negative values count back from the current tip\n"


But, negative values don't count back from the current tip, they give an error?

ajtowns · 2018-05-18T06:55:32Z

src/rpc/blockchain.cpp

+
+static UniValue getblockstats(const JSONRPCRequest& request)
+{
+    if (request.fHelp || request.params.size() < 1 || request.params.size() > 4)


Multiline if statement should have braces.

ajtowns · 2018-05-18T06:58:13Z

src/rpc/blockchain.cpp

+            "      \"time\",     (string, optional) Selected statistic\n"
+            "      ,...\n"
+            "    ]\n"
+            "\nResult: (all values are in reverse order height-wise)\n"


"(all values are in reverse order height-wise)" is outdated now that this only does one block's stats, I think? If it's not, it doesn't make sense to me...

@ajtowns

Includes commit from Anthony Towns @ajtowns: Tests: Save and load block and corresponding expected statistics

jtimon · 2018-05-22T21:29:58Z

Fixed help nits (also made other improvements, some other things in the help were ugly).

41d0476 Tests: Add data file (Anthony Towns) 4cbfb6a Tests: Test new getblockstats RPC (Jorge Timón) 35e77a0 RPC: Introduce getblockstats (Jorge Timón) cda8e36 Refactor: RPC: Separate GetBlockChecked() from getblock() (Jorge Timón) Pull request description: It returns per block statistics about several things. It should be easy to add more if people think of other things to add or remove some if I went too far (but once written, why not keep it? EDIT: answer: not to test or maintain them). The currently available options are: minfee,maxfee,totalfee,minfeerate,maxfeerate,avgfee,avgfeerate,txs,ins,outs (EDIT: see updated list in the rpc call documentation) For the x axis, one can use height or block.nTime (I guess I could add mediantime if there's interest [EDIT: nobody showed interest but I implemented mediantime nonetheless, in fact there's no distinction between x or y axis anymore, that's for the caller to judge]). To calculate fees, -txindex is required. Tree-SHA512: 2b2787a3c7dc4a11df1fce62c8a4c748f5347d7f7104205d5f0962ffec1e0370c825b49fd4d58ce8ce86bf39d8453f698bcd46206eea505f077541ca7d59b18c

… pruned or not e9a1881 refactor: add a function for determining if a block is pruned or not (Karl-Johan Alm) Pull request description: The check for whether a block is pruned or not is sufficiently obscure that it deserves a macro. It is also used in 2 places, ~~with more coming, e.g. #10757~~ (turns out it was a move, not an addition). Tree-SHA512: b9aeb60663e1d1196df5371d5aa00b32ff5d4cdea6a77e4b566f28115cce09570c18e45e4b81a4033f67c4135c8e32c027f67bae3b75c2ea4564285578a3f4dd

41d0476 Tests: Add data file (Anthony Towns) 4cbfb6a Tests: Test new getblockstats RPC (Jorge Timón) 35e77a0 RPC: Introduce getblockstats (Jorge Timón) cda8e36 Refactor: RPC: Separate GetBlockChecked() from getblock() (Jorge Timón) Pull request description: It returns per block statistics about several things. It should be easy to add more if people think of other things to add or remove some if I went too far (but once written, why not keep it? EDIT: answer: not to test or maintain them). The currently available options are: minfee,maxfee,totalfee,minfeerate,maxfeerate,avgfee,avgfeerate,txs,ins,outs (EDIT: see updated list in the rpc call documentation) For the x axis, one can use height or block.nTime (I guess I could add mediantime if there's interest [EDIT: nobody showed interest but I implemented mediantime nonetheless, in fact there's no distinction between x or y axis anymore, that's for the caller to judge]). To calculate fees, -txindex is required. Tree-SHA512: 2b2787a3c7dc4a11df1fce62c8a4c748f5347d7f7104205d5f0962ffec1e0370c825b49fd4d58ce8ce86bf39d8453f698bcd46206eea505f077541ca7d59b18c

* Merge bitcoin#10757: RPC: Introduce getblockstats to plot things 41d0476 Tests: Add data file (Anthony Towns) 4cbfb6a Tests: Test new getblockstats RPC (Jorge Timón) 35e77a0 RPC: Introduce getblockstats (Jorge Timón) cda8e36 Refactor: RPC: Separate GetBlockChecked() from getblock() (Jorge Timón) Pull request description: It returns per block statistics about several things. It should be easy to add more if people think of other things to add or remove some if I went too far (but once written, why not keep it? EDIT: answer: not to test or maintain them). The currently available options are: minfee,maxfee,totalfee,minfeerate,maxfeerate,avgfee,avgfeerate,txs,ins,outs (EDIT: see updated list in the rpc call documentation) For the x axis, one can use height or block.nTime (I guess I could add mediantime if there's interest [EDIT: nobody showed interest but I implemented mediantime nonetheless, in fact there's no distinction between x or y axis anymore, that's for the caller to judge]). To calculate fees, -txindex is required. Tree-SHA512: 2b2787a3c7dc4a11df1fce62c8a4c748f5347d7f7104205d5f0962ffec1e0370c825b49fd4d58ce8ce86bf39d8453f698bcd46206eea505f077541ca7d59b18c * Replace get_mocktime() usage with self.mocktime

…hpay#3058) * Merge bitcoin#10757: RPC: Introduce getblockstats to plot things 41d0476 Tests: Add data file (Anthony Towns) 4cbfb6a Tests: Test new getblockstats RPC (Jorge Timón) 35e77a0 RPC: Introduce getblockstats (Jorge Timón) cda8e36 Refactor: RPC: Separate GetBlockChecked() from getblock() (Jorge Timón) Pull request description: It returns per block statistics about several things. It should be easy to add more if people think of other things to add or remove some if I went too far (but once written, why not keep it? EDIT: answer: not to test or maintain them). The currently available options are: minfee,maxfee,totalfee,minfeerate,maxfeerate,avgfee,avgfeerate,txs,ins,outs (EDIT: see updated list in the rpc call documentation) For the x axis, one can use height or block.nTime (I guess I could add mediantime if there's interest [EDIT: nobody showed interest but I implemented mediantime nonetheless, in fact there's no distinction between x or y axis anymore, that's for the caller to judge]). To calculate fees, -txindex is required. Tree-SHA512: 2b2787a3c7dc4a11df1fce62c8a4c748f5347d7f7104205d5f0962ffec1e0370c825b49fd4d58ce8ce86bf39d8453f698bcd46206eea505f077541ca7d59b18c * Replace get_mocktime() usage with self.mocktime

…lock is pruned or not e9a1881 refactor: add a function for determining if a block is pruned or not (Karl-Johan Alm) Pull request description: The check for whether a block is pruned or not is sufficiently obscure that it deserves a macro. It is also used in 2 places, ~~with more coming, e.g. bitcoin#10757~~ (turns out it was a move, not an addition). Tree-SHA512: b9aeb60663e1d1196df5371d5aa00b32ff5d4cdea6a77e4b566f28115cce09570c18e45e4b81a4033f67c4135c8e32c027f67bae3b75c2ea4564285578a3f4dd

jtimon mentioned this pull request Jul 6, 2017

Rpc: Introduce call for per-block fee statistics ElementsProject/elements#185

Merged

jtimon force-pushed the b15-rpc-plotter branch from 457c30d to 53ea4de Compare July 7, 2017 00:01

fanquake added the RPC/REST/ZMQ label Jul 7, 2017

promag reviewed Jul 7, 2017

View reviewed changes

jtimon force-pushed the b15-rpc-plotter branch from 53ea4de to c316a9f Compare July 8, 2017 06:06

promag reviewed Jul 10, 2017

View reviewed changes

jtimon changed the title ~~RPC: Introduce getperblockstats to plot things~~ RPC: Introduce getblockstats to plot things Jul 11, 2017

jtimon force-pushed the b15-rpc-plotter branch 3 times, most recently from 79d9907 to 90c7264 Compare July 12, 2017 05:43

jnewbery mentioned this pull request Jul 13, 2017

Add histunspent RPC #10804

Closed

jtimon force-pushed the b15-rpc-plotter branch from 90c7264 to 33d2bda Compare August 24, 2017 20:15

jtimon force-pushed the b15-rpc-plotter branch 3 times, most recently from 8bc1d6a to b2d93e4 Compare August 31, 2017 05:37

promag reviewed May 7, 2018

View reviewed changes

jnewbery mentioned this pull request May 8, 2018

Some ACKs missing from PRs PierreRochard/bitcoin-acks#12

Closed

jtimon force-pushed the b15-rpc-plotter branch 2 times, most recently from cedac47 to 563eee9 Compare May 9, 2018 00:00

kallewoof mentioned this pull request May 17, 2018

refactoring: add a method for determining if a block is pruned or not #13259

Merged

kallewoof approved these changes May 17, 2018

View reviewed changes

ajtowns approved these changes May 18, 2018

View reviewed changes

jtimon and others added 3 commits May 22, 2018 23:26

RPC: Introduce getblockstats

35e77a0

Tests: Test new getblockstats RPC

4cbfb6a

Includes commit from Anthony Towns @ajtowns: Tests: Save and load block and corresponding expected statistics

Tests: Add data file

41d0476

jtimon force-pushed the b15-rpc-plotter branch from 563eee9 to 41d0476 Compare May 22, 2018 21:28

laanwj merged commit 41d0476 into bitcoin:master May 23, 2018

Bushstar mentioned this pull request May 26, 2018

commits from bitcoin/master FeatherCoin/Feathercoin#336

Merged

marcinja mentioned this pull request Jun 18, 2018

Add getblockstats RPC to rpcclient btcsuite/btcd#1199

Closed

marcinja mentioned this pull request Jul 16, 2018

Add new statistics to getblockstats RPC bitcoinops/bitcoin#1

Closed

jtimon mentioned this pull request Aug 16, 2018

TODO for release notes 0.17.0 #12391

Closed

This was referenced Dec 4, 2018

Rpc: Introduce call for per-block fee statistics dashpay/dash#2519

Closed

feature request: rpc introduce call for per-block fee statistics zcash/zcash#3723

Closed

dagurval mentioned this pull request Sep 11, 2019

RPC: Introduce getblockstats to plot things BitcoinUnlimited/BitcoinUnlimited#1913

Merged

bitcoin locked as resolved and limited conversation to collaborators Dec 16, 2021

		ReadBlockCheckPruned(pindex, block);

		for (const auto& tx : block.vtx) {

		assert_raises_rpc_error(-5, 'Block not found', self.nodes[0].getblockstats,
		hash_or_height='000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f')

	if (fHavePruned && !(pblockindex->nStatus & BLOCK_HAVE_DATA) && pblockindex->nTx > 0)
	throw JSONRPCError(RPC_MISC_ERROR, "Block not available (pruned data)");

	if (!ReadBlockFromDisk(block, pblockindex, Params().GetConsensus()))
	// Block not found on disk. This could be because we have the block
	// header in our index but don't have the block (for example if a
	// non-whitelisted node sends us an unrequested long chain of valid
	// blocks, we add the headers to our index, but don't accept the
	// block).
	throw JSONRPCError(RPC_MISC_ERROR, "Block not found on disk");

RPC: Introduce getblockstats to plot things #10757

RPC: Introduce getblockstats to plot things #10757

Uh oh!

Conversation

jtimon commented Jul 6, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

clarkmoody commented Jul 7, 2017

Uh oh!

jnewbery commented Jul 7, 2017

Uh oh!

clarkmoody commented Jul 7, 2017

Uh oh!

jnewbery commented Jul 7, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jtimon commented Jul 7, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

clarkmoody commented Jul 7, 2017

Uh oh!

promag left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jtimon commented Jul 8, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

promag commented Jul 8, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

promag Jul 10, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

jtimon commented Jul 6, 2017 •

edited

Loading

jnewbery commented Jul 7, 2017 •

edited

Loading

jtimon commented Jul 7, 2017 •

edited

Loading

jtimon commented Jul 8, 2017 •

edited

Loading

promag Jul 10, 2017 •

edited

Loading

jtimon commented Jul 11, 2017 •

edited

Loading

trippysalmon commented Sep 3, 2017 •

edited

Loading