Skip to content

Add block height support in rpc call getblock #8457

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed

Conversation

pedrobranco
Copy link
Contributor

@pedrobranco pedrobranco commented Aug 4, 2016

This is a simple PR: add support for getting a block by its height via RPC call getblock.

Instead of 2 calls bitcoin-cli getblockhash 0 + bitcoin-cli getblock <hash>
we call directly bitcoin-cli getblock 0.

Note: it will continue to support via block hash.

@laanwj
Copy link
Member

laanwj commented Aug 4, 2016

I don't like overloading the argument, making it ambiguous. It seems messy, and we don't do that anywhere else on the API.
E.g. a long hash with no hexadecimal characters could be interpreted as a long number.
This would need to be a new RPC call getbockbyheight or so.
Though even then I'm not convinced it's a worthwhile addition.

@pedrobranco
Copy link
Contributor Author

This would need to be a new RPC call getbockbyheight or so.

Given that getblock is not called getblockbyhash was the reason that I've added the height support.

Though even then I'm not convinced it's a worthwhile addition.

For a new RPC I also share your opinion, about the getblock IMHO I think is useful for developers.

@jonasschnelli
Copy link
Contributor

I think getbockbyheight could be handy. Concept ACK.

@promag
Copy link
Contributor

promag commented Aug 5, 2016

I don't like overloading the argument, making it ambiguous.

Disagree, there is a place for overloading in APIs. In this case blocks are usually referenced by height or hash.

There is also overloading in the protocol: 4 lock_time uint32_t A time (Unix epoch time) or block number. See the locktime parsing rules.

It seems messy, and we don't do that anywhere else on the API.

From help importaddress: 1. "script" (string, required) The hex-encoded script (or address) either.

E.g. a long hash with no hexadecimal characters could be interpreted as a long number.

Unlikely? Still it's simple to distinguish that block hash from a number.

Though even then I'm not convinced it's a worthwhile addition.

From the API point of view, I think it's fair to fetch a block from it's height. From a technical point of view, making 2 calls causing locks on cs_main etc on a heavy loaded daemon is resource wasting.

So, +1 for getblock hash|height (verbose).

@paveljanik
Copy link
Contributor

paveljanik commented Aug 5, 2016

I like the concept of one RPC call. But unfortunately there is one big caveat. In the future (and because of probability even in not so distant one), we can have a block whose hash is a hexadecimal number parseable as a decimal one resulting in the uncertain block referenced. We can't afford to have uncertain result...

So NACK.

So if we want to have an API to get block by hash, we can have one more RPC call or to live with

btc getblock `btc getblockhash 423787`

I'm fine even without the new RPC call...

FWIW: as of block 423792 we do not have any block whose hash written in hex is decimal number.

@pedrobranco
Copy link
Contributor Author

we can have a block whose hash is a hexadecimal number parseable as a decimal one

IMO it is possible to detect if a given input is a block hash, as it comes as string, and, given that block hashes have fixed length, I don't think that if we want get the block 5125122 we call like this:

getblock 0000000000000000000000000000000000000000000000000000000005125122

So it will push a fix to this special case.

So if we want to have an API to get block by hash, we can have one more RPC call or to live with

Imagine that you need to scan the whole blockchain, given that getblockhash RPC call takes 1 second to process (can be more or less):

  • 1 second * 423789 blocks (at this moment) = takes more 4.90 days to process.

So, IMO, a call like btc getblock 'btc getblockhash 423787' is not very optimized.

@paveljanik
Copy link
Contributor

Compare results of

btc getblock 00c937983704a73af28acdec37b049d214adbda81d7e2a3dd146f6ed09
btc getblock c937983704a73af28acdec37b049d214adbda81d7e2a3dd146f6ed09

@pedrobranco pedrobranco force-pushed the feature/add-get-block-by-number branch from b6638a6 to f32de13 Compare August 5, 2016 12:33
@pedrobranco
Copy link
Contributor Author

@laanwj @promag @paveljanik WDYT of supporting something like this:

  • As is today (to be deprecated maybe):
bitcoin-cli getblock hash (verbose)
  • With JSON input:
bitcoin-cli getblock '{ "hash": "0x123", "verbose": true}'
bitcoin-cli getblock '{ "block": 1, "verbose": false}'

@laanwj
Copy link
Member

laanwj commented Aug 13, 2016

It's better. But still this is special casing to work around one specific round-trip. This opens the flood-gates to many more.

To solve this more generally I plan on working on a cute little extension RFC to the JSONRPC 2.0 spec. See discussion in #7783 (comment) .

This will add a simple form of (in-batch) promise pipelining to JSON-RPC batching, which will allow doing this kind of composition in a more flexible way. Eg ext_store and ext_load below:

[
{
    "version": "x.x",
    "method": "getblockhash",
    "id": 0,
    "params": ["123456"],
    "ext_store": True   // Store the result of this request for later use
},
{
    "version": "x.x",
    "method": "getblock",
    "id": 1,
    "params": [
        null // Placeholder for argument 0, will be replaced by ext_load below
    ],
    "ext_load": [
        {
            "src": [0, "result"],  // Load the output of request id 0, ["result"]
            "dst": [0]             // into into params[0]
        }
    ],
}
]

The effect of which will be:

tmp0 = getblockhash(123456);
tmp1 = getblock(tmp0);

In contrast to simple nesting of calls, which would be the alternative extension, this also allows for more complex composition, and re-use of values.

Returning:

[
{
    "id": 0,
    "error": null,
    "result": "0000000000002917ed80650c6174aac8dfc46f5fe36480aaef682ff6cd83c3ca"
},
{
    "id": 1,
    "error": null,
    "result": {
        "hash": ...,
        "confirmations": ...,
        ...
    }
}
]

@pedrobranco
Copy link
Contributor Author

In contrast to simple nesting of calls, which would be the alternative extension, this also allows for more complex composition, and re-use of values.

Nice JSONRPC feature, delegating the responsibility of forwarding the output from the RPC client to the RPC server, but in our case it will still have the wallet locks and performance issues.

@sipa
Copy link
Member

sipa commented Aug 22, 2016

@pedrobranco None of these calls should grab a wallet lock...

@pedrobranco
Copy link
Contributor Author

Sorry, I've meant locks on cs_main.

@luke-jr
Copy link
Member

luke-jr commented Aug 29, 2016

  1. JSON has distinct types for String and Number, and we use String for hashes, so Number is free to be overloaded for height if so desired.
  2. I think it's a security risk to support getblock by height directly. People shouldn't use height to refer to specific blocks, since reorgs can change the block at a given height.

@dcousens
Copy link
Contributor

dcousens commented Aug 30, 2016

@luke-jr granted, but sometimes a quick way to check for re-orgs on a locally managed chain is to compare various heights between the chains [local & bitcoind], if they don't match up, you can start to check for where the orphan-fork occurred.

@laanwj
Copy link
Member

laanwj commented Sep 5, 2016

but in our case it will still have the wallet locks and performance issues.

That's an implementation detail and will likely go away in the future, and we should not design the RPC protocol around that. In any case what use-case to you have that requires shaving a few milliseconds from a getblock call? Have you benchmarked? I'm fairly sure most of the bottleneck is reading and parsing the block and formatting JSON [hopefully done without holding the cs_main lock] - if the latter, have you looked at the REST interface?

Edit: If you need a lot of getblock calls you can pipeline without any RPC extensions: in the first batch, run getblockhash on block number 0..X - this is what our linearize.py script does too. In the second batch you can then request all the actual block data using getblock.

I think it's a security risk to support getblock by height directly. People shouldn't use height to refer to specific blocks, since reorgs can change the block at a given height.

Interesting point. It's safe for old blocks, but for working at the tip, using it probably means you're not handling reorgs properly.

@laanwj
Copy link
Member

laanwj commented Sep 13, 2016

Closing this. See posts above for argumentation.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants