Skip to content

doc/zmq: Note about endianness does not match reality #31856

@jirijakes

Description

@jirijakes

Is there an existing issue for this?

  • I have searched the existing issues

Current behaviour

PR #23471 added a note to ZMQ's documentation page saying that:

[…] 32-byte hashes are in Little Endian and not in the Big Endian format that the RPC interface and block explorers use to display transaction and block hashes.

Also:

| hashtx | <32-byte transaction hash in Little Endian> | <uint32 sequence number in Little Endian>
| hashblock | <32-byte block hash in Little Endian> | <uint32 sequence number in Little Endian>

However, unless I am missing something, transaction and block hashes in both ZMQ and RPC appear in the same, reversed byte order (big endian).

If this is confirmed, I would like to prepare PR for the documentation.

Expected behaviour

Documentation would not contain references to hashes being in Little Endian.

Steps to reproduce

Start regtest with ZMQ hashblock:

bitcoind -regtest -datadir=zmqtest -server --daemon -zmqpubhashblock=tcp://0.0.0.0:43441
bitcoin-cli -regtest -datadir=zmqtest createwallet ""

Run independent ZMQ client in Python:

  import zmq
  import binascii

  context = zmq.Context()
  socket = context.socket(zmq.SUB)

  socket.connect("tcp://localhost:43441")
  socket.setsockopt_string(zmq.SUBSCRIBE, "hashblock")

  topic = socket.recv()
  data = socket.recv()
  seq = socket.recv()

  print(f"Topic: {topic}")
  print(f"Data:  {binascii.hexlify(data)}")
  print(f"Seq:   {binascii.hexlify(seq)}")

Generate block:

bitcoin-cli -regtest -datadir=zmqtest -generate 1
{
  "address": "bcrt1qpjy0a3ply66pwpj6mp8sn36aslwr3gvdq727v6",
  "blocks": [
    "4f36a8e4e1ff6ebcc13f6bec62841e44a8b4c281f1f1346f148987735fb72e0e"
  ]
}

Output of ZMQ client:

Topic: b'hashblock'
Data:  b'4f36a8e4e1ff6ebcc13f6bec62841e44a8b4c281f1f1346f148987735fb72e0e'
Seq:   b'00000000'

Calculate hash of the block header to crosscheck:

  header=$(bitcoin-cli -regtest -datadir=zmqtest getblockheader 4f36a8e4e1ff6ebcc13f6bec62841e44a8b4c281f1f1346f148987735fb72e0e false)
  echo $header | xxd -r -p | sha256sum | xxd -r -p | sha256sum
0e2eb75f738789146f34f1f181c2b4a8441e8462ec6b3fc1bc6effe1e4a8364f  -

0e2eb75f738789146f34f1f181c2b4a8441e8462ec6b3fc1bc6effe1e4a8364f ← hash in natural order (output of SHA256; little endian)
4f36a8e4e1ff6ebcc13f6bec62841e44a8b4c281f1f1346f148987735fb72e0e ← hash in reversed order (ZMQ, RPC; big endian)


I assume hashes are printed by GetHex(). The bytes are printed in reversed order:

bitcoin/src/uint256.cpp

Lines 10 to 18 in 55cf39e

template <unsigned int BITS>
std::string base_blob<BITS>::GetHex() const
{
uint8_t m_data_rev[WIDTH];
for (int i = 0; i < WIDTH; ++i) {
m_data_rev[i] = m_data[WIDTH - 1 - i];
}
return HexStr(m_data_rev);
}

To ZMQ, they are also sent in reversed order:

bool CZMQPublishHashBlockNotifier::NotifyBlock(const CBlockIndex *pindex)
{
uint256 hash = pindex->GetBlockHash();
LogDebug(BCLog::ZMQ, "Publish hashblock %s to %s\n", hash.GetHex(), this->address);
uint8_t data[32];
for (unsigned int i = 0; i < 32; i++) {
data[31 - i] = hash.begin()[i];
}
return SendZmqMessage(MSG_HASHBLOCK, data, 32);
}

Relevant log output

No response

How did you obtain Bitcoin Core

Package manager

What version of Bitcoin Core are you using?

28.0.0

Operating system and version

Arch Linux, 6.12

Machine specifications

No response

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions