index: store per-block transaction locations for efficient lookups #32541

romanz · 2025-05-17T07:27:58Z

Currently, electrs and other indexers map between an address/scripthash to the list of the relevant transactions.

However, in order to fetch those transactions from bitcoind, electrs relies on reading the whole block and post-filtering for a specific transaction¹. Other indexers use a txindex to fetch a transaction using its txid ²³⁴.

The above approach has significant storage and CPU overhead, since the txid is a pseudo-random 32-byte value.

This PR is adding support for using the transaction's position within its block to be able to fetch it directly using REST API, using the following HTTP request (to fetch the N-th transaction from BLOCKHASH):

GET /rest/txfromblock/BLOCKHASH-N.bin

If binary response format is used, the transaction data will be read directly from the storage and sent back to the client, without any deserialization overhead.

The resulting index is much smaller (allowing it to be cached in RAM):

$ du -sh indexes/locations/ indexes/txindex/
2.5G	indexes/locations/
57G	indexes/txindex/

The new index is using the following DB schema:

struct DBKey {
    uint256 hash;   // blockhash
    uint32_t row;   // allow splitting one block's transactions into multiple DB rows
};

struct DBValue {
    FlatFilePos block_pos;          // file id + offset of the block
    std::vector<uint32_t> offsets;  // a list of transaction offsets within the block
};

I am working on a proof-of-concept indexer (https://github.com/romanz/bindex-rs) which is using #32540 & #32541 - please let me know if there are any questions/comments/concerns :)

DrahtBot · 2025-05-17T07:28:00Z

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/32541.

Reviews

See the guideline for information on the review process.

Type	Reviewers
Concept ACK	TheCharlatan, hodlinator

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#30469 (index: Fix coinstats overflow by fjahr)
#26966 (index: initial sync speedup, parallelize process by furszy)
#17783 (common: Disallow calling IsArgSet() on ALLOW_LIST options by ryanofsky)
#17581 (refactor: Remove settings merge reverse precedence code by ryanofsky)
#17580 (refactor: Add ALLOW_LIST flags and enforce usage in CheckArgFlags by ryanofsky)
#17493 (util: Forbid ambiguous multiple assignments in config file by ryanofsky)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

DrahtBot · 2025-05-17T07:37:15Z

🚧 At least one of the CI tasks failed.
_{Task lint: https://github.com/bitcoin/bitcoin/runs/42402043332}
_{LLM reason (✨ experimental): The CI failure is due to missing include guards in src/index/locationsindex.h.}

Hints

Try to run the tests locally, according to the documentation. However, a CI failure may still
happen due to a number of reasons, for example:

Possibly due to a silent merge conflict (the changes in this pull request being
incompatible with the current code in the target branch). If so, make sure to rebase on the latest
commit of the target branch.
A sanitizer issue, which can only be found by compiling with the sanitizer and running the
affected test.
An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

TheCharlatan · 2025-05-17T09:37:52Z

Concept ACK

Can you add the schema of the index and the expected arguments for the REST API to the pull request description? I was a bit confused at first if this now exposes the file position, but if I read it correctly now, this just allows querying a transaction by its index in the block.

romanz · 2025-05-17T10:00:56Z

Concept ACK

Thanks!

Can you add the schema of the index and the expected arguments for the REST API to the pull request description?

Sure - updated in #32541 (comment).

romanz · 2025-05-17T11:58:10Z

Fixed a few issues (following SonarQube run).

luke-jr · 2025-05-20T18:15:32Z

How does this compare to getrawtransaction <txid> 0 <blockhash> without a txindex?

romanz · 2025-05-21T09:51:22Z

I have used ApacheBench 2.3 for benchmarking REST API, and the following Rust client for getrawtransaction RPC:

fetching using the new index

$ ab -k -c 1 -n 100000 http://localhost:8332/rest/txfromblock/0000000000000000000083a0cff38278aae196d6d923a7e8ee7e5a0e371226fe-42.bin

Document Path:          /rest/txfromblock/0000000000000000000083a0cff38278aae196d6d923a7e8ee7e5a0e371226fe-42.bin
Document Length:        301 bytes

Concurrency Level:      1
Time taken for tests:   13.760 seconds
Complete requests:      100000
Failed requests:        0
Keep-Alive requests:    100000
Total transferred:      40500000 bytes
HTML transferred:       30100000 bytes
Requests per second:    7267.65 [#/sec] (mean)
Time per request:       0.138 [ms] (mean)
Time per request:       0.138 [ms] (mean, across all concurrent requests)
Transfer rate:          2874.41 [Kbytes/sec] received

fetching using `txindex`

$ ab -k -c 1 -n 100000 http://localhost:8332/rest/tx/4137d0dbad434d68a4f52b7bebcba91ddac3f7f5c92b84130432bd6b5e2ea57a.bin

Document Path:          /rest/tx/4137d0dbad434d68a4f52b7bebcba91ddac3f7f5c92b84130432bd6b5e2ea57a.bin
Document Length:        301 bytes

Concurrency Level:      1
Time taken for tests:   14.075 seconds
Complete requests:      100000
Failed requests:        0
Keep-Alive requests:    100000
Total transferred:      40500000 bytes
HTML transferred:       30100000 bytes
Requests per second:    7104.78 [#/sec] (mean)
Time per request:       0.141 [ms] (mean)
Time per request:       0.141 [ms] (mean, across all concurrent requests)
Transfer rate:          2810.00 [Kbytes/sec] received

fetching without `txindex`

time cargo run --release -- 4137d0dbad434d68a4f52b7bebcba91ddac3f7f5c92b84130432bd6b5e2ea57a 0000000000000000000083a0cff38278aae196d6d923a7e8ee7e5a0e371226fe
    Finished `release` profile [optimized] target(s) in 0.02s
     Running `target/release/bench-getrawtx 4137d0dbad434d68a4f52b7bebcba91ddac3f7f5c92b84130432bd6b5e2ea57a 0000000000000000000083a0cff38278aae196d6d923a7e8ee7e5a0e371226fe`
iterations = 1000
average RPC duration = 8.563491ms

real	0m8.628s
user	0m0.070s
sys	0m0.052s

Conclusions

The new LocationsIndex is only a few percent faster than the old TxIndex, but the on-disk footprint is ~22x smaller.
getrawtransaction which is used in the last benchmark has an average RPC duration of ~8.6ms vs ~0.14ms for the ones above.

DrahtBot · 2025-06-13T17:34:51Z

🚧 At least one of the CI tasks failed.
_{Task previous releases, depends DEBUG: https://github.com/bitcoin/bitcoin/runs/42406243587}
_{LLM reason (✨ experimental): The CI failure is caused by a missing header file test/util/index.h during compilation.}

Hints

Try to run the tests locally, according to the documentation. However, a CI failure may still
happen due to a number of reasons, for example:

Possibly due to a silent merge conflict (the changes in this pull request being
incompatible with the current code in the target branch). If so, make sure to rebase on the latest
commit of the target branch.
A sanitizer issue, which can only be found by compiling with the sanitizer and running the
affected test.
An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

romanz · 2025-06-13T19:02:07Z

Rebased to fix #32541 (comment).

TheCharlatan · 2025-06-15T14:34:19Z

How does this compare to getrawtransaction 0 without a txindex?

As far as I understand the index makes this operation faster by not requiring to read the entire block and then iterating through the transactions to find the match, which I am guessing is what the last benchmark is showing. romanz, would this new endpoint be used while creating the entire index initially, or to serve certain requests? It is not quite clear to me when an indexing client wouldn't want to read through the entire block and instead only get its transactions selectively.

romanz · 2025-06-15T17:20:24Z

As far as I understand the index makes this operation faster by not requiring to read the entire block and then iterating through the transactions to find the match

Correct - the proposed index improves the performance of fetching a single transaction (similar to txindex), requiring significantly less storage.

would this new endpoint be used while creating the entire index initially, or to serve certain requests?

I would like the new index to be used to serve history-related queries.
For example, https://electrum-protocol.readthedocs.io/en/latest/protocol-methods.html#blockchain-scripthash-get-history.

You are right that during the history indexing process, the client doesn't need the proposed index, since it needs to read both the entire block (and undo) data in order to map from ScriptPubKey to list of { block hash + transaction index within the block }.

BTW, I am working on a proof-of-concept indexer (https://github.com/romanz/bindex-rs) which is using #32540 & #32541 - please let me know if there are any questions/comments/concerns :)

maflcko

lgtm. I wonder if there should be a fallback?

src/rest.cpp

romanz · 2025-06-21T09:58:51Z

Rebased over master to use std::vector<std::byte> (following #32743).

romanz · 2025-07-14T18:54:35Z

Ping :)

romanz · 2025-07-19T17:18:51Z

I have split the content of this PR into 2 commits:

784459a is adding the new /rest/txfromblock/ endpoint
1b928f5 is adding an optional locationsindex for optimizing lookups

hodlinator

Concept ACK 1b928f5

Cool way of compressing transaction metadata to significantly reduce on-disk footprint compared to TxIndex and make it more likely this one fits in RAM. (Fine as long as external code can fetch by block hash + tx index instead of by txid).

Optimization as I understand it: All transactions within the same block have the same FlatFilePos base, which is compressed by a factor of 128 thanks to TXS_PER_ROW. Instead of storing an individual tx offset and size for each tx, it uses 2 offsets for a tx request, which allows re-use of offsets between adjacent txs.

Thanks for electrs!

src/node/blockstorage.cpp

src/index/locationsindex.cpp

doc/REST-interface.md

src/index/locationsindex.cpp

src/rest.cpp

src/index/locationsindex.cpp

src/index/locationsindex.h

src/test/locationsindex_tests.cpp

romanz · 2025-08-03T07:50:14Z

Many thanks for the review!
Applied the comments: 1b928f5 -> b2a22ce

romanz · 2025-08-04T12:48:03Z

Applied a few more fixes: b2a22ce -> 4441827

romanz · 2025-08-25T19:20:05Z

Ping :)

hodlinator

Reviewed 4441827

Sorry for the bag of nits, take from it what you will. The questions have higher priority for sure. Tried adding coverage of LocationsIndex to test/functional/feature_init.py but ran into interference with TxIndex, could be some more general issue though. Haven't had time to uncover the reason yet.

Suggestions+test branch:
4441827...hodlinator:bitcoin:pr/32541_suggestions

Further research idea:
It would be awesome to use sendfile(socket, filein, offset, count)/TransmitFile() for this to move that part to 100% kernel-space but not sure how we would cut through the abstractions for that:
https://www.man7.org/linux/man-pages/man2/sendfile.2.html
Edit regarding sendfile:
Oof, this is somewhat impossible unless one turns off XOR obfuscation.
libevent had native support for sendfile via evbuffer_add_file and I did a quick & dirty implementation for /rest/block/0000000000000000000515e202c8ae73c8155fc472422d7593af87aa74f2cf3d.bin (fetching the taproot wizards 3.96MB block). Measured a 27% speedup (avg. 89.3ms vs 122.6ms for 1000 runs each).

hodlinator · 2025-08-26T12:47:56Z

src/index/locationsindex.cpp

+// file COPYING or http://www.opensource.org/licenses/mit-license.php.
+
+#include <blockencodings.h>
+#include <clientversion.h>


nit: Unused: clientversion.h

4441827...5e1c80a

hodlinator · 2025-08-26T12:54:30Z

src/index/locationsindex.cpp

+    const uint32_t nTx = block.data->vtx.size();
+    uint32_t nTxOffset = HEADER_SIZE + GetSizeOfCompactSize(nTx);


nit: snake_case and narrower types:

Suggested change

const uint32_t nTx = block.data->vtx.size();

uint32_t nTxOffset = HEADER_SIZE + GetSizeOfCompactSize(nTx);

Assume(block.data->vtx.size() <= std::numeric_limits<uint32_t>::max());

const uint32_t tx_count{static_cast<uint32_t>(block.data->vtx.size())};

uint32_t tx_offset{HEADER_SIZE + GetSizeOfCompactSize(tx_count)};

4441827...5e1c80a

hodlinator · 2025-08-26T13:05:48Z

src/index/locationsindex.cpp

+bool LocationsIndex::ReadRawTransaction(const uint256& block_hash, size_t i, std::vector<std::byte>& out) const
+{
+    uint32_t row = i / TXS_PER_ROW;      // used to find the correct DB row
+    const auto column = i % TXS_PER_ROW; // index within a single DB row


nit: Could make row const as well and nail down the types:

Suggested change

bool LocationsIndex::ReadRawTransaction(const uint256& block_hash, size_t i, std::vector<std::byte>& out) const

{

uint32_t row = i / TXS_PER_ROW; // used to find the correct DB row

const auto column = i % TXS_PER_ROW; // index within a single DB row

bool LocationsIndex::ReadRawTransaction(const uint256& block_hash, uint32_t i, std::vector<std::byte>& out) const

{

const uint32_t row{i / TXS_PER_ROW}; // used to find the correct DB row

const uint32_t column{i % TXS_PER_ROW}; // index within a single DB row

4441827...5e1c80a

hodlinator · 2025-08-26T13:11:35Z

src/index/locationsindex.cpp

+        return false;
+    }
+
+    out.resize(tx_size); // Zeroing of memory is intentional here


Don't see a good way around zeroing the memory, but comment makes it sound like a good thing?
I guess it's slightly better than returning a vector containing garbage upon failure, but even better would be to clear it upon failure?

nit: Could change:

- bool ReadRawTransaction(const uint256& block_hash, size_t i, std::vector<std::byte>& out) const; + std::vector<std::byte> ReadRawTransaction(const uint256& block_hash, uint32_t i) const;

And only return non-empty vector on success instead of having a separate bool? Somewhat unorthodox compared to other Index-implementations but more modern in that it exploits the fact that vectors are movable.

4441827...5e1c80a

hodlinator · 2025-08-26T17:34:45Z

src/rest.cpp

+        } else {
+            strHex = HexStr(tx_bytes);
+        }
+        strHex.append("\n");


nit: Might as well use the fact that it's only 1 char:

Suggested change

strHex.append("\n");

strHex.push_back('\n');

4441827...5e1c80a

hodlinator · 2025-08-26T19:17:24Z

src/rest.cpp

+    ChainstateManager* maybe_chainman = GetChainman(context, req);
+    if (!maybe_chainman) return false;
+    ChainstateManager& chainman = *maybe_chainman;


nit: chainman is only used in the !locations_index scenario, so could be skipped unless you want to always require one.

4441827...5e1c80a

hodlinator · 2025-08-26T20:13:25Z

src/index/locationsindex.cpp

+{
+    fs::path path = gArgs.GetDataDirNet() / "indexes" / "locations";
+    fs::create_directories(path);
+
+    m_db = std::make_unique<BaseIndex::DB>(path / "db", n_cache_size, f_memory, f_wipe);


Not sure why explicit create_directories() is needed, this seems to work:

Suggested change

{

fs::path path = gArgs.GetDataDirNet() / "indexes" / "locations";

fs::create_directories(path);

m_db = std::make_unique<BaseIndex::DB>(path / "db", n_cache_size, f_memory, f_wipe);

, m_db{std::make_unique<BaseIndex::DB>(gArgs.GetDataDirNet() / "indexes" / "locations" / "db",

n_cache_size, f_memory, f_wipe)}

{

4441827...5e1c80a

hodlinator · 2025-08-26T20:27:30Z

test/functional/interface_rest.py

@@ -53,7 +53,7 @@ def filter_output_indices_by_value(vouts, value):
 class RESTTest (BitcoinTestFramework):


(Inline comment in random location to keep main thread cleaner).

Played around with adding coverage in feature_init.py. However, I encountered some weird interaction between txindex and locationsindex. Not sure how incidental it is (going above a certain number of files) or whether some kind of interference is happening. See comment "If we add this, then we fail later while perturbing txindex":

diff --git a/test/functional/feature_init.py b/test/functional/feature_init.py index 15b3e8595f..47e29655ce 100755 --- a/test/functional/feature_init.py +++ b/test/functional/feature_init.py @@ -49,7 +49,7 @@ class InitTest(BitcoinTestFramework): def start_expecting_error(err_fragment): node.assert_start_raises_init_error( - extra_args=['-txindex=1', '-blockfilterindex=1', '-coinstatsindex=1', '-checkblocks=200', '-checklevel=4'], + extra_args=['-txindex=1', '-blockfilterindex=1', '-coinstatsindex=1', '-locationsindex=1', '-checkblocks=200', '-checklevel=4'], expected_msg=err_fragment, match=ErrorMatch.PARTIAL_REGEX, ) @@ -77,6 +77,7 @@ class InitTest(BitcoinTestFramework): b'txindex thread start', b'block filter index thread start', b'coinstatsindex thread start', + b'locationsindex thread start', b'msghand thread start', b'net thread start', b'addcon thread start', @@ -84,7 +85,7 @@ class InitTest(BitcoinTestFramework): if self.is_wallet_compiled(): lines_to_terminate_after.append(b'Verifying wallet') - args = ['-txindex=1', '-blockfilterindex=1', '-coinstatsindex=1'] + args = ['-txindex=1', '-blockfilterindex=1', '-coinstatsindex=1', '-locationsindex=1'] for terminate_line in lines_to_terminate_after: self.log.info(f"Starting node and will terminate after line {terminate_line}") with node.busy_wait_for_debug_log([terminate_line]): @@ -101,12 +102,13 @@ class InitTest(BitcoinTestFramework): self.stop_node(0) self.log.info("Test startup errors after removing certain essential files") - files_to_delete = { 'blocks/index/*.ldb': 'Error opening block database.', 'chainstate/*.ldb': 'Error opening coins database.', 'blocks/blk*.dat': 'Error loading block database.', 'indexes/txindex/MANIFEST*': 'LevelDB error: Corruption: CURRENT points to a non-existent file', + # If we add this, then we fail later while perturbing txindex!?: + #'indexes/locations/db/MANIFEST*': 'LevelDB error: Corruption: CURRENT points to a non-existent file', # Removing these files does not result in a startup error: # 'indexes/blockfilter/basic/*.dat', 'indexes/blockfilter/basic/db/*.*', 'indexes/coinstats/db/*.*', # 'indexes/txindex/*.log', 'indexes/txindex/CURRENT', 'indexes/txindex/LOCK' @@ -120,6 +122,8 @@ class InitTest(BitcoinTestFramework): 'indexes/coinstats/db/*.*': 'LevelDB error: Corruption', 'indexes/txindex/*.log': 'LevelDB error: Corruption', 'indexes/txindex/CURRENT': 'LevelDB error: Corruption', + 'indexes/locations/db/*.log': 'LevelDB error: Corruption', + 'indexes/locations/db/CURRENT': 'LevelDB error: Corruption', # Perturbing these files does not result in a startup error: # 'indexes/blockfilter/basic/*.dat', 'indexes/txindex/MANIFEST*', 'indexes/txindex/LOCK' }

Good catch, will investigate.

94389c2 seems to be passing (after a rebase over master) 🤔

@hodlinator would it be OK to squash 94389c2 into bdd90ab?

hodlinator · 2025-08-26T20:42:27Z

src/node/blockstorage.h

@@ -417,6 +417,8 @@ class BlockManager

    bool ReadBlockUndo(CBlockUndo& blockundo, const CBlockIndex& index) const;

+    bool ReadTxFromBlock(CTransactionRef& tx, const FlatFilePos& block_pos, size_t tx_index) const;


nit: Could use more modern style instead of returning duplicate bool state (also nail down index type):

Suggested change

bool ReadTxFromBlock(CTransactionRef& tx, const FlatFilePos& block_pos, size_t tx_index) const;

CTransactionRef ReadTxFromBlock(const FlatFilePos& block_pos, uint32_t tx_index) const;

4441827...5e1c80a

hodlinator · 2025-08-27T07:03:45Z

src/rest.cpp

+    std::string param;
+    const RESTResponseFormat rf = ParseDataFormat(param, uri_part);
+
+    // request is sent over URI scheme /rest/txfromblock/blockhash-index


Have you considered the possibility of supporting batches of block-hash+tx-index in one GET?

rest_getutxos() does something similar:

bitcoin/src/rest.cpp

Line 1007 in 4441827

//inputs is sent over URI scheme (/rest/getutxos/checkmempool/txid1-n/txid2-n/...)

No sure if it's best practices in the REST world though. Another option could be using a JSON blob in a HTTP header, but that may be even more frowned upon.

cc @stickies-v who might have a feel for this kind of thing (came across his f959fc0).

romanz · 2025-08-30T18:02:30Z

Many thanks for the review and the suggested fixes!
Squashed most the fixes into the above commits and force-pushed: 4441827...5e1c80a

DrahtBot · 2025-08-30T19:08:09Z

🚧 At least one of the CI tasks failed.
_{Task TSan, depends, no gui: https://github.com/bitcoin/bitcoin/runs/49248056402}
_{LLM reason (✨ experimental): ThreadSanitizer detected a data race (width in ios:501:18) causing the mptest to fail.}

Hints

Try to run the tests locally, according to the documentation. However, a CI failure may still
happen due to a number of reasons, for example:

Possibly due to a silent merge conflict (the changes in this pull request being
incompatible with the current code in the target branch). If so, make sure to rebase on the latest
commit of the target branch.
A sanitizer issue, which can only be found by compiling with the sanitizer and running the
affected test.
An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

romanz · 2025-08-31T05:36:12Z

Rebasing to resolve a conflict with master.

DrahtBot added UTXO Db and Indexes CI failed labels May 17, 2025

romanz force-pushed the locations-index branch from 40ee9a9 to 1f62974 Compare May 17, 2025 07:45

romanz force-pushed the locations-index branch from 1f62974 to c074ad2 Compare May 17, 2025 11:55

DrahtBot removed the CI failed label May 17, 2025

This was referenced Jun 6, 2025

index: initial sync speedup, parallelize process #26966

Open

docs: adds correct updated documentation links #32699

Open

DrahtBot added the CI failed label Jun 13, 2025

romanz force-pushed the locations-index branch from c074ad2 to d962c9a Compare June 13, 2025 18:35

DrahtBot removed the CI failed label Jun 13, 2025

romanz marked this pull request as draft June 14, 2025 08:58

romanz marked this pull request as ready for review June 14, 2025 12:29

DrahtBot mentioned this pull request Jun 16, 2025

net: Fix Discover() not running when using -bind=0.0.0.0:port #32757

Open

TheCharlatan mentioned this pull request Jun 19, 2025

Add a "tx output spender" index #24539

Open

maflcko reviewed Jun 20, 2025

View reviewed changes

src/rest.cpp Outdated Show resolved Hide resolved

romanz force-pushed the locations-index branch from d962c9a to 8d446fc Compare June 21, 2025 09:57

romanz marked this pull request as draft June 21, 2025 14:29

DrahtBot added the CI failed label Jun 21, 2025

DrahtBot mentioned this pull request Jul 14, 2025

index: Fix coinstats overflow #30469

Open

romanz force-pushed the locations-index branch 2 times, most recently from 024a91f to 1b928f5 Compare July 19, 2025 17:15

This was referenced Jul 29, 2025

refactor: Add ALLOW_LIST flags and enforce usage in CheckArgFlags #17580

Draft

kernel, refactor: return error status on all fatal errors #29700

Draft

common: Disallow calling IsArgSet() on ALLOW_LIST options #17783

Draft

hodlinator reviewed Aug 2, 2025

View reviewed changes

romanz force-pushed the locations-index branch from 1b928f5 to b2a22ce Compare August 3, 2025 06:42

romanz requested review from hodlinator and maflcko August 3, 2025 12:41

romanz force-pushed the locations-index branch from b2a22ce to 4441827 Compare August 4, 2025 11:35

This was referenced Aug 5, 2025

refactor: Remove settings merge reverse precedence code #17581

Draft

util: Forbid ambiguous multiple assignments in config file #17493

Draft

hodlinator reviewed Aug 27, 2025

View reviewed changes

macgyver13 mentioned this pull request Aug 28, 2025

Bitcoin indexing library (bindex-rs) silent-payments/roadmap#21

Open

romanz force-pushed the locations-index branch from 4441827 to 5e1c80a Compare August 30, 2025 18:01

DrahtBot added the CI failed label Aug 30, 2025

romanz and others added 3 commits August 31, 2025 08:37

rest: allow fetching specific transaction from a block

1213f2d

index: store per-block transaction locations for efficient lookups

bdd90ab

test: add locationindex to feature_init.py functional tests

94389c2

romanz force-pushed the locations-index branch from 37876b0 to 94389c2 Compare August 31, 2025 06:00

DrahtBot removed the CI failed label Aug 31, 2025

		const uint32_t nTx = block.data->vtx.size();
		uint32_t nTxOffset = HEADER_SIZE + GetSizeOfCompactSize(nTx);

-    const uint32_t nTx = block.data->vtx.size();
-    uint32_t nTxOffset = HEADER_SIZE + GetSizeOfCompactSize(nTx);
+    Assume(block.data->vtx.size() <= std::numeric_limits<uint32_t>::max());
+    const uint32_t tx_count{static_cast<uint32_t>(block.data->vtx.size())};
+    uint32_t tx_offset{HEADER_SIZE + GetSizeOfCompactSize(tx_count)};

		@@ -53,7 +53,7 @@ def filter_output_indices_by_value(vouts, value):
		class RESTTest (BitcoinTestFramework):

		@@ -417,6 +417,8 @@ class BlockManager

		bool ReadBlockUndo(CBlockUndo& blockundo, const CBlockIndex& index) const;

		bool ReadTxFromBlock(CTransactionRef& tx, const FlatFilePos& block_pos, size_t tx_index) const;

	bool ReadTxFromBlock(CTransactionRef& tx, const FlatFilePos& block_pos, size_t tx_index) const;
	CTransactionRef ReadTxFromBlock(const FlatFilePos& block_pos, uint32_t tx_index) const;

index: store per-block transaction locations for efficient lookups #32541

Are you sure you want to change the base?

index: store per-block transaction locations for efficient lookups #32541

Conversation

romanz commented May 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Footnotes

Uh oh!

DrahtBot commented May 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Coverage & Benchmarks

Reviews

Conflicts

Uh oh!

DrahtBot commented May 17, 2025

Uh oh!

TheCharlatan commented May 17, 2025

Uh oh!

romanz commented May 17, 2025

Uh oh!

romanz commented May 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

luke-jr commented May 20, 2025

Uh oh!

romanz commented May 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

fetching using the new index

fetching using txindex

fetching without txindex

Conclusions

Uh oh!

DrahtBot commented Jun 13, 2025

Uh oh!

romanz commented Jun 13, 2025

Uh oh!

TheCharlatan commented Jun 15, 2025

Uh oh!

romanz commented Jun 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

maflcko left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

romanz commented Jun 21, 2025

Uh oh!

romanz commented Jul 14, 2025

Uh oh!

romanz commented Jul 19, 2025

Uh oh!

hodlinator left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

romanz commented Aug 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

romanz commented Aug 4, 2025

Uh oh!

romanz commented Aug 25, 2025

Uh oh!

hodlinator left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

romanz commented May 17, 2025 •

edited

Loading

DrahtBot commented May 17, 2025 •

edited

Loading

romanz commented May 17, 2025 •

edited

Loading

romanz commented May 21, 2025 •

edited

Loading

fetching using `txindex`

fetching without `txindex`

romanz commented Jun 15, 2025 •

edited

Loading

romanz commented Aug 3, 2025 •

edited

Loading

hodlinator left a comment •

edited

Loading