-
Notifications
You must be signed in to change notification settings - Fork 37.8k
Add index to wallet UTXO #7823
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add index to wallet UTXO #7823
Conversation
Yes, this makes sense. How does it affect memory usage? |
Concept ACK, it's silly that we didn't have this. Memory usage: I expect it to consume around 80 bytes per unspent output on 64-bit platforms. |
ACK Rename first commit to something like |
The overall memory usage will depend on the wallet size, more particularly on the UTXO count. Given the info being stored per UTXO (an outpoint and a pair with |
@promag that makes sense. This is not a cache, but more like an index.. |
Tested ACK. In my case the improvements of listunspent are from 12s. to 0.1s . |
7e39fb7
to
f61677b
Compare
Nice work! |
So this only uses (~80 bytes) memory per actually unspent output that is in the wallet? That's great. I then don't expect the extra memory use for that to be significant. The bulk is wasted on keeping historical transactions anyway. Concept ACK. |
4939d13
to
2892f4e
Compare
Concept ACK. Thanks for working on this. |
Solves #6573. |
lightly tested ACK |
2892f4e
to
6fbc9bf
Compare
This change greatly improves the performance of AvailableCoins and indirectly listunspent and others. Is it reasonable to expect this in next 0.12 update? |
@promag Only bug fixes and consensus changes are backported to old releases. It's well on track for 0.13, though. |
I think this needs at least one more code review ACK before merge (to master, not 0.12) |
@@ -498,9 +502,12 @@ bool CWallet::IsSpent(const uint256& hash, unsigned int n) const | |||
|
|||
void CWallet::AddToSpends(const COutPoint& outpoint, const uint256& wtxid) | |||
{ | |||
// since this output is being marked as a spend output, we remove it from the UTXO index |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Typo: spent output
How does this deal with:
|
d12287e
to
2b9c1a5
Compare
This needs a rebase. |
2b9c1a5
to
55abe3a
Compare
@jpdffonseca Feel like answering my questions above? |
@sipa I'm working to unify |
@jpdffonseca Are you still working on this? We need performance improvements especially for large wallets. Needs rebase. |
@paveljanik lately I haven't had too much time to work on it. After testing more deeply, I realized that this solution does not always handles well some of the cases mentioned above by @sipa, so it should have a different approach. I've been trying another improvements that optimize not the |
Closing due to inactivity. @joaopaulofonseca if you do end up finishing work on your other improvements PRs are welcome. |
Deprecated partially by #9939. Caching accurately these values gets a little tricky because of all the scenarios that could make the cache invalidated (conflicts, double-spends, reorganizations, abandoned transactions, etc, ...). |
This PR implements an index of unspent transaction outputs (UTXOs).
This approach optimizes
CWallet::AvailableCoins
, by doing some of its work each time a wallet transaction changes. For instance, usingCWallet::IsMine
earlier for each transaction output, it manages to keep track only of UTXOs related to wallet addresses.That brings large improvements on RPCs such as
listunspent
andfundrawtransaction
. Depending on the size of your wallet and number of UTXOs, performance gains may be up to 98%.This way, instead of going through all the wallet transactions, the search for UTXOs is made only on this updated list of outputs.
Depends on #7822 to ensure functionality behaviour remains the same.