-
Notifications
You must be signed in to change notification settings - Fork 37.7k
rpc, cli: add multiwallet balances rpc and use it in -getinfo #18453
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
rpc, cli: add multiwallet balances rpc and use it in -getinfo #18453
Conversation
Concept ACK |
Adds one more call to utACK f73ace0 |
What would make me really happy, when you're making changes here anyway, is implementing point 2 from #17314: show balance per wallet. This is much more useful in the multi-wallet case. |
Concept ACK. Agree in making @laanwj happy. The change requires a |
Good idea, will do. |
|
Something like this? (getbalances version) $ bitcoin-cli -getinfo -regtest
{
"version": 199900,
"blocks": 15599,
"headers": 15599,
"verificationprogress": 1,
"timeoffset": 0,
"connections": 0,
"proxy": "",
"difficulty": 4.656542373906925e-10,
"chain": "regtest",
"balances": [
{
"": {
"mine": {
"trusted": 0.00001000,
"untrusted_pending": 0.00000000,
"immature": 0.00000000
}
}
},
{
"encrypted": {
"mine": {
"trusted": 0.00003500,
"untrusted_pending": 0.00000000,
"immature": 0.00000000
}
}
},
],
"relayfee": 0.00001000
} or like? (mine.trusted balances only) $ bitcoin-cli -getinfo -regtest
{
"version": 199900,
"blocks": 15599,
"headers": 15599,
"verificationprogress": 1,
"timeoffset": 0,
"connections": 0,
"proxy": "",
"difficulty": 4.656542373906925e-10,
"chain": "regtest",
"balances": [
{
"name": "",
"amount": 0.00001000
},
{
"name": "encrypted",
"amount": 0.00003500
}
],
"relayfee": 0.00001000
} |
It's best if {
"version": 199900,
"blocks": 15599,
"headers": 15599,
"verificationprogress": 1,
"timeoffset": 0,
"connections": 0,
"proxy": "",
"difficulty": 4.656542373906925e-10,
"chain": "regtest",
"balances": {
"": 0.00001000,
"encrypted": 0.00003500,
},
"relayfee": 0.00001000
} |
Perfect -- thanks 👍 |
f73ace0
to
11676f9
Compare
11676f9
to
0da1a8f
Compare
Done. If this approach is ok, I'll add tests for the new call. |
Why not iterate on the cli? |
I could be wrong, but this seemed cleaner, and I think we'll ultimately want either an rpc like this or allow passing a multi bool to getbalance or getbalances to do the same. |
0da1a8f
to
6a2f1db
Compare
Not sure about the approach TBH. I'd rather see the cli use existing calls. The new RPC |
6a2f1db
to
a5ea571
Compare
This works, but I understand. Interested in feedback. Note that for now |
I'm not opposed to having "across-all-wallets" RPCs, but perhaps it's worth making them visually distinct? How about "getallwalletbalances" or "allwalletsgetbalance"? |
Yes. I was debating internally about "listbalances"... |
ACK a5ea571 Reviewed code and did some light testing locally. I think multi wallet RPCs will be useful for a lot of users. For naming I don't have strong feelings but since "multi wallet" is already the term most frequently used for this scenario verbally, I think I would prefer if that gets used if people think the distinction is necessary. So I would prefer something like "(get)multiwalletbalances" to something like listbalances or something with allwallets. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ACK. Code change looks great.
Local testing:
- Unit tests passed
- Functional tests passed (
interface_bitcoin_cli.py
andwallet_multiwallet.py
) - Manual smoke test passed (created multiple wallets, generated coins, made sure
bitcoin-cli -regtest -getinfo
andbitcoin-cli -regtest getwalletbalances
behaved as expected)
Thanks for hosting the next PR Review Club! During the session, I'd also be curious to learn when it's appropriate to add new RPCs to the bitcoin-cli
documentation.
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. ConflictsReviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Approach ACK.
The modified tests test/functional/interface_bitcoin_cli.py
and test/functional/wallet_multiwallet.py
pass on my computer and the output of bitcoin-cli -getinfo -regtest
looks as intended.
auto locked_chain = wallet.chain().lock(); | ||
LOCK(wallet.cs_wallet); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These locks are acquired inside GetBalance()
, so they can be removed from here?
A wallet's chain will never change and, I assume, all wallets' chains are the same. That is - we are operating on one chain here, for all wallets. Can we / should we lock that chain before the for-loop in order to get a consistent view (i.e. avoid getting one wallet's balance as of height H and another wallet's balance as of height H+1)?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A wallet's chain will never change and, I assume, all wallets' chains are the same. That is - we are operating on one chain here, for all wallets. Can we / should we lock that chain before the for-loop in order to get a consistent view (i.e. avoid getting one wallet's balance as of height H and another wallet's balance as of height H+1)?
We generally don't want wallet code to lock up the node, and after #16426 we are removing the ability of wallet code to be able to do this. It synchronization between different wallets is a concern, a solution could be to return tip hashes alongside balances. Or to restart the wallet loop if the tip changes midway.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see. So only the first question above remains (remove the locks acquisition).
{ | ||
UniValue obj{UniValue::VOBJ}; | ||
for (const std::shared_ptr<CWallet>& rpc_wallet : GetWallets()) { | ||
if (!EnsureWalletIsAvailable(rpc_wallet.get(), request.fHelp)) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we be doing any of this if request.fHelp
is true
? It happens when bitcoin-cli help
is executed even though none of its output contains getwalletbalances
or any balances.
@@ -4264,6 +4286,7 @@ static const CRPCCommand commands[] = | |||
{ "wallet", "encryptwallet", &encryptwallet, {"passphrase"} }, | |||
{ "wallet", "getaddressesbylabel", &getaddressesbylabel, {"label"} }, | |||
{ "wallet", "getaddressinfo", &getaddressinfo, {"address"} }, | |||
{ "wallet", "getwalletbalances", &getwalletbalances, {} }, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you plan to document this before merge? If "no", then are there other undocumented commands?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If the consensus is to add a public RPC, yes, it should have documentation.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, makes sense. IMO the current approach where the iterate-over-wallets logic is inside bitcoind and exposed via RPC is better than having that logic inside bitcoin-cli because in the latter case it cannot be reused by RPC users - should they need that functionality they would have to replicate the logic inside their applications.
Concept ACK. I strongly agree with @sipa. Across-all-wallets calls should have a naming convention. IMO Another approach would be to add a special RPC endpoint like
I'm not convince that this is the better approach (has probably some performance and flexibility downsides), just wanted to highlight the importantness of defining the first across-all-wallets RPC. |
f9391c3
to
a96508e
Compare
98e37e0
to
d8177fd
Compare
d8177fd
to
d720f7f
Compare
ACK this. Right now I have bash scripts for doing multi-wallet things like balances and coins but it's pretty horrible and slow since I can't batch requests. It would be so much nicer if there was a special wallet endpoint you could hit which signals that you want the result from all wallets, or at least a way to batch requests for different wallets. |
d720f7f
to
c414b42
Compare
Rebased on master for the CI fix. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Concept NACK specifically to enabling access to the wallet list (or non-selected wallets) for existing interfaces.
Up until now, RPCs would only ever access the wallet the request was sent to...
🐙 This pull request conflicts with the target branch and needs rebase. |
Thanks to everyone for reviewing. Summary: To take the feedback into account, a client-side, smaller-scope version of this feature for |
5edad5c test: add -getinfo multiwallet functional tests (Jon Atack) 903b6c1 rpc: drop unused JSONRPCProcessBatchReply size arg, refactor (Jon Atack) afce85e cli: use GetWalletBalances() functionality for -getinfo (Jon Atack) 9f01849 cli: create GetWalletBalances() to fetch multiwallet balances (Jon Atack) 7430775 cli: lift -rpcwallet logic up to CommandLineRPC() (Jon Atack) 29f2cbd cli: extract connection exception handler, -rpcwait logic (Jon Atack) Pull request description: This PR is a client-side version of #18453, per review feedback there and [review club discussions](https://bitcoincore.reviews/18453#meeting-log). It updates `bitcoin-cli -getinfo` on the client side to display wallet name and balance for the loaded wallets when more than one is loaded (e.g. you are in "multiwallet mode") and `-rpcwallet=` is not passed; otherwise, behavior is unchanged. before ```json $ bitcoin-cli -getinfo -regtest { "version": 199900, "blocks": 15599, "headers": 15599, "verificationprogress": 1, "timeoffset": 0, "connections": 0, "proxy": "", "difficulty": 4.656542373906925e-10, "chain": "regtest", "balance": 0.00001000, "relayfee": 0.00001000 } ``` after ```json $ bitcoin-cli -getinfo -regtest { "version": 199900, "blocks": 15599, "headers": 15599, "verificationprogress": 1, "timeoffset": 0, "connections": 0, "proxy": "", "difficulty": 4.656542373906925e-10, "chain": "regtest", "balances": { "": 0.00001000, "Encrypted": 0.00003500, "day-to-day": 0.00000120, "side project": 0.00000094 } } ``` ----- `Review club` discussion about this PR is here: https://bitcoincore.reviews/18453 This PR can be manually tested by building, creating/loading/unloading several wallets with `bitcoin-cli createwallet/loadwallet/unloadwallet` and running `bitcoin-cli -getinfo` and `bitcoin-cli -rpcwallet=<wallet-name> -getinfo`. `wallet_multiwallet.py --usecli` provides regression test coverage on this change, along with `interface_bitcoin_cli.py` where this PR adds test coverage. Credit to Wladimir J. van der Laan for the idea in #17314 and #18453 (comment). ACKs for top commit: promag: Tested ACK 5edad5c. jnewbery: utACK 5edad5c meshcollider: Code review ACK 5edad5c Tree-SHA512: 4ca36c5f6c49936b40afb605c44459c1d5b80b5bd84df634007ca276b3f6c102a0cb382f9d528370363ee32c94b0d7ffa15184578eaf8de74179e566c5c5cee5
5edad5c test: add -getinfo multiwallet functional tests (Jon Atack) 903b6c1 rpc: drop unused JSONRPCProcessBatchReply size arg, refactor (Jon Atack) afce85e cli: use GetWalletBalances() functionality for -getinfo (Jon Atack) 9f01849 cli: create GetWalletBalances() to fetch multiwallet balances (Jon Atack) 7430775 cli: lift -rpcwallet logic up to CommandLineRPC() (Jon Atack) 29f2cbd cli: extract connection exception handler, -rpcwait logic (Jon Atack) Pull request description: This PR is a client-side version of bitcoin#18453, per review feedback there and [review club discussions](https://bitcoincore.reviews/18453#meeting-log). It updates `bitcoin-cli -getinfo` on the client side to display wallet name and balance for the loaded wallets when more than one is loaded (e.g. you are in "multiwallet mode") and `-rpcwallet=` is not passed; otherwise, behavior is unchanged. before ```json $ bitcoin-cli -getinfo -regtest { "version": 199900, "blocks": 15599, "headers": 15599, "verificationprogress": 1, "timeoffset": 0, "connections": 0, "proxy": "", "difficulty": 4.656542373906925e-10, "chain": "regtest", "balance": 0.00001000, "relayfee": 0.00001000 } ``` after ```json $ bitcoin-cli -getinfo -regtest { "version": 199900, "blocks": 15599, "headers": 15599, "verificationprogress": 1, "timeoffset": 0, "connections": 0, "proxy": "", "difficulty": 4.656542373906925e-10, "chain": "regtest", "balances": { "": 0.00001000, "Encrypted": 0.00003500, "day-to-day": 0.00000120, "side project": 0.00000094 } } ``` ----- `Review club` discussion about this PR is here: https://bitcoincore.reviews/18453 This PR can be manually tested by building, creating/loading/unloading several wallets with `bitcoin-cli createwallet/loadwallet/unloadwallet` and running `bitcoin-cli -getinfo` and `bitcoin-cli -rpcwallet=<wallet-name> -getinfo`. `wallet_multiwallet.py --usecli` provides regression test coverage on this change, along with `interface_bitcoin_cli.py` where this PR adds test coverage. Credit to Wladimir J. van der Laan for the idea in bitcoin#17314 and bitcoin#18453 (comment). ACKs for top commit: promag: Tested ACK 5edad5c. jnewbery: utACK 5edad5c meshcollider: Code review ACK 5edad5c Tree-SHA512: 4ca36c5f6c49936b40afb605c44459c1d5b80b5bd84df634007ca276b3f6c102a0cb382f9d528370363ee32c94b0d7ffa15184578eaf8de74179e566c5c5cee5
This PR implements point 2 from #17314 (show balance per wallet):
bitcoin-cli -getinfo
to use the RPCbefore
after
Review club
discussion about this PR is here: https://bitcoincore.reviews/18453.This approach is simpler, adds no additional rpc calls in -getinfo and runs faster than doing it on the client side inside bitcoin-cli.cpp, but it also adds the first multiwallet RPC and would need to be well-considered in terms of the API.