Skip to content

Conversation

kouloumos
Copy link
Contributor

@kouloumos kouloumos commented Sep 1, 2022

listreceivedbyaddress & listreceivedbylabel RPCs list balances by receiving address and label respectively. That works by creating a tally (mapTally[address]) from all the transaction outputs in the wallet and then cross-reference that with the address book in order to calculate which addresses received funds.

If an address is in the address book but not in the tally, it's considered empty (never received funds) and is not returned unless include_empty=true. That effectively means that when include_empty=true the entire address book is included in the result.

This creates 2 issues:

  1. As reported in listrecievedbyaddress with include_empty not filtering out "send" side of address book #16159, the address book also includes addresses with a "send" purpose which should not be part of the response.
  2. As reported in listreceivedbyaddress incorrectly shows watchonly addresses as empty #10468, if include_watchonly=false the watchonly addresses are not included in mapTally leading to them being considered empty during the subsequent cross-reference with the address book thus becoming part of the result when include_empty=true.

This PR fixes #16159 by filtering-out "send" addresses, see #16159 (comment) for more details on the issue. I haven't thought of a nice solution to fix (2) yet.

Edit: Split wallet_listreceivedby.py tests for code maintainability as tests for more than one RPC have been added since the creation of the file. Added test coverage for the introduced change.

@kouloumos kouloumos force-pushed the listreceivedby-added-filter branch from 44aa363 to ce7fc73 Compare September 1, 2022 15:23
@DrahtBot DrahtBot added the Wallet label Sep 1, 2022
@kouloumos kouloumos force-pushed the listreceivedby-added-filter branch from 0c0544d to 62880a9 Compare September 2, 2022 14:12
Copy link
Contributor

@aureleoules aureleoules left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 62880a9.
I verified that the commit ce7fc73 fixes the tests in 62880a9.
527154a looks good to me, only a refactor.

@DrahtBot
Copy link
Contributor

DrahtBot commented Sep 23, 2022

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Reviews

See the guideline for information on the review process.

Type Reviewers
ACK aureleoules, achow101, ryanofsky

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

No conflicts as of last run.

@@ -103,6 +104,27 @@ def test_listreceivedbyaddress(self):
res = self.nodes[1].listreceivedbyaddress(0, True, True, other_addr)
assert_equal(len(res), 0)

def test_listreceivedby(self):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In 62880a9 "test: add coverage for listreceivedby* no "send" purpose addrs return"

nit: s/test_listreceivedby/test_listreceivedby_excludes_send

Copy link
Member

@furszy furszy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

listreceivedbyaddress & listreceivedbylabel RPCs list balances by receiving address and label respectively. That works by creating a tally (mapTally[address]) from all the transaction outputs in the wallet and then cross-reference that with the address book in order to calculate which addresses received funds.

Why do we need to cross-reference with the address book if we already traversed the whole wallet's transactions outputs and filtered-out all the ones that are not belonging to the wallet?

Seems that we could directly use the mapTally information without looping around the address book. Only would need to implement a CWallet::GetAddrLabel(const CTxDestination& dest) to obtain the label whenever is needed.

@davidgumberg
Copy link
Contributor

Is it possible for an address that has been received from to get marked as a "send" address and subsequently not get returned by listreceived*?

@willcl-ark
Copy link
Member

@kouloumos would #26813 also be closed by this?

@furszy
Copy link
Member

furszy commented Jan 23, 2023

@willcl-ark no, possibly by #25979 (or any other attempt to improve/standardize the change detection)

@achow101
Copy link
Member

achow101 commented Mar 9, 2023

ACK 62880a9

@achow101
Copy link
Member

achow101 commented May 3, 2023

Why do we need to cross-reference with the address book if we already traversed the whole wallet's transactions outputs and filtered-out all the ones that are not belonging to the wallet?

To lookup the label.


There is a silent merge conflict with master.

Copy link
Contributor

@ryanofsky ryanofsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review ACK 62880a9, but this needs to be rebased due to a silent conflict with #27217. This seems like a helpful fix and the test cleanup is also nice.

Another suggestion if you feel like it would be to add the new test before making the code change and have it verify the previous behavior. Then fix the bug and update the test in the same commit, so it is easier to see how behavior changes in the diff.

@@ -137,6 +137,8 @@ static UniValue ListReceived(const CWallet& wallet, const UniValue& params, cons
const auto& func = [&](const CTxDestination& address, const std::string& label, const std::string& purpose, bool is_change) {
if (is_change) return; // no change addresses

if (purpose == "send") return; // no send addresses
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In commit "wallet: Filter-out "send" addresses from listreceivedby*" (ce7fc73)

It like it would be better to call IsMine here instead of relying on the purpose field. The PR description says this fix only covers the include_empty=true, include_watchonly=true case, not the include_watchonly=false case, so maybe you could fix both cases by using the isminetype result returned by IsMine instead of purpose == "send".

Also, I'm not 100% sure about this, and am curious about other opinions, but in general the purpose field seems like it is set pretty haphazardly in code, so things that rely on it could easily break, and probably better not to use it when alternatives are available.

@achow101
Copy link
Member

Are you still working on this?

@achow101
Copy link
Member

Closing as up for grabs due to lack of activity.

@achow101 achow101 closed this Sep 20, 2023
@bitcoin bitcoin locked and limited conversation to collaborators Sep 19, 2024
@fanquake
Copy link
Member

fanquake commented Oct 2, 2024

Some of this was picked up in #30972.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

listrecievedbyaddress with include_empty not filtering out "send" side of address book
10 participants