Skip to content

Conversation

murchandamus
Copy link
Contributor

@murchandamus murchandamus commented Nov 30, 2023

Please refer to the topic on Delving Bitcoin discussing Gutter Guard/Coingrinder simulation results.

Gutter Guard Selector bounds the worst case on coin selection outcomes by ensuring that there is at least one candidate input set that exceeds the minimal necessary input count by at most three.

Does a random selection limited to three more output groups than largest-first selection would select to fund the transaction. If the limit is exceeded during selection, the output group with the lowest effective value is discarded.

Motivations

  • At high feerates, using unnecessary inputs can significantly increase the fees
  • Users are upset when fees are relatively large compared to the amount sent
  • Minimizing the input set is an NP-hard problem
  • Always minimizing the weight of the input set can lead to fragmentation of the wallet’s UTXO pool

Approach

Bitcoin Core uses multiple coin selection algorithms to generate candidate input sets. It then picks the least wasteful input set per the weight metric, but none of the deployed algorithms explicitly look for low-weight input sets. Gutter Guard Selector determines a likely minimum count c of output groups that are necessary to fund the transaction via largest-first selection. It then randomly selects output groups into a lowest-effective-value heap. If the heap exceeds c + 3 output groups, the output group with the lowest-effective value is discarded. It then checks whether the selection can fund the transaction. This approach will always succeed given the effective value available exceeds the selection target.

Trade-offs

Largest-first selection (LF) (and to a lesser degree an approach that minimizes the weight of an input set) might grind a wallet’s UTXO pool to dust if overused. Incoming payments grow the wallet’s UTXO pool, while transactions with a single input and a change output result in the same UTXO count as before. LF will additionally decrease range of amounts in the wallet by reducing the amount of the supreme element.

Branch and Bound looks for a changeless input set. If there are multiple solutions, it prefers the one with the lowest waste, but there might not be a solution, or even the least wasteful one could be composed of many inputs. Knapsack minimizes the distance between target amount and selected amount which is independent from the input set’s weight. Single Random Draw just does a single random drawing from the UTXO pool—if there are a ton of UTXOs in the wallet, this will likely result in a large input set. While we pick the least wasteful among the presented input set candidates, there is no guarantee that any of the three deployed algorithms will produce a thrifty input set.

In contrast, Gutter Guard Selector is expected to reduce the UTXO pool of a fragmented wallet by at least three UTXOs. When many UTXOs exist that are larger than the target, a solution with fewer inputs and less weight may be found. When transactions with low feerates are being built, the waste metric prefers heavier input sets which may be proposed by other coin selection algorithms anyway. At high feerates, Gutter Guard Selector ensures that there is an input set candidate that isn’t absurdly wasteful and therefore delimits the worst case.

@DrahtBot
Copy link
Contributor

DrahtBot commented Nov 30, 2023

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage

For detailed information about the code coverage, see the test coverage report.

Reviews

See the guideline for information on the review process.

Type Reviewers
ACK kashifs
Concept ACK achow101

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #28985 (Avoid changeless input sets when SFFO is active by murchandamus)
  • #27877 (wallet: Add CoinGrinder coin selection algorithm by murchandamus)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@murchandamus murchandamus force-pushed the 2023-11-gutter-guard-selector branch from 884bfb2 to d0f8d9e Compare November 30, 2023 21:29
@murchandamus murchandamus marked this pull request as draft November 30, 2023 21:30
@murchandamus murchandamus force-pushed the 2023-11-gutter-guard-selector branch from d0f8d9e to 1e19f92 Compare November 30, 2023 21:38
Copy link
Member

@achow101 achow101 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concept ACK

Does a random selection limited to three more output groups than
largest-first selection would select to fund the transaction. If the
limit is exceeded during selection, the output group with the lowest
effective value is discarded.
@murchandamus murchandamus force-pushed the 2023-11-gutter-guard-selector branch from 1e19f92 to 4a94fc8 Compare December 22, 2023 18:53
Copy link
Contributor Author

@murchandamus murchandamus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed comments by @achow101

@kashifs
Copy link
Contributor

kashifs commented Dec 24, 2023

tested ACK 4a94fc8

I read through the conversation, understood the problem and use case for this solution, compiled from source, wrote tests similar to those included

@DrahtBot DrahtBot requested a review from achow101 December 24, 2023 12:09
@achow101
Copy link
Member

Is there a benefit to using SRD here over something oldest first? We are already doing SRD so it might be better to use a different underlying strategy. Having a selection based on oldest first would also be useful for consuming negative EV UTXOs at very low feerates, e.g. less than 3 s/vb.

It also seems like having the limit be inversely scaled by feerate would be better than the fixed limit. That way, as the feerates increase, we get less consolidatory instead of current behavior where it switches at a particular threshold.

@murchandamus
Copy link
Contributor Author

murchandamus commented Jan 10, 2024

Yeah, oldest first might be better for soaking up tiny UTXOs at low feerates that have been piling up during bouts of high feerates.

@achow101: What do you think about one of the two like the following progression

<2s/vB <3s/vB <4s/vB <5s/vB <6s/vB <7s/vB <8s/vB <9s/vB <10s/vB >=10s/vB
[A:] UTXO Limit +21 +18 +15 +12 +10 +8 +6 +5 +4 +3
[B:] UTXO Limit +48 +39 +31 +24 +18 +13 +9 +6 +4 +3

allow negative effective value inputs

Additionally the unneeded UTXOs should be bounded dependent on the count of the wallet UTXO pool such as the minimum of this progress and e.g. 10% of the wallet’s UTXO count.

I guess I’ll have to simulate this.

@DrahtBot
Copy link
Contributor

DrahtBot commented Feb 9, 2024

🐙 This pull request conflicts with the target branch and needs rebase.

@achow101 achow101 removed their request for review April 16, 2024 19:31
@achow101
Copy link
Member

Closing as this is going to be superseded by Sand Compactor

@achow101 achow101 closed this Apr 24, 2024
@bitcoin bitcoin locked and limited conversation to collaborators Apr 24, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants