Skip to content

Conversation

darosior
Copy link
Member

@darosior darosior commented Nov 25, 2022

Alternative to #23502.

Currently, when estimating the size an input spending a Taproot output will have once signed, we always assume the key path will be used. Even if there are script paths. This can lead to pretty severe fee underestimation if the script path turns out to be used in the end. So instead assume the most expensive between all script paths and the key path will be used.

This is still not ideal, as there may be a huge gap between the size of a script path spend and a key path spend. Still, this is less bad than undershooting the fees.

@DrahtBot
Copy link
Contributor

DrahtBot commented Nov 25, 2022

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/26573.

Reviews

See the guideline for information on the review process.

Type Reviewers
Concept ACK sipa, murchandamus, furszy
Stale ACK achow101

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

No conflicts as of last run.

@luke-jr
Copy link
Member

luke-jr commented Nov 27, 2022

Alternative to #23502.

Not really an alternative, since #26567 shouldn't be eligible for backporting, but this is arguably a fix.

IMO #23502 should still get reviewed & merged first, then let #26567 include this in its refactor.

@achow101
Copy link
Member

achow101 commented Feb 4, 2023

Concept ACK

@darosior darosior force-pushed the taproot_over_dont_under_estimate branch from 03fb943 to 08f288f Compare February 5, 2023 18:18
@darosior
Copy link
Member Author

darosior commented Feb 5, 2023

Rebased.

@fanquake
Copy link
Member

Moving this to draft for now, given it's based on #26567.

@fanquake fanquake marked this pull request as draft February 24, 2023 15:27
@darosior darosior force-pushed the taproot_over_dont_under_estimate branch from 08f288f to 9120567 Compare July 19, 2023 13:17
@darosior
Copy link
Member Author

Rebased. Still based on #26567.

@achow101
Copy link
Member

ACK 52cc58a modulo rebase

@DrahtBot DrahtBot requested review from murchandamus and sipa April 15, 2024 21:56
@darosior darosior force-pushed the taproot_over_dont_under_estimate branch from 52cc58a to efb5d24 Compare June 28, 2024 11:07
@darosior
Copy link
Member Author

Rebased.

@darosior
Copy link
Member Author

darosior commented Jun 28, 2024

CI is erroring on

2024-06-28T11:41:00.1099453Z 5/309 - �[1mwallet_fundrawtransaction.py --descriptors�[0m failed, Duration: 30 s
2024-06-28T11:41:00.1100206Z 
2024-06-28T11:41:00.1100438Z �[1mstdout:
2024-06-28T11:41:00.1101244Z �[0m2024-06-28T11:40:29.220000Z TestFramework (INFO): PRNG seed is: 3960890322720202555
2024-06-28T11:41:00.1108001Z 2024-06-28T11:40:29.221000Z TestFramework (INFO): Initializing test directory /home/runner/work/_temp/ci/scratch/test_runner/test_runner_₿_🏃_20240628_113945/wallet_fundrawtransaction_297
2024-06-28T11:41:00.1110607Z 2024-06-28T11:40:32.793000Z TestFramework (INFO): Connect nodes, set fees, generate blocks, and sync
2024-06-28T11:41:00.1112058Z 2024-06-28T11:40:37.036000Z TestFramework (INFO): Test 'add_inputs' default value
2024-06-28T11:41:00.1116113Z 2024-06-28T11:40:38.689000Z TestFramework (INFO): Test wallet preset inputs are not double-counted or reused in coin selection
2024-06-28T11:41:00.1118223Z 2024-06-28T11:40:39.538000Z TestFramework (INFO): Test weight calculation with external inputs
2024-06-28T11:41:00.1119520Z 2024-06-28T11:40:40.376000Z TestFramework (INFO): Test weight limits
2024-06-28T11:41:00.1120605Z 2024-06-28T11:40:57.670000Z TestFramework (ERROR): Assertion failed
2024-06-28T11:41:00.1121410Z Traceback (most recent call last):
2024-06-28T11:41:00.1123136Z   File "/home/runner/work/_temp/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_framework/util.py", line 165, in try_rpc
2024-06-28T11:41:00.1124528Z     fun(*args, **kwds)
2024-06-28T11:41:00.1126042Z   File "/home/runner/work/_temp/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_framework/coverage.py", line 50, in __call__
2024-06-28T11:41:00.1127695Z     return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
2024-06-28T11:41:00.1128575Z                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-28T11:41:00.1130261Z   File "/home/runner/work/_temp/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_framework/authproxy.py", line 146, in __call__
2024-06-28T11:41:00.1131879Z     raise JSONRPCException(response['error'], status)
2024-06-28T11:41:00.1139938Z test_framework.authproxy.JSONRPCException: The combination of the pre-selected inputs and the wallet automatic inputs selection exceeds the transaction maximum weight. Please try sending a smaller amount or manually consolidating your wallet's UTXOs (-4)
2024-06-28T11:41:00.1141911Z 
2024-06-28T11:41:00.1142328Z During handling of the above exception, another exception occurred:
2024-06-28T11:41:00.1143194Z 
2024-06-28T11:41:00.1143417Z Traceback (most recent call last):
2024-06-28T11:41:00.1145069Z   File "/home/runner/work/_temp/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 132, in main
2024-06-28T11:41:00.1146447Z     self.run_test()
2024-06-28T11:41:00.1148039Z   File "/home/runner/work/_temp/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/wallet_fundrawtransaction.py", line 117, in run_test
2024-06-28T11:41:00.1149453Z     self.test_weight_limits()
2024-06-28T11:41:00.1151071Z   File "/home/runner/work/_temp/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/wallet_fundrawtransaction.py", line 1335, in test_weight_limits
2024-06-28T11:41:00.1157890Z     assert_raises_rpc_error(-4, "Transaction too large", wallet.fundrawtransaction, hexstring=rawtx, input_weights=input_weights)
2024-06-28T11:41:00.1160275Z   File "/home/runner/work/_temp/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_framework/util.py", line 156, in assert_raises_rpc_error
2024-06-28T11:41:00.1161994Z     assert try_rpc(code, message, fun, *args, **kwds), "No exception raised"
2024-06-28T11:41:00.1162821Z            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-06-28T11:41:00.1174793Z   File "/home/runner/work/_temp/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_framework/util.py", line 171, in try_rpc
2024-06-28T11:41:00.1176133Z     raise AssertionError(
2024-06-28T11:41:00.1176759Z AssertionError: Expected substring not found in error message:
2024-06-28T11:41:00.1177569Z substring: 'Transaction too large'
2024-06-28T11:41:00.1179589Z error message: 'The combination of the pre-selected inputs and the wallet automatic inputs selection exceeds the transaction maximum weight. Please try sending a smaller amount or manually consolidating your wallet's UTXOs'.
2024-06-28T11:41:00.1181671Z 2024-06-28T11:40:57.728000Z TestFramework (INFO): Stopping nodes
2024-06-28T11:41:00.1184029Z 2024-06-28T11:40:58.407000Z TestFramework (WARNING): Not cleaning up dir /home/runner/work/_temp/ci/scratch/test_runner/test_runner_₿_🏃_20240628_113945/wallet_fundrawtransaction_297
2024-06-28T11:41:00.1187216Z 2024-06-28T11:40:58.407000Z TestFramework (ERROR): Test failed. Test logging available at /home/runner/work/_temp/ci/scratch/test_runner/test_runner_₿_🏃_20240628_113945/wallet_fundrawtransaction_297/test_framework.log
2024-06-28T11:41:00.1189460Z 2024-06-28T11:40:58.407000Z TestFramework (ERROR): 
2024-06-28T11:41:00.1192381Z 2024-06-28T11:40:58.408000Z TestFramework (ERROR): Hint: Call /home/runner/work/_temp/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/combine_logs.py '/home/runner/work/_temp/ci/scratch/test_runner/test_runner_₿_🏃_20240628_113945/wallet_fundrawtransaction_297' to consolidate all logs
2024-06-28T11:41:00.1195268Z 2024-06-28T11:40:58.408000Z TestFramework (ERROR): 
2024-06-28T11:41:00.1197110Z 2024-06-28T11:40:58.408000Z TestFramework (ERROR): If this failure happened unexpectedly or intermittently, please file a bug and provide a link or upload of the combined log.
2024-06-28T11:41:00.1199145Z 2024-06-28T11:40:58.408000Z TestFramework (ERROR): https://github.com/bitcoin/bitcoin/issues
2024-06-28T11:41:00.1200240Z 2024-06-28T11:40:58.408000Z TestFramework (ERROR): 

I'll look into it.

So i can't reproduce locally, even when using the same RNG seed. I'm trying building with ASAN since that's where it seems to show up in CI. It also seems very unrelated to this change, the line where it errors is when spending legacy outputs...

Couldn't reproduce with sanitizers either.

We were previously assuming the key path was always used for size
estimation, which could lead to underestimate the fees if one of the
script paths was used in the end.

Instead, overestimate: use the most expensive between the key path and
all existing script paths.

The functional test changes were authored by Ava Chow for PR 23502.
Co-Authored-by: Ava Chow <github@achow101.com>
@darosior darosior force-pushed the taproot_over_dont_under_estimate branch from efb5d24 to f74a668 Compare June 28, 2024 15:48
@darosior
Copy link
Member Author

Force-pushed to add some Assumes. Looks like the CI issue was unrelated.

Copy link
Contributor

@murchandamus murchandamus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concept ACK

res = rpc_online.sendtoaddress(address=self.boring.getnewaddress(), amount=Decimal(ret_amnt) / 100000000, subtractfeefromamount=True, fee_rate=200)
res = rpc_online.sendtoaddress(address=self.boring.getnewaddress(), amount=Decimal(ret_amnt) / 100000000, subtractfeefromamount=True, fee_rate=10)
txinfo = rpc_online.gettransaction(txid=res, verbose=True)
assert_fee_enough(-txinfo["fee"], txinfo["decoded"]["vsize"], Decimal(0.00010000))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do I understand right that this test will try the whole battery of different script trees seen below and uses the maximal estimate across script trees, so if there are multiple different leaves that we can satisfy, our transaction might overpay, but no longer underpay?

@furszy
Copy link
Member

furszy commented Feb 6, 2025

Concept ACK. Will review soon.

@DrahtBot
Copy link
Contributor

Needs rebase, if still relevant

@darosior
Copy link
Member Author

Closing my old open PRs. I have intended to come back to it for a while but as a matter of fact i did not. Anyone interested feel free to pick it up.

@fanquake
Copy link
Member

Picked up in #32964.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants