Skip to content

Conversation

instagibbs
Copy link
Member

@instagibbs instagibbs commented Apr 15, 2024

In master we are hitting a case where vsize transactions much larger than max standard size are causing an overflow in not-yet-exposed RBF diagram checking code: #29757 (comment)

ConsumeTxMemPoolEntry is creating entries with tens of thousands of sigops cost, causing the resulting RBFs to be "overly large".

To fix this I cause the fuzz test to stop adding transactions to the mempool when we reach a potential overflow of int32_t.

@DrahtBot
Copy link
Contributor

DrahtBot commented Apr 15, 2024

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage

For detailed information about the code coverage, see the test coverage report.

Reviews

See the guideline for information on the review process.

Type Reviewers
ACK glozow, marcofleon
Stale ACK sipa

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #29757 (feefrac: avoid explicitly computing diagram; compare based on chunks by sipa)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@DrahtBot DrahtBot added the Tests label Apr 15, 2024
@@ -23,7 +23,15 @@ namespace {
const BasicTestingSetup* g_setup;
} // namespace

// Maximum number of things we will effect during an RBF
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is hopefully a conservative over-estimate since clusters should only be 101kvB total each

@sipa
Copy link
Member

sipa commented Apr 15, 2024

I'm not convinced about this approach. A nice thing about fuzz test is that they can be written to exercise scenarios which go beyond what matters in practice, as more extreme values may trigger bugs more easily (and those bugs may affect realistic cases too, but be harder to trigger there).

More specifically, in this case it seems the issue is the code under test not working if the sum of sizes exceeds 2^31. The standard transaction size shouldn't matter for this code, so I'd prefer to restrict to supported cases more directly. E.g. by reading from the fuzzer only sizes in the range (1...2^31-1-sum_of_sizes_so_far) and/or stop adding transactions when the sum of sizes overflows.

If that's too hard to do, the current approach is fine of course, but even then, there is no strict need to tie it to standard sizes.

@instagibbs
Copy link
Member Author

instagibbs commented Apr 15, 2024

there is no strict need to tie it to standard sizes.

Pretty sure we can overflow if we allow ~1MvB transactions(which is effectively being done via sigops), as 10,000 * 1,000,000 > 2^31

Made a more direct limiting attempt. I will squash if it is preferred

@instagibbs instagibbs force-pushed the 2024-04-package-rbf-overflow branch 2 times, most recently from 9ec18cf to 914b6db Compare April 15, 2024 17:30
@DrahtBot
Copy link
Contributor

🚧 At least one of the CI tasks failed. Make sure to run all tests locally, according to the
documentation.

Possibly this is due to a silent merge conflict (the changes in this pull request being
incompatible with the current code in the target branch). If so, make sure to rebase on the latest
commit of the target branch.

Leave a comment here, if you need help tracking down a confusing failure.

Debug: https://github.com/bitcoin/bitcoin/runs/23837896395

Copy link
Member

@glozow glozow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approach of breaking when adding another tx would overflow seems fine to me. Did you forget to squash?

@instagibbs instagibbs force-pushed the 2024-04-package-rbf-overflow branch from 914b6db to 9d76d77 Compare April 16, 2024 11:31
@instagibbs
Copy link
Member Author

@glozow was waiting for positive feedback. Squashed.

@sipa
Copy link
Member

sipa commented Apr 16, 2024

utACK 9d76d77

@instagibbs instagibbs force-pushed the 2024-04-package-rbf-overflow branch from 9d76d77 to 016ed24 Compare April 16, 2024 15:28
@instagibbs
Copy link
Member Author

pushed a fixup since I think it could have resulted in UB when selecting direct conflicts from the mempool_txs vector.

Copy link
Member

@glozow glozow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 016ed24

@DrahtBot DrahtBot requested a review from sipa April 17, 2024 20:40
Copy link
Contributor

@marcofleon marcofleon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ran into this bug when I first started fuzzing package_rbf a couple weeks ago... was unsure of what to do about it at the time. But glad to see it addressed here.

ACK 016ed24. I ran libFuzzer on package_rbf on the current master branch until the overflow was encountered. Then I built the PR branch and ran the fuzzer using the crash input.

FUZZ=package_rbf ./src/test/fuzz/fuzz ../crash-d96abc99c0e424a47aa8f4080040e1bc3e3851f7

No crash.

INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 1016851959
INFO: Loaded 1 modules   (1208798 inline 8-bit counters): 1208798 [0x107caa4c0, 0x107dd169e), 
INFO: Loaded 1 PC tables (1208798 PCs): 1208798 [0x107dd16a0,0x109043480), 
./src/test/fuzz/fuzz: Running 1 inputs 1 time(s) each.
Running: ../crash-d96abc99c0e424a47aa8f4080040e1bc3e3851f7
Executed ../crash-d96abc99c0e424a47aa8f4080040e1bc3e3851f7 in 1 ms
***
*** NOTE: fuzzing was not performed, you have only
***       executed the target code on a fixed set of inputs.
***

I've also been running libFuzzer on package_rbf for a while now and the overflow bug hasn't come up.

@glozow glozow merged commit ba7c67f into bitcoin:master Apr 22, 2024
@bitcoin bitcoin locked and limited conversation to collaborators Apr 22, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants