fuzz: execute each file in dir without fuzz engine #22763
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ajtowns
reviewed
Oct 15, 2021
| buffer.clear(); | ||
| } | ||
| } | ||
| std::cout << "tested " << tested << " files\n"; |
There was a problem hiding this comment.
I think this message would be much better if it included the name of the fuzzer doing the testing. (Need to copy it into a global from initialize to make that happen)
Reporting the time taken might also be good?
nosakhare1717
approved these changes
Oct 30, 2021
|
🐙 This pull request conflicts with the target branch and needs rebase. Want to unsubscribe from rebase notifications on this pull request? Just convert this pull request to a "draft". |
maflcko
pushed a commit
to maflcko/bitcoin-core
that referenced
this pull request
Mar 17, 2022
f59bee3 fuzz: execute each file in dir without fuzz engine (Anthony Towns) Pull request description: Phony fuzzing (phuzzing)! Run the fuzz testing code against known inputs to detect errors. Advantage is you can easily test using the existing qa-assets datasets without having to compile with fuzzing enabled; disadvantage is that it doesn't do any actual fuzzing. Example usage: ``` $ for a in ${QA_ASSETS}/fuzz_seed_corpus/*; do echo ${a##*/}; done | xargs -P8 -I {} /bin/sh -c "FUZZ={} test/fuzz/fuzz ${QA_ASSETS}/fuzz_seed_corpus/{}" No fuzzer for address_deserialize. No fuzzer for addrdb. No fuzzer for banentry_deserialize. addition_overflow: succeeded against 848 files in 0s. asmap: succeeded against 981 files in 0s. checkqueue: succeeded against 211 files in 0s. ... ``` (`-P8` says run 8 of the tasks in parallel) If there are failures, the first one will be reported and the program will abort with output like: ``` fuzz: test/fuzz/versionbits.cpp:336: void (anonymous namespace)::versionbits_fuzz_target(FuzzBufferType): Assertion `exp_state != ThresholdState::FAILED' failed. Error processing seed "corpus/versionbits/35345ae8e722234095810b1117a29b63af7621af" ``` Rebase of bitcoin#22763, which was a rebase of bitcoin#21496, but also reports the name of the fuzzer and the time taken. Fixes bitcoin#21461 Top commit has no ACKs. Tree-SHA512: d8d046d4a309652eb13de42116276bf992480bc887ad3535a8ff18b354cb24826bc562b06af63802ec945c637f046563b6a5601d6321b46a5543127daafea09b
sidhujag
pushed a commit
to syscoin/syscoin
that referenced
this pull request
Mar 18, 2022
f59bee3 fuzz: execute each file in dir without fuzz engine (Anthony Towns) Pull request description: Phony fuzzing (phuzzing)! Run the fuzz testing code against known inputs to detect errors. Advantage is you can easily test using the existing qa-assets datasets without having to compile with fuzzing enabled; disadvantage is that it doesn't do any actual fuzzing. Example usage: ``` $ for a in ${QA_ASSETS}/fuzz_seed_corpus/*; do echo ${a##*/}; done | xargs -P8 -I {} /bin/sh -c "FUZZ={} test/fuzz/fuzz ${QA_ASSETS}/fuzz_seed_corpus/{}" No fuzzer for address_deserialize. No fuzzer for addrdb. No fuzzer for banentry_deserialize. addition_overflow: succeeded against 848 files in 0s. asmap: succeeded against 981 files in 0s. checkqueue: succeeded against 211 files in 0s. ... ``` (`-P8` says run 8 of the tasks in parallel) If there are failures, the first one will be reported and the program will abort with output like: ``` fuzz: test/fuzz/versionbits.cpp:336: void (anonymous namespace)::versionbits_fuzz_target(FuzzBufferType): Assertion `exp_state != ThresholdState::FAILED' failed. Error processing seed "corpus/versionbits/35345ae8e722234095810b1117a29b63af7621af" ``` Rebase of bitcoin#22763, which was a rebase of bitcoin#21496, but also reports the name of the fuzzer and the time taken. Fixes bitcoin#21461 Top commit has no ACKs. Tree-SHA512: d8d046d4a309652eb13de42116276bf992480bc887ad3535a8ff18b354cb24826bc562b06af63802ec945c637f046563b6a5601d6321b46a5543127daafea09b
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Rebased PR #21496
Fixes #21461
This supports whole directory runs of the fuzzing seeds without needing a fuzzing library.
Reviewers are also requested look at the issue #76 in
bitcoin-core/qa-assets.Testing instructions:
To build without libFuzzer, exclude the sanitizers.
Tests: