fuzz: execute each file in dir without fuzz engine #24472
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Concept ACK. |
|
ACK. Looks good to me, sorry for dropping that originally. Ran through some of the same tests I was doing before, all working as expected. Example of that: |
maflcko
approved these changes
Mar 16, 2022
| if (signal == SIGABRT) { | ||
| std::cerr << "Error processing seed " << g_seed_path << std::endl; | ||
| } else { | ||
| std::cerr << "Unexpected signal " << signal << " received\n"; |
There was a problem hiding this comment.
Unclear if this stl call is allowed? Might be UB, but I guess it doesn't matter either way?
There was a problem hiding this comment.
It matches the code in https://en.cppreference.com/w/cpp/utility/program/abort so should be mostly okay hopefully?
I suppose in theory you could setup worker threads, and have a monitor thread watch for them aborting -- then you could do multiple inputs in parallel, and report on multiple failures rather than exiting after the first one.
Co-Authored-By: Anthony Ronning <anthonyronning@gmail.com>
80dd301 to
f59bee3
Compare
|
Renamed seed to input, added static/std, switched from signal.h to csignal header. |
maflcko
approved these changes
Mar 17, 2022
sidhujag
pushed a commit
to syscoin/syscoin
that referenced
this pull request
Mar 18, 2022
f59bee3 fuzz: execute each file in dir without fuzz engine (Anthony Towns) Pull request description: Phony fuzzing (phuzzing)! Run the fuzz testing code against known inputs to detect errors. Advantage is you can easily test using the existing qa-assets datasets without having to compile with fuzzing enabled; disadvantage is that it doesn't do any actual fuzzing. Example usage: ``` $ for a in ${QA_ASSETS}/fuzz_seed_corpus/*; do echo ${a##*/}; done | xargs -P8 -I {} /bin/sh -c "FUZZ={} test/fuzz/fuzz ${QA_ASSETS}/fuzz_seed_corpus/{}" No fuzzer for address_deserialize. No fuzzer for addrdb. No fuzzer for banentry_deserialize. addition_overflow: succeeded against 848 files in 0s. asmap: succeeded against 981 files in 0s. checkqueue: succeeded against 211 files in 0s. ... ``` (`-P8` says run 8 of the tasks in parallel) If there are failures, the first one will be reported and the program will abort with output like: ``` fuzz: test/fuzz/versionbits.cpp:336: void (anonymous namespace)::versionbits_fuzz_target(FuzzBufferType): Assertion `exp_state != ThresholdState::FAILED' failed. Error processing seed "corpus/versionbits/35345ae8e722234095810b1117a29b63af7621af" ``` Rebase of bitcoin#22763, which was a rebase of bitcoin#21496, but also reports the name of the fuzzer and the time taken. Fixes bitcoin#21461 Top commit has no ACKs. Tree-SHA512: d8d046d4a309652eb13de42116276bf992480bc887ad3535a8ff18b354cb24826bc562b06af63802ec945c637f046563b6a5601d6321b46a5543127daafea09b
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Phony fuzzing (phuzzing)! Run the fuzz testing code against known inputs to detect errors. Advantage is you can easily test using the existing qa-assets datasets without having to compile with fuzzing enabled; disadvantage is that it doesn't do any actual fuzzing.
Example usage:
(
-P8says run 8 of the tasks in parallel)If there are failures, the first one will be reported and the program will abort with output like:
Rebase of #22763, which was a rebase of #21496, but also reports the name of the fuzzer and the time taken.
Fixes #21461