Skip to content

ci: Use clang-8 for fuzzing to run on aarch64 ci systems #17470

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Nov 18, 2019
Merged

ci: Use clang-8 for fuzzing to run on aarch64 ci systems #17470

merged 2 commits into from
Nov 18, 2019

Conversation

maflcko
Copy link
Member

@maflcko maflcko commented Nov 13, 2019 •
edited
Loading

Ubuntu bionic clang is clang version 6, which does not come with libfuzzer. So the ci system breaks down when run on aarch64.

Fix that by using clang-8

For reference, the previous error on my ci system was:

/usr/bin/ld: cannot find /usr/lib/llvm-6.0/lib/clang/6.0.0/lib/linux/libclang_rt.fuzzer-aarch64.a: No such file or directory

@maflcko maflcko added the Tests label Nov 13, 2019
@maflcko
Copy link
Member Author

maflcko commented Nov 13, 2019

Excerpt from my ci system:

Options used to compile and link:
  with wallet   = no
  with gui / qt = no
  with zmq      = no
  with test     = yes
    with prop   = no
    with fuzz   = yes
  with bench    = no
  with upnp     = no
  use asm       = yes
  sanitizers    = fuzzer,address,undefined
  debug enabled = no
  gprof enabled = no
  werror        = no

  target os     = linux
  build os      = 

  CC            = /usr/bin/ccache clang-8
  CFLAGS        = -g -O2
  CPPFLAGS      =   -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -Qunused-arguments  -DHAVE_BUILD_INFO -D__STDC_FORMAT_MACROS
  CXX           = /usr/bin/ccache clang++-8 -std=c++11
  CXXFLAGS      =   -Wstack-protector -fstack-protector-all  -Wall -Wextra -Wformat -Wvla -Wswitch -Wformat-security -Wthread-safety-analysis -Wrange-loop-analysis -Wredundant-decls  -Wno-unused-parameter -Wno-self-assign -Wno-unused-local-typedef -Wno-deprecated-register -Wno-implicit-fallthrough   -g -O2
  LDFLAGS       = -pthread  -Wl,-z,relro -Wl,-z,now -pie  
  ARFLAGS       = cr

travis_fold:end:configure

travis_fold:start:build
Making install in src
make[1]: Entering directory '/root/workspace/bitcoin-core/build/bitcoin-aarch64-unknown-linux-gnu/src'
make[2]: Entering directory '/root/workspace/bitcoin-core/build/bitcoin-aarch64-unknown-linux-gnu/src'
  CXX      test/fuzz/test_fuzz_address_deserialize-fuzz.o
  CXX      test/util/test_fuzz_address_deserialize-setup_common.o

@practicalswift
Copy link
Contributor

ACK fa6e01b -- diff looks correct

fanquake
fanquake approved these changes Nov 14, 2019
View reviewed changes
Copy link
Member

@fanquake fanquake left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK fa6e01b

@maflcko
Copy link
Member Author

maflcko commented Nov 14, 2019

Bumped the timeout in a new commit, because it turns out that just running over the existing seeds takes 5 minutes for me:

root@odroid:~/workspace/bitcoin-core/build/bitcoin-aarch64-unknown-linux-gnu# time ./test/fuzz/test_runner.py ../../qa-assets/fuzz_seed_corpus/
Fuzz targets found: ['address_deserialize', 'addrman_deserialize', 'banentry_deserialize', 'bech32', 'block_deserialize', 'blockheader_deserialize', 'blocklocator_deserialize', 'blockmerkleroot', 'blocktransactions_deserialize', 'blocktransactionsrequest_deserialize', 'blockundo_deserialize', 'bloomfilter_deserialize', 'coins_deserialize', 'descriptor_parse', 'diskblockindex_deserialize', 'eval_script', 'inv_deserialize', 'messageheader_deserialize', 'netaddr_deserialize', 'parse_iso8601', 'script', 'script_flags', 'service_deserialize', 'spanparsing', 'transaction', 'txoutcompressor_deserialize', 'txundo_deserialize']
Fuzz targets selected: ['blocktransactions_deserialize', 'bloomfilter_deserialize', 'parse_iso8601', 'txoutcompressor_deserialize', 'blocklocator_deserialize', 'diskblockindex_deserialize', 'service_deserialize', 'banentry_deserialize', 'address_deserialize', 'inv_deserialize', 'script_flags', 'coins_deserialize', 'descriptor_parse', 'blockheader_deserialize', 'block_deserialize', 'addrman_deserialize', 'messageheader_deserialize', 'blocktransactionsrequest_deserialize', 'txundo_deserialize', 'blockmerkleroot', 'eval_script', 'bech32', 'blockundo_deserialize', 'script', 'netaddr_deserialize', 'spanparsing', 'transaction']

real	5m5.762s
user	4m9.200s
sys	0m8.390s

@laanwj
Copy link
Member

laanwj commented Nov 18, 2019

ACK fa2ec9f

laanwj added a commit that referenced this pull request Nov 18, 2019
@laanwj
Merge #17470: ci: Use clang-8 for fuzzing to run on aarch64 ci systems
24647a0 
fa2ec9f fuzz: Bump timeout in test_runner to accomodate for slow arm64 CPUs (MarcoFalke)
fa6e01b ci: Use clang-8 for fuzzing to run on aarch64 ci systems (MarcoFalke)

Pull request description:

  Ubuntu bionic clang is clang version 6, which does not come with libfuzzer. So the ci system breaks down when run on aarch64.

  Fix that by using clang-8

  For reference, the previous error on my ci system was:

  ```
  /usr/bin/ld: cannot find /usr/lib/llvm-6.0/lib/clang/6.0.0/lib/linux/libclang_rt.fuzzer-aarch64.a: No such file or directory

ACKs for top commit:
  laanwj:
    ACK fa2ec9f

Tree-SHA512: 4954dbc36c444d1ae145290115eea6291753c9810c92003ab8d75433c3fe3bfee439d3a99dc394418275527157a8b89f04038c8b16e08c69ec9ded50fb869e70
@laanwj laanwj merged commit fa2ec9f into bitcoin:master Nov 18, 2019
@maflcko maflcko deleted the 1911-ciFuzzClang8 branch November 18, 2019 13:38
sidhujag pushed a commit to syscoin/syscoin that referenced this pull request Nov 18, 2019
@laanwj
Merge bitcoin#17470: ci: Use clang-8 for fuzzing to run on aarch64 ci…
1052cfd 
… systems

fa2ec9f fuzz: Bump timeout in test_runner to accomodate for slow arm64 CPUs (MarcoFalke)
fa6e01b ci: Use clang-8 for fuzzing to run on aarch64 ci systems (MarcoFalke)

Pull request description:

  Ubuntu bionic clang is clang version 6, which does not come with libfuzzer. So the ci system breaks down when run on aarch64.

  Fix that by using clang-8

  For reference, the previous error on my ci system was:

  ```
  /usr/bin/ld: cannot find /usr/lib/llvm-6.0/lib/clang/6.0.0/lib/linux/libclang_rt.fuzzer-aarch64.a: No such file or directory

ACKs for top commit:
  laanwj:
    ACK fa2ec9f

Tree-SHA512: 4954dbc36c444d1ae145290115eea6291753c9810c92003ab8d75433c3fe3bfee439d3a99dc394418275527157a8b89f04038c8b16e08c69ec9ded50fb869e70
@MarkLTZ MarkLTZ mentioned this pull request Apr 4, 2020
Closed
jasonbcox pushed a commit to Bitcoin-ABC/bitcoin-abc that referenced this pull request Oct 27, 2020
@Fabcien
fuzz: Bump timeout in test_runner to accomodate for slow arm64 CPUs
b4bca97 
Summary:
Backport of core [[bitcoin/bitcoin#17470 | PR17470]].

Only this commit is relevant:
bitcoin/bitcoin@fa2ec9f

Test Plan:
  ninja bitcoin-fuzzers
  ./test/fuzz/test_runner.py <path_to_corpus>

Reviewers: #bitcoin_abc, deadalnix

Reviewed By: #bitcoin_abc, deadalnix

Differential Revision: https://reviews.bitcoinabc.org/D8139
sidhujag pushed a commit to syscoin-core/syscoin that referenced this pull request Nov 10, 2020
@laanwj
Merge bitcoin#17470: ci: Use clang-8 for fuzzing to run on aarch64 ci…
189c285 
… systems

fa2ec9f fuzz: Bump timeout in test_runner to accomodate for slow arm64 CPUs (MarcoFalke)
fa6e01b ci: Use clang-8 for fuzzing to run on aarch64 ci systems (MarcoFalke)

Pull request description:

  Ubuntu bionic clang is clang version 6, which does not come with libfuzzer. So the ci system breaks down when run on aarch64.

  Fix that by using clang-8

  For reference, the previous error on my ci system was:

  ```
  /usr/bin/ld: cannot find /usr/lib/llvm-6.0/lib/clang/6.0.0/lib/linux/libclang_rt.fuzzer-aarch64.a: No such file or directory

ACKs for top commit:
  laanwj:
    ACK fa2ec9f

Tree-SHA512: 4954dbc36c444d1ae145290115eea6291753c9810c92003ab8d75433c3fe3bfee439d3a99dc394418275527157a8b89f04038c8b16e08c69ec9ded50fb869e70
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Dec 16, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@fanquake fanquake fanquake approved these changes

Assignees
No one assigned
Labels
Tests
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@maflcko @practicalswift @laanwj @fanquake