build: Add format string linter #13705

practicalswift · 2018-07-18T21:40:08Z

Add format string linter.

This linter checks that the number of arguments passed to each variadic format string function matches the number of format specifiers in the format string.

Example output:

$ test/lint/lint-format-strings.sh
src/init.cpp: Expected 2 argument(s) after format string but found 1 argument(s): 
  LogPrintf("We have a mismatch here: foo=%s bar=%d\n", foo)
src/init.cpp: Expected 1 argument(s) after format string but found 2 argument(s):
  LogPrint(BCLog::RPC, "RPC stopped. This is a mismatch: %s\n", s1, s2)
$ echo $?
1

fanquake · 2018-07-19T01:51:15Z

@practicalswift Is this for a specific bug/issue you've seen previously?

practicalswift · 2018-07-19T13:47:36Z

@fanquake In the past there has been at least one incident where a mistake in the log message formatting resulted in a near DoS vulnerability. It was an under-tested error path in the network code with an invalid LogPrintf() spec. @laanwj has more details.

@laanwj suggested writing a format string linter in #9423 (comment).

laanwj · 2018-07-25T13:45:33Z

Yes, I think this is very useful! we've had some issues with this in the past.
Better to find these issues before they cause runtime problems.

Thank for writing it in Python instead of making a long shell mess, this means I can usefully review it.

laanwj · 2018-08-07T12:03:59Z

Needs rebase for some reason

This linter checks that the number of arguments passed to each variadic format string function matches the number of format specifiers in the format string.

practicalswift · 2018-08-07T12:07:38Z

@laanwj Weird! Now rebased!

laanwj · 2018-08-07T12:36:21Z

Thanks, new error:

src/wallet/wallet.h: Expected 1 argument(s) after format string but found 2 argument(s): LogPrintf(("%s " + fmt).c_str(), GetDisplayName(), parameters...)

practicalswift · 2018-08-07T12:58:55Z

@laanwj Oh, a good chance to add linting also of WalletLogPrintf format strings. Fixed. Please re-review :-)

laanwj · 2018-08-07T13:27:40Z

ACK bcd4b0f

bcd4b0f Add linting of WalletLogPrintf(...) format strings (practicalswift) a3e4556 build: Add format string linter (practicalswift) Pull request description: Add format string linter. This linter checks that the number of arguments passed to each variadic format string function matches the number of format specifiers in the format string. Example output: ``` $ test/lint/lint-format-strings.sh src/init.cpp: Expected 2 argument(s) after format string but found 1 argument(s): LogPrintf("We have a mismatch here: foo=%s bar=%d\n", foo) src/init.cpp: Expected 1 argument(s) after format string but found 2 argument(s): LogPrint(BCLog::RPC, "RPC stopped. This is a mismatch: %s\n", s1, s2) $ echo $? 1 ``` Tree-SHA512: 19ab844a63f04bf193d66682ca42745a1c7d6c454b30222491b9fe8dc047054c4a6d3ee7921ec0676fb9ca2e7f6f93bd6c97996fb09667269bd491cb875349f3

ken2812221 · 2018-08-10T10:39:20Z

This won't allow have template inside: https://travis-ci.org/bitcoin/bitcoin/jobs/414449333

practicalswift · 2018-08-10T11:15:01Z

@ken2812221 Oh, interesting! I’ll fix that. Thanks for reporting. In the meantime you can just add an exception (FALSE_POSITIVES) in the script to get around it.

…se of uninitialized memory 870f0cd build: Add MemorySanitizer (MSan) in Travis to detect use of uninitialized memory (practicalswift) Pull request description: Add MemorySanitizer (MSan) in Travis to detect use of uninitialized memory. First UBSan, then ASan followed by TSan... and now: yes, the wait is over -- **MSan is finally here!** :) Some historical context: * 2017: Continuous compilation with Clang Thread Safety analysis enabled (#10866, #10923) * 2018: Continuous testing with trapping on signed integer overflows (`-ftrapv`) (#12686) * 2018: Continuous testing of use of locale dependent functions (#13041) * 2018: Continuous testing of format strings (#13705) * 2018: Continuous compilation with MSVC `TreatWarningAsError` (#14151) * 2018: Continuous testing under UndefinedBehaviorSanitizer – UBSan (#14252, #14673, #17006) * 2018: Continuous testing under AddressSanitizer – ASan (#14794, #17205, #17674) * 2018: Continuous testing under ThreadSanitizer – TSan (#14829) * 2019: Continuous testing in an unsigned char environment (`-funsigned-char`) (#15134) * 2019: Continuous compile-time testing of assumptions we're making (#15391) * 2019: Continuous testing of fuzz test cases under Valgrind (#17633, #18159, #18166) * 2020: Finally... MemorySanitizer – MSAN! :) What is the next step? What tools should we add to CI to keep bugs from entering `master`? :) ACKs for top commit: MarcoFalke: ACK 870f0cd Tree-SHA512: 38327c8b75679d97d469fe42e704cacd1217447a5a603701dd8a58ee50b3be2c10248f8d68a479ed081c0c4b254589d3081c9183f991640b06ef689061f75578

bcd4b0f Add linting of WalletLogPrintf(...) format strings (practicalswift) a3e4556 build: Add format string linter (practicalswift) Pull request description: Add format string linter. This linter checks that the number of arguments passed to each variadic format string function matches the number of format specifiers in the format string. Example output: ``` $ test/lint/lint-format-strings.sh src/init.cpp: Expected 2 argument(s) after format string but found 1 argument(s): LogPrintf("We have a mismatch here: foo=%s bar=%d\n", foo) src/init.cpp: Expected 1 argument(s) after format string but found 2 argument(s): LogPrint(BCLog::RPC, "RPC stopped. This is a mismatch: %s\n", s1, s2) $ echo $? 1 ``` Tree-SHA512: 19ab844a63f04bf193d66682ca42745a1c7d6c454b30222491b9fe8dc047054c4a6d3ee7921ec0676fb9ca2e7f6f93bd6c97996fb09667269bd491cb875349f3

@ken2812221

… template parameter syntax 4441ad6 Make format string linter understand basic template parameter syntax (practicalswift) Pull request description: Make format string linter understand basic template parameter syntax. Fixes issue described in bitcoin#13705 (comment). Thanks to @ken2812221 for reporting! Tree-SHA512: 8fb995bc6b29d8b9926ef5969e02cf71c494e829434fcdeb4ed5fabad5ab96e86e5b8eea705e8a416927757b4fa4e58abc0fd4f483daa58c94e2c6fdcb8ee822

@ken2812221

… template parameter syntax 4441ad6 Make format string linter understand basic template parameter syntax (practicalswift) Pull request description: Make format string linter understand basic template parameter syntax. Fixes issue described in bitcoin#13705 (comment). Thanks to @ken2812221 for reporting! Tree-SHA512: 8fb995bc6b29d8b9926ef5969e02cf71c494e829434fcdeb4ed5fabad5ab96e86e5b8eea705e8a416927757b4fa4e58abc0fd4f483daa58c94e2c6fdcb8ee822

@ken2812221

… template parameter syntax 4441ad6 Make format string linter understand basic template parameter syntax (practicalswift) Pull request description: Make format string linter understand basic template parameter syntax. Fixes issue described in bitcoin#13705 (comment). Thanks to @ken2812221 for reporting! Tree-SHA512: 8fb995bc6b29d8b9926ef5969e02cf71c494e829434fcdeb4ed5fabad5ab96e86e5b8eea705e8a416927757b4fa4e58abc0fd4f483daa58c94e2c6fdcb8ee822

@ken2812221

… template parameter syntax 4441ad6 Make format string linter understand basic template parameter syntax (practicalswift) Pull request description: Make format string linter understand basic template parameter syntax. Fixes issue described in bitcoin#13705 (comment). Thanks to @ken2812221 for reporting! Tree-SHA512: 8fb995bc6b29d8b9926ef5969e02cf71c494e829434fcdeb4ed5fabad5ab96e86e5b8eea705e8a416927757b4fa4e58abc0fd4f483daa58c94e2c6fdcb8ee822

@ken2812221

… template parameter syntax 4441ad6 Make format string linter understand basic template parameter syntax (practicalswift) Pull request description: Make format string linter understand basic template parameter syntax. Fixes issue described in bitcoin#13705 (comment). Thanks to @ken2812221 for reporting! Tree-SHA512: 8fb995bc6b29d8b9926ef5969e02cf71c494e829434fcdeb4ed5fabad5ab96e86e5b8eea705e8a416927757b4fa4e58abc0fd4f483daa58c94e2c6fdcb8ee822

@ken2812221

… template parameter syntax 4441ad6 Make format string linter understand basic template parameter syntax (practicalswift) Pull request description: Make format string linter understand basic template parameter syntax. Fixes issue described in bitcoin#13705 (comment). Thanks to @ken2812221 for reporting! Tree-SHA512: 8fb995bc6b29d8b9926ef5969e02cf71c494e829434fcdeb4ed5fabad5ab96e86e5b8eea705e8a416927757b4fa4e58abc0fd4f483daa58c94e2c6fdcb8ee822

@ken2812221

… template parameter syntax 4441ad6 Make format string linter understand basic template parameter syntax (practicalswift) Pull request description: Make format string linter understand basic template parameter syntax. Fixes issue described in bitcoin#13705 (comment). Thanks to @ken2812221 for reporting! Tree-SHA512: 8fb995bc6b29d8b9926ef5969e02cf71c494e829434fcdeb4ed5fabad5ab96e86e5b8eea705e8a416927757b4fa4e58abc0fd4f483daa58c94e2c6fdcb8ee822

…etect use of uninitialized memory 870f0cd build: Add MemorySanitizer (MSan) in Travis to detect use of uninitialized memory (practicalswift) Pull request description: Add MemorySanitizer (MSan) in Travis to detect use of uninitialized memory. First UBSan, then ASan followed by TSan... and now: yes, the wait is over -- **MSan is finally here!** :) Some historical context: * 2017: Continuous compilation with Clang Thread Safety analysis enabled (bitcoin#10866, bitcoin#10923) * 2018: Continuous testing with trapping on signed integer overflows (`-ftrapv`) (bitcoin#12686) * 2018: Continuous testing of use of locale dependent functions (bitcoin#13041) * 2018: Continuous testing of format strings (bitcoin#13705) * 2018: Continuous compilation with MSVC `TreatWarningAsError` (bitcoin#14151) * 2018: Continuous testing under UndefinedBehaviorSanitizer – UBSan (bitcoin#14252, bitcoin#14673, bitcoin#17006) * 2018: Continuous testing under AddressSanitizer – ASan (bitcoin#14794, bitcoin#17205, bitcoin#17674) * 2018: Continuous testing under ThreadSanitizer – TSan (bitcoin#14829) * 2019: Continuous testing in an unsigned char environment (`-funsigned-char`) (bitcoin#15134) * 2019: Continuous compile-time testing of assumptions we're making (bitcoin#15391) * 2019: Continuous testing of fuzz test cases under Valgrind (bitcoin#17633, bitcoin#18159, bitcoin#18166) * 2020: Finally... MemorySanitizer – MSAN! :) What is the next step? What tools should we add to CI to keep bugs from entering `master`? :) ACKs for top commit: MarcoFalke: ACK 870f0cd Tree-SHA512: 38327c8b75679d97d469fe42e704cacd1217447a5a603701dd8a58ee50b3be2c10248f8d68a479ed081c0c4b254589d3081c9183f991640b06ef689061f75578

bcd4b0f Add linting of WalletLogPrintf(...) format strings (practicalswift) a3e4556 build: Add format string linter (practicalswift) Pull request description: Add format string linter. This linter checks that the number of arguments passed to each variadic format string function matches the number of format specifiers in the format string. Example output: ``` $ test/lint/lint-format-strings.sh src/init.cpp: Expected 2 argument(s) after format string but found 1 argument(s): LogPrintf("We have a mismatch here: foo=%s bar=%d\n", foo) src/init.cpp: Expected 1 argument(s) after format string but found 2 argument(s): LogPrint(BCLog::RPC, "RPC stopped. This is a mismatch: %s\n", s1, s2) $ echo $? 1 ``` Tree-SHA512: 19ab844a63f04bf193d66682ca42745a1c7d6c454b30222491b9fe8dc047054c4a6d3ee7921ec0676fb9ca2e7f6f93bd6c97996fb09667269bd491cb875349f3

practicalswift force-pushed the lint-format-strings branch 2 times, most recently from 933c042 to d418963 Compare July 18, 2018 21:49

fanquake added the Scripts and tools label Jul 18, 2018

laanwj self-assigned this Jul 25, 2018

build: Add format string linter

a3e4556

This linter checks that the number of arguments passed to each variadic format string function matches the number of format specifiers in the format string.

practicalswift force-pushed the lint-format-strings branch from d418963 to a3e4556 Compare August 7, 2018 12:07

Add linting of WalletLogPrintf(...) format strings

bcd4b0f

laanwj merged commit bcd4b0f into bitcoin:master Aug 7, 2018

practicalswift mentioned this pull request Aug 10, 2018

lint: Make format string linter understand basic template parameter syntax #13939

Merged

Bushstar mentioned this pull request Aug 13, 2018

commits from bitcoin/master FeatherCoin/Feathercoin#358

Merged

ryanofsky mentioned this pull request Jan 7, 2019

Add fee_est tool for debugging fee estimation code #10443

Closed

practicalswift mentioned this pull request Mar 7, 2020

build: Add MemorySanitizer (MSan) in Travis to detect use of uninitialized memory #18288

Merged

practicalswift deleted the lint-format-strings branch April 10, 2021 19:35

laanwj mentioned this pull request Apr 6, 2022

Port lint scripts to Python #24783

Closed

25 tasks

bitcoin locked as resolved and limited conversation to collaborators Aug 16, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

build: Add format string linter #13705

build: Add format string linter #13705

Uh oh!

practicalswift commented Jul 18, 2018

Uh oh!

fanquake commented Jul 19, 2018

Uh oh!

practicalswift commented Jul 19, 2018 •

edited

Loading

Uh oh!

laanwj commented Jul 25, 2018

Uh oh!

laanwj commented Aug 7, 2018

Uh oh!

practicalswift commented Aug 7, 2018

Uh oh!

laanwj commented Aug 7, 2018 •

edited

Loading

Uh oh!

practicalswift commented Aug 7, 2018

Uh oh!

laanwj commented Aug 7, 2018 •

edited

Loading

Uh oh!

ken2812221 commented Aug 10, 2018

Uh oh!

practicalswift commented Aug 10, 2018 •

edited

Loading

Uh oh!

Uh oh!

build: Add format string linter #13705

build: Add format string linter #13705

Uh oh!

Conversation

practicalswift commented Jul 18, 2018

Uh oh!

fanquake commented Jul 19, 2018

Uh oh!

practicalswift commented Jul 19, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

laanwj commented Jul 25, 2018

Uh oh!

laanwj commented Aug 7, 2018

Uh oh!

practicalswift commented Aug 7, 2018

Uh oh!

laanwj commented Aug 7, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

practicalswift commented Aug 7, 2018

Uh oh!

laanwj commented Aug 7, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ken2812221 commented Aug 10, 2018

Uh oh!

practicalswift commented Aug 10, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

practicalswift commented Jul 19, 2018 •

edited

Loading

laanwj commented Aug 7, 2018 •

edited

Loading

laanwj commented Aug 7, 2018 •

edited

Loading

practicalswift commented Aug 10, 2018 •

edited

Loading