Skip to content

Conversation

achow101
Copy link
Member

As noted in #17156 (comment), the PSBT fuzzer is triggering a UBSan condition which is fixed by this PR. Additionally, by fixing this, some UBSan suppressions can be removed.

Split from #17156

@@ -412,6 +412,9 @@ class CScript : public CScriptBase
CScript(std::vector<unsigned char>::const_iterator pbegin, std::vector<unsigned char>::const_iterator pend) : CScriptBase(pbegin, pend) { }
CScript(const unsigned char* pbegin, const unsigned char* pend) : CScriptBase(pbegin, pend) { }

// Define a copy constructor so that operator= is a copy instead of a move to avoid undefined behavior
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is needed, there is another problem. Why does a move cause UB? And why is a move being invoked in the first place with operator=?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No idea. Maybe @practicalswift can shed some light here?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See also: #17156 (comment)

@maflcko
Copy link
Member

maflcko commented Nov 18, 2019

For reference, one tb is this:

vim $(pwd)/test/sanitizer_suppressions # Remove the prevector suppressions 
./configure  --with-sanitizers=address,undefined CC=clang CXX=clang++
make
export LSAN_OPTIONS="suppressions=$(pwd)/test/sanitizer_suppressions/lsan"
export TSAN_OPTIONS="suppressions=$(pwd)/test/sanitizer_suppressions/tsan"
export UBSAN_OPTIONS="suppressions=$(pwd)/test/sanitizer_suppressions/ubsan:print_stacktrace=1:halt_on_error=1"
$ ./src/test/test_bitcoin -t denialofservice_tests
Running 6 test cases...
prevector.h:452:19: runtime error: reference binding to misaligned address 0x7ffdaaebe012 for type 'prevector<28, unsigned char, unsigned int, int>::size_type' (aka 'unsigned int'), which requires 4 byte alignment
0x7ffdaaebe012: note: pointer points here
 00 00  00 00 00 00 00 00 6c 00  00 00 20 9e 01 00 b0 60  00 00 00 00 00 00 00 00  00 00 00 00 00 00
              ^ 
    #0 0x56076599c81d in prevector<28u, unsigned char, unsigned int, int>::swap(prevector<28u, unsigned char, unsigned int, int>&) ./src/./prevector.h:452:9
    #1 0x560765991f58 in prevector<28u, unsigned char, unsigned int, int>::operator=(prevector<28u, unsigned char, unsigned int, int>&&) ./src/./prevector.h:272:9
    #2 0x560765991f58 in CScript::operator=(CScript&&) ./src/./script/script.h:390
    #3 0x5607673635b5 in ProduceSignature(SigningProvider const&, BaseSignatureCreator const&, CScript const&, SignatureData&) ./src/script/sign.cpp:240:23
    #4 0x56076736766c in SignSignature(SigningProvider const&, CScript const&, CMutableTransaction&, unsigned int, long const&, int) ./src/script/sign.cpp:375:16
    #5 0x5607673679bb in SignSignature(SigningProvider const&, CTransaction const&, CMutableTransaction&, unsigned int, int) ./src/script/sign.cpp:387:12
    #6 0x560765d5c515 in denialofservice_tests::DoS_mapOrphans::test_method() ./src/test/denialofservice_tests.cpp:403:9
    #7 0x560765d5a4bc in denialofservice_tests::DoS_mapOrphans_invoker() ./src/test/denialofservice_tests.cpp:369:1
    #8 0x560765aa4cc5 in boost::detail::function::void_function_invoker0<void (*)(), void>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:117:11
    #9 0x7fa1a39b7581  (/lib64/libboost_unit_test_framework.so.1.69.0+0x30581)
    #10 0x7fa1a39b65ec in boost::execution_monitor::catch_signals(boost::function<int ()> const&) (/lib64/libboost_unit_test_framework.so.1.69.0+0x2f5ec)
    #11 0x7fa1a39b6677 in boost::execution_monitor::execute(boost::function<int ()> const&) (/lib64/libboost_unit_test_framework.so.1.69.0+0x2f677)
    #12 0x7fa1a39b674d in boost::execution_monitor::vexecute(boost::function<void ()> const&) (/lib64/libboost_unit_test_framework.so.1.69.0+0x2f74d)
    #13 0x7fa1a39e124e in boost::unit_test::unit_test_monitor_t::execute_and_translate(boost::function<void ()> const&, unsigned int) (/lib64/libboost_unit_test_framework.so.1.69.0+0x5a24e)
    #14 0x7fa1a39c667f  (/lib64/libboost_unit_test_framework.so.1.69.0+0x3f67f)
    #15 0x7fa1a39c6913  (/lib64/libboost_unit_test_framework.so.1.69.0+0x3f913)
    #16 0x7fa1a39c6913  (/lib64/libboost_unit_test_framework.so.1.69.0+0x3f913)
    #17 0x7fa1a39bdc73 in boost::unit_test::framework::run(unsigned long, bool) (/lib64/libboost_unit_test_framework.so.1.69.0+0x36c73)
    #18 0x7fa1a39e00d1 in boost::unit_test::unit_test_main(bool (*)(), int, char**) (/lib64/libboost_unit_test_framework.so.1.69.0+0x590d1)
    #19 0x5607659a1526 in main /usr/include/boost/test/unit_test.hpp:63:12
    #20 0x7fa1a2f8df42 in __libc_start_main (/lib64/libc.so.6+0x23f42)
    #21 0x560765861f8d in _start (./src/test/test_bitcoin+0x2a59f8d)

SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior prevector.h:452:19 in 

@DrahtBot
Copy link
Contributor

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #17208 (Make all tests pass UBSan without using any UBSan suppressions by practicalswift)
  • #10785 (Serialization improvements by sipa)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@laanwj
Copy link
Member

laanwj commented Nov 19, 2019

If this is needed, there is another problem. Why does a move cause UB? And why is a move being invoked in the first place with operator=?

Agree. This is kind of scary, and working around it like this seems a bad idea if we don't understand why it happens.

@laanwj
Copy link
Member

laanwj commented Nov 19, 2019

BTW, this reverts 2ddfcfd (#9349), with the difference that the new code invokes CScript instead of CScriptBase.

@laanwj
Copy link
Member

laanwj commented Nov 19, 2019

Could this pragma in prevector.h be causing alignment issues? (doesn't it mean something like "ignore all padding requirements within the struct")

#pragma pack(push, 1)

@maflcko
Copy link
Member

maflcko commented Nov 19, 2019

Smaller code to reproduce:

struct Test {
    uint8_t a : 8;
    CScript script;
};


{
    const std::array<unsigned char, 5> D{1, 2, 3, 4, 5};
    Test t{0x0f, {}};
    t.script = CScript{D.begin(), D.end()};
}

Result:

prevector.h:452:19: runtime error: reference binding to misaligned address 0x7ffe6446c741 for type 'prevector<28, unsigned char, unsigned int, int>::size_type' (aka 'unsigned int'), which requires 4 byte alignment
0x7ffe6446c741: note: pointer points here
 7f 00 00  0f 00 00 00 00 01 02 03  04 05 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00
              ^ 
    #0 0x555dd33f9a7d in prevector<28u, unsigned char, unsigned int, int>::swap(prevector<28u, unsigned char, unsigned int, int>&) /home/marco/workspace/btc_bitcoin_core/src/./prevector.h:452:9
    #1 0x555dd33ef1b8 in prevector<28u, unsigned char, unsigned int, int>::operator=(prevector<28u, unsigned char, unsigned int, int>&&) /home/marco/workspace/btc_bitcoin_core/src/./prevector.h:272:9
    #2 0x555dd33ef1b8 in CScript::operator=(CScript&&) /home/marco/workspace/btc_bitcoin_core/src/./script/script.h:390
    #3 0x555dd37a0aaa in a() /tmp/a.cpp:79:11

@laanwj
Copy link
Member

laanwj commented Nov 19, 2019

Does it go away if you remove the pragma?

@maflcko
Copy link
Member

maflcko commented Nov 19, 2019

Yes, with this diff it goes away:

diff --git a/src/prevector.h b/src/prevector.h
index d307495fbe..a55bcb50fe 100644
--- a/src/prevector.h
+++ b/src/prevector.h
@@ -14,7 +14,6 @@
 #include <cstddef>
 #include <type_traits>
 
-#pragma pack(push, 1)
 /** Implements a drop-in replacement for std::vector<T> which stores up to N
  *  elements directly (without heap allocation). The types Size and Diff are
  *  used to store element counts, and can be any unsigned + signed type.
@@ -522,6 +521,5 @@ public:
         return item_ptr(0);
     }
 };
-#pragma pack(pop)
 
 #endif // BITCOIN_PREVECTOR_H

@laanwj
Copy link
Member

laanwj commented Nov 19, 2019

I would prefer that solution, then. I'm not sure in how far unaligned integers are really UB in C++ (versus architecture/implementation defined), but it'd be better to avoid that for architectures that don't support (or have slow fallback paths for) unaligned reads/writes. (E.g. SiFive U540 traps into the kernel/bootloader to handle unaligned reads and writes, with a function, in software. Wouldn't be surprised if it's 1000 times slower)

@practicalswift
Copy link
Contributor

@laanwj John Regehr and Pascal Cuoq's excellent post "Undefined Behavior in 2017" has a nice section on Alignment Violations. Highly recommended reading!

laanwj added a commit to laanwj/bitcoin that referenced this pull request Nov 20, 2019
`#pragma pack(1)` prevents aligning the struct and its members to
their required alignment. This can result in code that performs non-aligned reads and writes
to integers and pointers, which is problematic on some architectures.

It also triggers UBsan —
see bitcoin#17156 (comment) and bitcoin#17510.
@achow101 achow101 closed this Nov 20, 2019
ajtowns added a commit to ajtowns/bitcoin that referenced this pull request Dec 10, 2019
`#pragma pack(1)` prevents aligning the struct and its members to their
required alignment. This can result in code that performs non-aligned
reads and writes to integers and pointers, which is problematic on some
architectures.

It also triggers UBsan — see
  bitcoin#17156 (comment)
and bitcoin#17510.
HashUnlimited pushed a commit to HashUnlimited/chaincoin that referenced this pull request Apr 17, 2020
`#pragma pack(1)` prevents aligning the struct and its members to their
required alignment. This can result in code that performs non-aligned
reads and writes to integers and pointers, which is problematic on some
architectures.

It also triggers UBsan — see
  bitcoin#17156 (comment)
and bitcoin#17510.
backpacker69 referenced this pull request in peercoin/peercoin Mar 28, 2021
`#pragma pack(1)` prevents aligning the struct and its members to their
required alignment. This can result in code that performs non-aligned
reads and writes to integers and pointers, which is problematic on some
architectures.

It also triggers UBsan — see
  bitcoin/bitcoin#17156 (comment)
and #17510.
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Dec 16, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants