build: add stack-clash and control-flow protection options to hardening flags #18921
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
configure.ac
Outdated
| @@ -768,6 +768,9 @@ if test x$use_hardening != xno; then | |||
| AX_CHECK_COMPILE_FLAG([-Wstack-protector],[HARDENED_CXXFLAGS="$HARDENED_CXXFLAGS -Wstack-protector"]) | |||
| AX_CHECK_COMPILE_FLAG([-fstack-protector-all],[HARDENED_CXXFLAGS="$HARDENED_CXXFLAGS -fstack-protector-all"]) | |||
|
|
|||
| AX_CHECK_COMPILE_FLAG([-fcf-protection],[HARDENED_CXXFLAGS="$HARDENED_CXXFLAGS -fcf-protection=full"]) | |||
There was a problem hiding this comment.
Is the mismatch between -fcf-protection (left) and -fcf-protection=full (right) intentional? :)
|
The big question is what - if any - impact these options have on performance? |
|
Strongest possible concept ACK for additional hardening: thanks a lot for taking lead on this important work! Performance impact would be nice to have quantified as @sipa requested to be able to reason about trade-offs :) |
I was hoping I could tag in @jamesob for a couple benchmarks if possible? |
This was referenced Jun 14, 2020
For reference, the guix build fails for win64 |
Enables code instrumentation of control-flow transfers. Available in GCC 8 and Clang 7. This option is now on by default in Ubuntu GCC as of 19.10.
This option causes the compiler to insert probes whenever stack space is allocated statically or dynamically to reliably detect stack overflows and thus mitigate the attack vector that relies on jumping over a stack guard page as provided by the operating system. This option is now enabled by default in Ubuntu GCC as of 19.10. Available in GCC 8 and Clang 11.
45a7f48 to
b536813
Compare
|
I've modified this to include the test case from https://gcc.gnu.org/bugzilla/show_bug.cgi?id=90458, for |
|
Setting aside for a moment whether we should enable these, the code changes look good to me. I agree with @fanquake's point here:
Rather than letting distro's make the call, we should make a decision and explicitly opt-in or opt-out ourselves. Agree with @sipa as well:
With all of the stack safety options we're tweaking lately, I think it would be helpful to make sure we're not blowing up a bunch of our chained small+common hot-path functions with wasteful stack guarding operations. The most obvious that come to mind are the byteswapping and endian-changing functions in compat/. We generally assume (I always have, anyway) that those all end up inlined and thus having a negligible performance impact, but I think there's a slight chance that these options may change that assumption. That's not to say that we wouldn't want the additional stack protection safety in that case, only that we'd want to be more mindful of inlining annotations. So in addition to running our existing benchmarks, how about adding one specifically to exercise a bunch of those small functions on either side of the fringe of inlining? |
|
I think that we should ideally opt in to these options. If they're deemed suitable for general distribution use, I don't think the performance impact can be that substantial. It's still better to measure, of course. |
|
Concept ACK on more code hardening and explicit compiler flag settings. |
Sorry for the delay here, missed the notification. Will run a few benchmarks. |
|
Changes look good to me. Note that the peer servicing this IBD test was on my local network, not co-located on the same host (I don't have enough disk space for that on a single host) but I think that should be fine given that it sort of resembles realistic use and we're just looking for red flags here. |
|
ACK b536813 ( Tested locally via bench results above. |
sidhujag
pushed a commit
to syscoin/syscoin
that referenced
this pull request
Aug 30, 2020
…on options to hardening flags b536813 build: add -fstack-clash-protection to hardening flags (fanquake) 076183b build: add -fcf-protection=full to hardening options (fanquake) Pull request description: Beginning with Ubuntu `19.10`, it's packaged GCC now has some additional hardening options enabled by default (in addition to existing defaults like `-fstack-protector-strong` and reducing the minimum ssp buffer size). The new additions are`-fcf-protection=full` and `-fstack-clash-protection`. > -fcf-protection=[full|branch|return|none] > Enable code instrumentation of control-flow transfers to increase program security by checking that target addresses of control-flow transfer instructions (such as indirect function call, function return, indirect jump) are valid. This prevents diverting the flow of control to an unexpected target. This is intended to protect against such threats as Return-oriented Programming (ROP), and similarly call/jmp-oriented programming (COP/JOP). > -fstack-clash-protection > Generate code to prevent stack clash style attacks. When this option is enabled, the compiler will only allocate one page of stack space at a time and each page is accessed immediately after allocation. Thus, it prevents allocations from jumping over any stack guard page provided by the operating system. If your interested you can grab `gcc-9_9.3.0-10ubuntu2.debian.tar.xz` from https://packages.ubuntu.com/focal/g++-9. The relevant changes are part of the `gcc-distro-specs` patches, along with the relevant additions to the gcc manages: > NOTE: In Ubuntu 19.10 and later versions, -fcf-protection is enabled by default for C, C++, ObjC, ObjC++, if none of -fno-cf-protection nor -fcf-protection=* are found. > NOTE: In Ubuntu 19.10 and later versions, -fstack-clash-protection is enabled by default for C, C++, ObjC, ObjC++, unless -fno-stack-clash-protection is found. So, if you're C++ using GCC on Ubuntu 19.10 or later, these options will be active unless you explicitly opt out. This can be observed with a small test: ```c++ int main() { return 0; } ``` ```bash g++ --version g++ (Ubuntu 9.3.0-10ubuntu2) 9.3.0 g++ test.cpp objdump -dC a.out .. 0000000000001129 <main>: 1129: f3 0f 1e fa endbr64 112d: 55 push %rbp 112e: 48 89 e5 mov %rsp,%rbp 1131: b8 00 00 00 00 mov $0x0,%eax 1136: 5d pop %rbp 1137: c3 retq 1138: 0f 1f 84 00 00 00 00 nopl 0x0(%rax,%rax,1) 113f: 00 # recompile opting out of control flow protection g++ test.cpp -fcf-protection=none objdump -dC a.out ... 0000000000001129 <main>: 1129: 55 push %rbp 112a: 48 89 e5 mov %rsp,%rbp 112d: b8 00 00 00 00 mov $0x0,%eax 1132: 5d pop %rbp 1133: c3 retq 1134: 66 2e 0f 1f 84 00 00 nopw %cs:0x0(%rax,%rax,1) 113b: 00 00 00 113e: 66 90 xchg %ax,%ax ``` Note the insertion of an `endbr64` instruction when compiling and _not_ opting out. This instruction is part of the Intel Control-flow Enforcement Technology [spec](https://software.intel.com/sites/default/files/managed/4d/2a/control-flow-enforcement-technology-preview.pdf), which the GCC control flow implementation is based on. If we're still doing gitian builds for the `0.21.0` and `0.22.0` releases, we'd likely update the gitian image to Ubuntu Focal, which would mean that the GCC used for gitian builds would also be using these options by default. So we should decide whether we want to explicitly turn these options on as part of our hardening options (although not just for this reason), or, we should be opting-out. GCC has supported both options since 8.0.0. Clang has supported `-fcf-protection` from 7.0.0 and will support `-fstack-clash-protection` in it's upcoming [11.0.0 release](https://clang.llvm.org/docs/ReleaseNotes.html#id6). ACKs for top commit: jamesob: ACK b536813 ([`jamesob/ackr/18921.1.fanquake.build_add_stack_clash_an`](https://github.com/jamesob/bitcoin/tree/ackr/18921.1.fanquake.build_add_stack_clash_an)) laanwj: Code review ACK b536813 Tree-SHA512: abc9adf23cdf1be384f5fb9aa5bfffdda86b9ecd671064298d4cda0440828b509f070f9b19c88c7ce50ead9ff32afff9f14c5e78d75f01241568fbfa077be0b7
Closed
|
This seems to be causing |
You need to provide more information. Which version of Clang? What is the output of the test in
I've just re-tested using Clang 10 and as expected, it does not recognize configure:21072: checking whether C++ compiler accepts -fcf-protection=full
configure:21091: clang++-10 -std=c++11 -c -g -O2 -fcf-protection=full -DHAVE_BUILD_INFO -D__STDC_FORMAT_MACROS conftest.cpp >&5
configure:21091: $? = 0
configure:21099: result: yes
configure:21109: checking whether C++ compiler accepts -fstack-clash-protection
configure:21121: clang++-10 -std=c++11 -c -g -O2 -O0 -fstack-clash-protection -DHAVE_BUILD_INFO -D__STDC_FORMAT_MACROS conftest.cpp >&5
clang: error: unknown argument: '-fstack-clash-protection'
configure:21121: $? = 1
configure: failed program was:
...
HARDENED_CXXFLAGS=' -Wstack-protector -fstack-protector-all -fcf-protection=full' |
|
Clang 12 (or I guess would also happen with any version of clang which supports Relevant snippet from This should explain: |
PastaPastaPasta
pushed a commit
to PastaPastaPasta/dash
that referenced
this pull request
Jun 19, 2022
…on options to hardening flags b536813 build: add -fstack-clash-protection to hardening flags (fanquake) 076183b build: add -fcf-protection=full to hardening options (fanquake) Pull request description: Beginning with Ubuntu `19.10`, it's packaged GCC now has some additional hardening options enabled by default (in addition to existing defaults like `-fstack-protector-strong` and reducing the minimum ssp buffer size). The new additions are`-fcf-protection=full` and `-fstack-clash-protection`. > -fcf-protection=[full|branch|return|none] > Enable code instrumentation of control-flow transfers to increase program security by checking that target addresses of control-flow transfer instructions (such as indirect function call, function return, indirect jump) are valid. This prevents diverting the flow of control to an unexpected target. This is intended to protect against such threats as Return-oriented Programming (ROP), and similarly call/jmp-oriented programming (COP/JOP). > -fstack-clash-protection > Generate code to prevent stack clash style attacks. When this option is enabled, the compiler will only allocate one page of stack space at a time and each page is accessed immediately after allocation. Thus, it prevents allocations from jumping over any stack guard page provided by the operating system. If your interested you can grab `gcc-9_9.3.0-10ubuntu2.debian.tar.xz` from https://packages.ubuntu.com/focal/g++-9. The relevant changes are part of the `gcc-distro-specs` patches, along with the relevant additions to the gcc manages: > NOTE: In Ubuntu 19.10 and later versions, -fcf-protection is enabled by default for C, C++, ObjC, ObjC++, if none of -fno-cf-protection nor -fcf-protection=* are found. > NOTE: In Ubuntu 19.10 and later versions, -fstack-clash-protection is enabled by default for C, C++, ObjC, ObjC++, unless -fno-stack-clash-protection is found. So, if you're C++ using GCC on Ubuntu 19.10 or later, these options will be active unless you explicitly opt out. This can be observed with a small test: ```c++ int main() { return 0; } ``` ```bash g++ --version g++ (Ubuntu 9.3.0-10ubuntu2) 9.3.0 g++ test.cpp objdump -dC a.out .. 0000000000001129 <main>: 1129: f3 0f 1e fa endbr64 112d: 55 push %rbp 112e: 48 89 e5 mov %rsp,%rbp 1131: b8 00 00 00 00 mov $0x0,%eax 1136: 5d pop %rbp 1137: c3 retq 1138: 0f 1f 84 00 00 00 00 nopl 0x0(%rax,%rax,1) 113f: 00 # recompile opting out of control flow protection g++ test.cpp -fcf-protection=none objdump -dC a.out ... 0000000000001129 <main>: 1129: 55 push %rbp 112a: 48 89 e5 mov %rsp,%rbp 112d: b8 00 00 00 00 mov $0x0,%eax 1132: 5d pop %rbp 1133: c3 retq 1134: 66 2e 0f 1f 84 00 00 nopw %cs:0x0(%rax,%rax,1) 113b: 00 00 00 113e: 66 90 xchg %ax,%ax ``` Note the insertion of an `endbr64` instruction when compiling and _not_ opting out. This instruction is part of the Intel Control-flow Enforcement Technology [spec](https://software.intel.com/sites/default/files/managed/4d/2a/control-flow-enforcement-technology-preview.pdf), which the GCC control flow implementation is based on. If we're still doing gitian builds for the `0.21.0` and `0.22.0` releases, we'd likely update the gitian image to Ubuntu Focal, which would mean that the GCC used for gitian builds would also be using these options by default. So we should decide whether we want to explicitly turn these options on as part of our hardening options (although not just for this reason), or, we should be opting-out. GCC has supported both options since 8.0.0. Clang has supported `-fcf-protection` from 7.0.0 and will support `-fstack-clash-protection` in it's upcoming [11.0.0 release](https://clang.llvm.org/docs/ReleaseNotes.html#id6). ACKs for top commit: jamesob: ACK b536813 ([`jamesob/ackr/18921.1.fanquake.build_add_stack_clash_an`](https://github.com/jamesob/bitcoin/tree/ackr/18921.1.fanquake.build_add_stack_clash_an)) laanwj: Code review ACK b536813 Tree-SHA512: abc9adf23cdf1be384f5fb9aa5bfffdda86b9ecd671064298d4cda0440828b509f070f9b19c88c7ce50ead9ff32afff9f14c5e78d75f01241568fbfa077be0b7
PastaPastaPasta
pushed a commit
to PastaPastaPasta/dash
that referenced
this pull request
Jun 19, 2022
…on options to hardening flags b536813 build: add -fstack-clash-protection to hardening flags (fanquake) 076183b build: add -fcf-protection=full to hardening options (fanquake) Pull request description: Beginning with Ubuntu `19.10`, it's packaged GCC now has some additional hardening options enabled by default (in addition to existing defaults like `-fstack-protector-strong` and reducing the minimum ssp buffer size). The new additions are`-fcf-protection=full` and `-fstack-clash-protection`. > -fcf-protection=[full|branch|return|none] > Enable code instrumentation of control-flow transfers to increase program security by checking that target addresses of control-flow transfer instructions (such as indirect function call, function return, indirect jump) are valid. This prevents diverting the flow of control to an unexpected target. This is intended to protect against such threats as Return-oriented Programming (ROP), and similarly call/jmp-oriented programming (COP/JOP). > -fstack-clash-protection > Generate code to prevent stack clash style attacks. When this option is enabled, the compiler will only allocate one page of stack space at a time and each page is accessed immediately after allocation. Thus, it prevents allocations from jumping over any stack guard page provided by the operating system. If your interested you can grab `gcc-9_9.3.0-10ubuntu2.debian.tar.xz` from https://packages.ubuntu.com/focal/g++-9. The relevant changes are part of the `gcc-distro-specs` patches, along with the relevant additions to the gcc manages: > NOTE: In Ubuntu 19.10 and later versions, -fcf-protection is enabled by default for C, C++, ObjC, ObjC++, if none of -fno-cf-protection nor -fcf-protection=* are found. > NOTE: In Ubuntu 19.10 and later versions, -fstack-clash-protection is enabled by default for C, C++, ObjC, ObjC++, unless -fno-stack-clash-protection is found. So, if you're C++ using GCC on Ubuntu 19.10 or later, these options will be active unless you explicitly opt out. This can be observed with a small test: ```c++ int main() { return 0; } ``` ```bash g++ --version g++ (Ubuntu 9.3.0-10ubuntu2) 9.3.0 g++ test.cpp objdump -dC a.out .. 0000000000001129 <main>: 1129: f3 0f 1e fa endbr64 112d: 55 push %rbp 112e: 48 89 e5 mov %rsp,%rbp 1131: b8 00 00 00 00 mov $0x0,%eax 1136: 5d pop %rbp 1137: c3 retq 1138: 0f 1f 84 00 00 00 00 nopl 0x0(%rax,%rax,1) 113f: 00 # recompile opting out of control flow protection g++ test.cpp -fcf-protection=none objdump -dC a.out ... 0000000000001129 <main>: 1129: 55 push %rbp 112a: 48 89 e5 mov %rsp,%rbp 112d: b8 00 00 00 00 mov $0x0,%eax 1132: 5d pop %rbp 1133: c3 retq 1134: 66 2e 0f 1f 84 00 00 nopw %cs:0x0(%rax,%rax,1) 113b: 00 00 00 113e: 66 90 xchg %ax,%ax ``` Note the insertion of an `endbr64` instruction when compiling and _not_ opting out. This instruction is part of the Intel Control-flow Enforcement Technology [spec](https://software.intel.com/sites/default/files/managed/4d/2a/control-flow-enforcement-technology-preview.pdf), which the GCC control flow implementation is based on. If we're still doing gitian builds for the `0.21.0` and `0.22.0` releases, we'd likely update the gitian image to Ubuntu Focal, which would mean that the GCC used for gitian builds would also be using these options by default. So we should decide whether we want to explicitly turn these options on as part of our hardening options (although not just for this reason), or, we should be opting-out. GCC has supported both options since 8.0.0. Clang has supported `-fcf-protection` from 7.0.0 and will support `-fstack-clash-protection` in it's upcoming [11.0.0 release](https://clang.llvm.org/docs/ReleaseNotes.html#id6). ACKs for top commit: jamesob: ACK b536813 ([`jamesob/ackr/18921.1.fanquake.build_add_stack_clash_an`](https://github.com/jamesob/bitcoin/tree/ackr/18921.1.fanquake.build_add_stack_clash_an)) laanwj: Code review ACK b536813 Tree-SHA512: abc9adf23cdf1be384f5fb9aa5bfffdda86b9ecd671064298d4cda0440828b509f070f9b19c88c7ce50ead9ff32afff9f14c5e78d75f01241568fbfa077be0b7
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Beginning with Ubuntu
19.10, it's packaged GCC now has some additional hardening options enabled by default (in addition to existing defaults like-fstack-protector-strongand reducing the minimum ssp buffer size). The new additions are-fcf-protection=fulland-fstack-clash-protection.If your interested you can grab
gcc-9_9.3.0-10ubuntu2.debian.tar.xzfrom https://packages.ubuntu.com/focal/g++-9. The relevant changes are part of thegcc-distro-specspatches, along with the relevant additions to the gcc manages:So, if you're C++ using GCC on Ubuntu 19.10 or later, these options will be active unless you explicitly opt out. This can be observed with a small test:
Note the insertion of an
endbr64instruction when compiling and not opting out. This instruction is part of the Intel Control-flow Enforcement Technology spec, which the GCC control flow implementation is based on.If we're still doing gitian builds for the
0.21.0and0.22.0releases, we'd likely update the gitian image to Ubuntu Focal, which would mean that the GCC used for gitian builds would also be using these options by default. So we should decide whether we want to explicitly turn these options on as part of our hardening options (although not just for this reason), or, we should be opting-out.GCC has supported both options since 8.0.0. Clang has supported
-fcf-protectionfrom 7.0.0 and will support-fstack-clash-protectionin it's upcoming 11.0.0 release.