-
Notifications
You must be signed in to change notification settings - Fork 37.7k
build: fix unoptimized libraries in depends #22840
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
We need to append-to rather than set CXXFLAGS, otherwise we loose -O2 & -pipe. Currently this results in zeromq being built without optimizations at all (or whatever the compiler would default too, essentially always -O0). Bdb is the same, for the CXX portion of its code. C code has been built with -O2. Boost has actually been uneffected because it receives -O3 from it's own build flags.
Guix builds: 4e2d5ef3213127ffa65a18c96fa081ce6904acad338a1e65af3bd6923ca01717 guix-build-76f031b050a4/output/aarch64-linux-gnu/SHA256SUMS.part
7a0c9a153b48004802b326f4c3dc4b756d20c3aa406312e4270edcd1bfd0b1c9 guix-build-76f031b050a4/output/aarch64-linux-gnu/bitcoin-76f031b050a4-aarch64-linux-gnu-debug.tar.gz
bfc951c3428380cf04b80a2d922b60ae65f5b613cff58ca97849e430a3f10157 guix-build-76f031b050a4/output/aarch64-linux-gnu/bitcoin-76f031b050a4-aarch64-linux-gnu.tar.gz
bbbc78b77567306e5e4c77c87282f87a83f00fa8564c850995b7c9dabbe1fbc2 guix-build-76f031b050a4/output/arm-linux-gnueabihf/SHA256SUMS.part
1a46ab64e62d4930df6be17c7756dfeb0cc86a2a34396fa2c705606cf3fa85e6 guix-build-76f031b050a4/output/arm-linux-gnueabihf/bitcoin-76f031b050a4-arm-linux-gnueabihf-debug.tar.gz
910a1e25524c552e25ad3a3ea4d4bfa900c9bdfca749cf45637d61aae9267dd4 guix-build-76f031b050a4/output/arm-linux-gnueabihf/bitcoin-76f031b050a4-arm-linux-gnueabihf.tar.gz
b52343d96909560f3060d111d68ac0907e30d79925de8c685f96267fb6243848 guix-build-76f031b050a4/output/dist-archive/bitcoin-76f031b050a4.tar.gz
7596dd80249c2d27e5e3bae115fe064f3fbcc81d0d25b71dbe11674ed9f68aa4 guix-build-76f031b050a4/output/powerpc64-linux-gnu/SHA256SUMS.part
f62c4e5b06590c5e749c305ff2f1b3d287cf8adcea16a4f3a9183f5889543fe0 guix-build-76f031b050a4/output/powerpc64-linux-gnu/bitcoin-76f031b050a4-powerpc64-linux-gnu-debug.tar.gz
879447958aeb5c25e474cc063aa843af1ca4970599659ea2ffeedf303f4f9c0b guix-build-76f031b050a4/output/powerpc64-linux-gnu/bitcoin-76f031b050a4-powerpc64-linux-gnu.tar.gz
b90d31cd9fe889cb71581a74b4c5de07b72ae114f9b7f329796bf50bd82648de guix-build-76f031b050a4/output/powerpc64le-linux-gnu/SHA256SUMS.part
882019d801776b128ee13e6f8fcbcb76473e0d931314aaf4be2f4d57fdbe686b guix-build-76f031b050a4/output/powerpc64le-linux-gnu/bitcoin-76f031b050a4-powerpc64le-linux-gnu-debug.tar.gz
581d38d938d79ba0e3781bf83b567c0f49c2ed4fa7fa89f8eecfec231e6d9b16 guix-build-76f031b050a4/output/powerpc64le-linux-gnu/bitcoin-76f031b050a4-powerpc64le-linux-gnu.tar.gz
419291c599f7b4213c6fa9382b853e60cc2220d5414335ff92a516600fc45a06 guix-build-76f031b050a4/output/riscv64-linux-gnu/SHA256SUMS.part
148e6947c7bee695b9c57987818897ca98d169b8a14779ca170301468a0abe4b guix-build-76f031b050a4/output/riscv64-linux-gnu/bitcoin-76f031b050a4-riscv64-linux-gnu-debug.tar.gz
04d50318bea6af3cc73d7af8ddd91ebc8335046ce237fd0f7b23937ccc1f2ec6 guix-build-76f031b050a4/output/riscv64-linux-gnu/bitcoin-76f031b050a4-riscv64-linux-gnu.tar.gz
b75aa2133bf4c031474cbd716025a51416e6d2efe8bf8d8f5fed9a7c65918e43 guix-build-76f031b050a4/output/x86_64-apple-darwin18/SHA256SUMS.part
7a09ff585858a350153b0998186e58feb7439edb0804e18f6e54fac9f220d43b guix-build-76f031b050a4/output/x86_64-apple-darwin18/bitcoin-76f031b050a4-osx-unsigned.dmg
d22259825baaa07674f8471132c4f79a8328d31a1c12ae07fb0ba0d272ea9648 guix-build-76f031b050a4/output/x86_64-apple-darwin18/bitcoin-76f031b050a4-osx-unsigned.tar.gz
7121e7f426269f3a5807cf14e357eed970fbe9c39ebcfcf01f9fc89589f7cdba guix-build-76f031b050a4/output/x86_64-apple-darwin18/bitcoin-76f031b050a4-osx64.tar.gz
74d0673f95ac924c02e4f18694ee456c49cc3fec016a895f14b87d3ecd792951 guix-build-76f031b050a4/output/x86_64-linux-gnu/SHA256SUMS.part
e1d020b0db00815dba278be0b3bd52d99202c8ad54437380c71121e9a4c772a8 guix-build-76f031b050a4/output/x86_64-linux-gnu/bitcoin-76f031b050a4-x86_64-linux-gnu-debug.tar.gz
78f81196c3dfd5ea030abe505daa02e84f660c159b7d701c2606474f4725e046 guix-build-76f031b050a4/output/x86_64-linux-gnu/bitcoin-76f031b050a4-x86_64-linux-gnu.tar.gz
2fc42b5dc3aafe68980123c1f74958120ae9f2988aae11e5586c871a02d12498 guix-build-76f031b050a4/output/x86_64-w64-mingw32/SHA256SUMS.part
ea1207c00963937eab4f85495c3a16cf648f2da0df659791cf181ed688bee344 guix-build-76f031b050a4/output/x86_64-w64-mingw32/bitcoin-76f031b050a4-win-unsigned.tar.gz
fa31a979e4bc2a9c17ce03c1971f491ec8704ffb8e71a4757b80ce704ef33fe1 guix-build-76f031b050a4/output/x86_64-w64-mingw32/bitcoin-76f031b050a4-win64-debug.zip
f4b205e54894baba976a89b5fe6264f238ee532fbcda41daac06887a85cc2c0d guix-build-76f031b050a4/output/x86_64-w64-mingw32/bitcoin-76f031b050a4-win64-setup-unsigned.exe
30e7bdac48084b696ab6fabd5c5212b3fde2ce0c71983d4983e44da252b0508a guix-build-76f031b050a4/output/x86_64-w64-mingw32/bitcoin-76f031b050a4-win64.zip |
Good catch! Any idea if Qt is affected? I guess not, as it has its own build system like boost. But it would have the most performance consequences of all our dependencies. |
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. ConflictsReviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first. |
Guix builds:
|
Sigh, sorry about that :( Looks like it's been this way for ages. Great catch indeed! ACK 76f031b. |
Yes Qt is also unaffected, as we inject our flags into it's build. |
76f031b build: fix unoptimized libraries in depends (fanquake) Pull request description: We need to append-to rather than set CXXFLAGS, otherwise we loose `-O2` & `-pipe` from our defaults. Currently this results in zeromq being built without optimizations at all (or whatever the compiler would default too, essentially always `-O0`). Bdb is the same, for the CXX portion of its code. C code has been built with `-O2`. Boost has actually been unaffected because it receives `-O3` from it's own build flags. Noticed while reworking bitcoin#22380. For bdb & zeromq, I assume (haven't checked) this has been the case since bitcoin#7165. You can inspect the effcts in bitcoind comparing a function from a unoptimised library, i.e libzmq. Build bitcoind with zeromq from depends (7be143a): ``` gmake -C depends NO_QT=1 NO_UPNP=1 NO_NATPMP=1 NO_WALLET=1 -j9 ./autogen.sh CONFIG_SITE=/path/to/share/config.site ./configure gmake -C src bitcoind -j9 ``` Find a zeromq function: ```bash nm -C src/bitcoind | rg zmq ... 000000010053a7e0 T _zmq_ctx_new ``` Disassemble it: ```bash lldb src/bitcoind disassemble -a 000000010053a7e0 ... bitcoind`zmq_ctx_new: bitcoind[0x10053a7e0] <+0>: pushq %rbp bitcoind[0x10053a7e1] <+1>: movq %rsp, %rbp bitcoind[0x10053a7e4] <+4>: subq $0x50, %rsp bitcoind[0x10053a7e8] <+8>: callq 0x1004b2ee0 ; zmq::initialize_network() bitcoind[0x10053a7ed] <+13>: testb $0x1, %al bitcoind[0x10053a7ef] <+15>: jne 0x10053a802 ; <+34> bitcoind[0x10053a7f5] <+21>: movq $0x0, -0x8(%rbp) bitcoind[0x10053a7fd] <+29>: jmp 0x10053a8e8 ; <+264> bitcoind[0x10053a802] <+34>: movq 0xadab7(%rip), %rsi ; (void *)0x0000000000000000 bitcoind[0x10053a809] <+41>: movl $0x2b8, %edi ; imm = 0x2B8 bitcoind[0x10053a80e] <+46>: callq 0x100556c58 ; symbol stub for: operator new(unsigned long, std::nothrow_t const&) bitcoind[0x10053a813] <+51>: xorl %ecx, %ecx bitcoind[0x10053a815] <+53>: movl %ecx, %edx bitcoind[0x10053a817] <+55>: movb $0x0, -0x19(%rbp) bitcoind[0x10053a81b] <+59>: cmpq $0x0, %rax bitcoind[0x10053a81f] <+63>: movq %rax, -0x38(%rbp) bitcoind[0x10053a823] <+67>: movq %rdx, -0x40(%rbp) bitcoind[0x10053a827] <+71>: je 0x10053a858 ; <+120> bitcoind[0x10053a82d] <+77>: movq -0x38(%rbp), %rax bitcoind[0x10053a831] <+81>: movq %rax, -0x18(%rbp) bitcoind[0x10053a835] <+85>: movb $0x1, -0x19(%rbp) bitcoind[0x10053a839] <+89>: movq -0x38(%rbp), %rdi bitcoind[0x10053a83d] <+93>: movq %rax, -0x48(%rbp) bitcoind[0x10053a841] <+97>: callq 0x100497aa0 ; zmq::ctx_t::ctx_t() bitcoind[0x10053a846] <+102>: jmp 0x10053a84b ; <+107> bitcoind[0x10053a84b] <+107>: movq -0x48(%rbp), %rax bitcoind[0x10053a84f] <+111>: movq %rax, -0x40(%rbp) bitcoind[0x10053a853] <+115>: jmp 0x10053a858 ; <+120> bitcoind[0x10053a858] <+120>: movq -0x40(%rbp), %rax bitcoind[0x10053a85c] <+124>: movq %rax, -0x10(%rbp) bitcoind[0x10053a860] <+128>: cmpq $0x0, -0x10(%rbp) bitcoind[0x10053a865] <+133>: je 0x10053a8e0 ; <+256> bitcoind[0x10053a86b] <+139>: movq -0x10(%rbp), %rdi bitcoind[0x10053a86f] <+143>: callq 0x100497ee0 ; zmq::ctx_t::valid() const bitcoind[0x10053a874] <+148>: testb $0x1, %al bitcoind[0x10053a876] <+150>: jne 0x10053a8db ; <+251> bitcoind[0x10053a87c] <+156>: movq -0x10(%rbp), %rax bitcoind[0x10053a880] <+160>: cmpq $0x0, %rax bitcoind[0x10053a884] <+164>: movq %rax, -0x50(%rbp) bitcoind[0x10053a888] <+168>: je 0x10053a8a3 ; <+195> bitcoind[0x10053a88e] <+174>: movq -0x50(%rbp), %rdi bitcoind[0x10053a892] <+178>: callq 0x100497ec0 ; zmq::ctx_t::~ctx_t() bitcoind[0x10053a897] <+183>: movq -0x50(%rbp), %rax bitcoind[0x10053a89b] <+187>: movq %rax, %rdi bitcoind[0x10053a89e] <+190>: callq 0x100556c3a ; symbol stub for: operator delete(void*) bitcoind[0x10053a8a3] <+195>: movq $0x0, -0x8(%rbp) bitcoind[0x10053a8ab] <+203>: jmp 0x10053a8e8 ; <+264> bitcoind[0x10053a8b0] <+208>: movq %rax, -0x28(%rbp) bitcoind[0x10053a8b4] <+212>: movl %edx, -0x2c(%rbp) bitcoind[0x10053a8b7] <+215>: testb $0x1, -0x19(%rbp) bitcoind[0x10053a8bb] <+219>: jne 0x10053a8c6 ; <+230> bitcoind[0x10053a8c1] <+225>: jmp 0x10053a8d6 ; <+246> bitcoind[0x10053a8c6] <+230>: movq 0xad9f3(%rip), %rsi ; (void *)0x0000000000000000 bitcoind[0x10053a8cd] <+237>: movq -0x18(%rbp), %rdi bitcoind[0x10053a8d1] <+241>: callq 0x100556c40 ; symbol stub for: operator delete(void*, std::nothrow_t const&) bitcoind[0x10053a8d6] <+246>: jmp 0x10053a8f2 ; <+274> bitcoind[0x10053a8db] <+251>: jmp 0x10053a8e0 ; <+256> bitcoind[0x10053a8e0] <+256>: movq -0x10(%rbp), %rax bitcoind[0x10053a8e4] <+260>: movq %rax, -0x8(%rbp) bitcoind[0x10053a8e8] <+264>: movq -0x8(%rbp), %rax bitcoind[0x10053a8ec] <+268>: addq $0x50, %rsp bitcoind[0x10053a8f0] <+272>: popq %rbp bitcoind[0x10053a8f1] <+273>: retq bitcoind[0x10053a8f2] <+274>: movq -0x28(%rbp), %rdi bitcoind[0x10053a8f6] <+278>: callq 0x100556856 ; symbol stub for: _Unwind_Resume bitcoind[0x10053a8fb] <+283>: ud2 bitcoind[0x10053a8fd] <+285>: nopl (%rax) ``` Cleanup and repeat after changing the zeromq cxxflags to be `$(package)_cxxflags+=-std=c++17`: ```bash gmake clean gmake -C depends NO_QT=1 NO_UPNP=1 NO_NATPMP=1 NO_WALLET=1 -j9 gmake -C src bitcoind -j9 nm -C src/bitcoind | rg zmq ... 00000001004d5170 T _zmq_ctx_new ``` Disassemble the same function which has now been built with `-O2`: ```bash lldb src/bitcoind disassemble -a 00000001004d5170 ... bitcoind`zmq_ctx_new: bitcoind[0x1004d5170] <+0>: pushq %rbp bitcoind[0x1004d5171] <+1>: movq %rsp, %rbp bitcoind[0x1004d5174] <+4>: pushq %r14 bitcoind[0x1004d5176] <+6>: pushq %rbx bitcoind[0x1004d5177] <+7>: callq 0x10049cbc0 ; zmq::initialize_network() bitcoind[0x1004d517c] <+12>: testb %al, %al bitcoind[0x1004d517e] <+14>: je 0x1004d51bd ; <+77> bitcoind[0x1004d5180] <+16>: movq 0xab139(%rip), %rsi ; (void *)0x0000000000000000 bitcoind[0x1004d5187] <+23>: movl $0x2b8, %edi ; imm = 0x2B8 bitcoind[0x1004d518c] <+28>: callq 0x1004f0e5a ; symbol stub for: operator new(unsigned long, std::nothrow_t const&) bitcoind[0x1004d5191] <+33>: testq %rax, %rax bitcoind[0x1004d5194] <+36>: je 0x1004d51bd ; <+77> bitcoind[0x1004d5196] <+38>: movq %rax, %rbx bitcoind[0x1004d5199] <+41>: movq %rax, %rdi bitcoind[0x1004d519c] <+44>: callq 0x100493400 ; zmq::ctx_t::ctx_t() bitcoind[0x1004d51a1] <+49>: movq %rbx, %rdi bitcoind[0x1004d51a4] <+52>: callq 0x1004936e0 ; zmq::ctx_t::valid() const bitcoind[0x1004d51a9] <+57>: testb %al, %al bitcoind[0x1004d51ab] <+59>: jne 0x1004d51bf ; <+79> bitcoind[0x1004d51ad] <+61>: movq %rbx, %rdi bitcoind[0x1004d51b0] <+64>: callq 0x1004936d0 ; zmq::ctx_t::~ctx_t() bitcoind[0x1004d51b5] <+69>: movq %rbx, %rdi bitcoind[0x1004d51b8] <+72>: callq 0x1004f0e42 ; symbol stub for: operator delete(void*) bitcoind[0x1004d51bd] <+77>: xorl %ebx, %ebx bitcoind[0x1004d51bf] <+79>: movq %rbx, %rax bitcoind[0x1004d51c2] <+82>: popq %rbx bitcoind[0x1004d51c3] <+83>: popq %r14 bitcoind[0x1004d51c5] <+85>: popq %rbp bitcoind[0x1004d51c6] <+86>: retq bitcoind[0x1004d51c7] <+87>: movq %rax, %r14 bitcoind[0x1004d51ca] <+90>: movq 0xab0ef(%rip), %rsi ; (void *)0x0000000000000000 bitcoind[0x1004d51d1] <+97>: movq %rbx, %rdi bitcoind[0x1004d51d4] <+100>: callq 0x1004f0e48 ; symbol stub for: operator delete(void*, std::nothrow_t const&) bitcoind[0x1004d51d9] <+105>: movq %r14, %rdi bitcoind[0x1004d51dc] <+108>: callq 0x1004f0a6a ; symbol stub for: _Unwind_Resume bitcoind[0x1004d51e1] <+113>: ud2 bitcoind[0x1004d51e3] <+115>: nopw %cs:(%rax,%rax) bitcoind[0x1004d51ed] <+125>: nopl (%rax) ``` ACKs for top commit: theuni: ACK 76f031b. Tree-SHA512: 0f71d98387d88f36bd22fd4204f8116efc6d540b45a722281483f1f19f36a26daa197458006af6a35d80a52dd8f13c714c4c816ad6c279d6e52872c948fab987
We need to append-to rather than set CXXFLAGS, otherwise we loose
-O2
&-pipe
from our defaults. Currently this results in zeromq being built without optimizations at all (or whatever the compiler would default too, essentially always-O0
).Bdb is the same, for the CXX portion of its code. C code has been built with
-O2
. Boost has actually been unaffected because it receives-O3
from it's own build flags.Noticed while reworking #22380. For bdb & zeromq, I assume (haven't checked) this has been the case since #7165.
You can inspect the effcts in bitcoind comparing a function from a unoptimised library, i.e libzmq.
Build bitcoind with zeromq from depends (7be143a):
Find a zeromq function:
nm -C src/bitcoind | rg zmq ... 000000010053a7e0 T _zmq_ctx_new
Disassemble it:
Cleanup and repeat after changing the zeromq cxxflags to be
$(package)_cxxflags+=-std=c++17
:gmake clean gmake -C depends NO_QT=1 NO_UPNP=1 NO_NATPMP=1 NO_WALLET=1 -j9 gmake -C src bitcoind -j9 nm -C src/bitcoind | rg zmq ... 00000001004d5170 T _zmq_ctx_new
Disassemble the same function which has now been built with
-O2
: