Skip to content

Conversation

laanwj
Copy link
Member

@laanwj laanwj commented Jan 19, 2021

Rebase of #14066 by luke-jr.

Let's try to get PowerPC support in in the beginning of the 22.0 cycle so that it gets some testing, and is not a last-minute decision this time, like for last … 2 or 3 major versions.

The symbol/security tooling-related changes have been dropped since they were part of #20434.

@DrahtBot
Copy link
Contributor

DrahtBot commented Jan 19, 2021

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

No conflicts as of last run.

@laanwj
Copy link
Member Author

laanwj commented Jan 19, 2021

@dongcarl I guess a parallel set of changes to the guix build would be necessary ?

@dongcarl
Copy link
Contributor

I believe so, will look into it!

@laanwj laanwj requested review from theuni and dongcarl January 20, 2021 15:05
@DrahtBot
Copy link
Contributor

Gitian builds

File commit 80486e7
(master)
commit ee03c94
(master and this pull)
*-aarch64-linux-gnu-debug.tar.gz c465d0562d4f4b36... e8aabfb614ab25e0...
*-aarch64-linux-gnu.tar.gz a6641c173362914a... 2223b2b7c531a1d3...
*-arm-linux-gnueabihf-debug.tar.gz ba68555a607871cc... ca7d3b96eb37b4bb...
*-arm-linux-gnueabihf.tar.gz 4ce722a1269a9d2d... ca65fc8dc9d6b949...
*-osx-unsigned.dmg bdccbb7e72b4e99b... 1b2078a7ffee5aa7...
*-osx64.tar.gz 4472625dadeb2508... 58fa19de2989b673...
*-riscv64-linux-gnu-debug.tar.gz 04924da7a5c56db6... ae61fdde150b6161...
*-riscv64-linux-gnu.tar.gz 459b64c697c4420f... dad7214844fe8295...
*-win64-debug.zip 7285ea82230fcfaf... c8d0ce1114d72689...
*-win64-setup-unsigned.exe 0119bbc9b21c059f... 0f7e626e6be14754...
*-win64.zip 74a588c5b8b08fec... f2433f22292a412f...
*-x86_64-linux-gnu-debug.tar.gz a4857fbfaffbd5e4... c6d4f855475eaf90...
*-x86_64-linux-gnu.tar.gz 327cc993115833a3... 4ab6aba49f03c4c8...
*.tar.gz 48c6d93a410e9c2f... fd7a18d12847e62d...
bitcoin-core-linux-22-res.yml be9f74e2ed2432dc... 45b04bb5e2790748...
bitcoin-core-osx-22-res.yml be215d7dd97ca6a8... 8c445dd70e3f9e3a...
bitcoin-core-win-22-res.yml 874912f46b5cbe79... 5f881bb5fc7d026c...
linux-build.log 69e53fde951b7629... 1b595f50c366683f...
osx-build.log 9f75df11b2c9c875... a59c78f25753082f...
win-build.log 85d5b9e92561b947... a68b54187fe173e8...
bitcoin-core-linux-22-res.yml.diff 064f01f2fc8d8c00...
bitcoin-core-osx-22-res.yml.diff 1ee4f4ae99ccf9d8...
bitcoin-core-win-22-res.yml.diff dbc1cf0563700ae8...
linux-build.log.diff 400b943562af2541...
osx-build.log.diff b9a28df5a89863e4...
win-build.log.diff 06b194d3fbc8b40e...

@maflcko
Copy link
Member

maflcko commented Jan 21, 2021

This is missing b74c21f ...?

@laanwj
Copy link
Member Author

laanwj commented Jan 24, 2021

This is missing b74c21f ...?

While rebasing it looked to me that that was only a change to the security script. But it seems not. Will add the gitian yml change.

Edit: Done, and removed 25cd539 "gitian: Always specify noexecstack since we enforce it unconditionally", replacing it with a commit that extends the workaround for the powerpc architectures.

@@ -813,6 +813,11 @@ AX_GCC_FUNC_ATTRIBUTE([dllimport])
if test x$use_glibc_compat != xno; then
AX_CHECK_LINK_FLAG([[-Wl,--wrap=__divmoddi4]], [COMPAT_LDFLAGS="$COMPAT_LDFLAGS -Wl,--wrap=__divmoddi4"])
AX_CHECK_LINK_FLAG([[-Wl,--wrap=log2f]], [COMPAT_LDFLAGS="$COMPAT_LDFLAGS -Wl,--wrap=log2f"])
case $host in
powerpc64* | ppc64*)
AX_CHECK_LINK_FLAG([[-Wl,--no-tls-get-addr-optimize]], [COMPAT_LDFLAGS="$COMPAT_LDFLAGS -Wl,--no-tls-get-addr-optimize"])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this disable some optimization related to thread local storage? Is there a way to avoid that?

(just wondering why this is here; not an objection if it's needed)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know.
@luke-jr ?
If this is a workaround for a platform-specific compiler issue, let's add a comment (preferably mentioning the upstream issue) so that it can be removed when no longer necessary.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Older glibc don't have support for the optimisation (hence it being part of glibc compat ;) )

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@luke-jr Yes, of course. I guess my question is: do we know whether that has any performance impact (presumably not much for now, as thread_local variables are only used for thread names AFAIK), and/or has the alternative of requiring a higher minimum glibc been considered?

No problem if the answer is we don't know, or there are reasons for doing it this way. I just want to make sure we don't end up years from now chasing a weird performance regression on POWER9 in release binaries.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW in #18851 I had implemented the thread name system without use of TLS, which would have dropped that entire dependency.

That said, I doubt for our use (one query per log message?) a slightly slower TLS makes any noticeable difference in performance.

@laanwj
Copy link
Member Author

laanwj commented Jan 27, 2021

Also, if we're looking into command-line options for optimizing further the first thing would be to consider using -mcpu=power9 instead of power8/970 respectively. I strongly suspect that everyone interested in these binaries wants to run them on Talos II workstations. But this also can be changed later, it doesn't have to be decided in this PR.

@DrahtBot
Copy link
Contributor

Gitian builds

File commit 15a9df0
(master)
commit 627e297
(master and this pull)
*-aarch64-linux-gnu-debug.tar.gz 2de46bcf0f879816... 0869beff056c5552...
*-aarch64-linux-gnu.tar.gz 195f74e31b494b0b... a38a4453b683a115...
*-arm-linux-gnueabihf-debug.tar.gz 745064644885b403... 00a86433087dddfa...
*-arm-linux-gnueabihf.tar.gz fc0843f9c76fa049... deffbbd78440e700...
*-osx-unsigned.dmg d7c2b3b331c0170d... cc9e1bb1b0a551dd...
*-osx64.tar.gz b83656c71a45dafb... 6f62fdb74f503006...
*-riscv64-linux-gnu-debug.tar.gz da866a3d0511d523... 52dd96c75294c4aa...
*-riscv64-linux-gnu.tar.gz 27d3649f343e0ad6... b1419057c91ac3b0...
*-win64-debug.zip 30189a63e207f9c3... 78480c2ac739d352...
*-win64-setup-unsigned.exe 807d2d9629ec72ef... 69e225bb0bbd0f10...
*-win64.zip 706047c7a5b49d73... 796393c233b6313c...
*-x86_64-linux-gnu-debug.tar.gz 599768629238d732... cb1bec7a1831b5d1...
*-x86_64-linux-gnu.tar.gz 88f8b0625fd2e065... 76f0aa08fe46a94f...
*.tar.gz b7377a5354e934db... 111a13c87dc2dcd9...
bitcoin-core-linux-22-res.yml 0cec6dc965b5a1e5... e5365ec962e53528...
bitcoin-core-osx-22-res.yml 08cb2567f8470ee0... 2629252af46d82f8...
bitcoin-core-win-22-res.yml 9d8aa3f7e0636e6e... f5fb15efd00b4de5...
linux-build.log cd52ede60a8dbbde... 4f99d3fa044f2e80...
osx-build.log 86ea2870a70d9d2d... 2a6a78b201e8647c...
win-build.log edfffe6d6b7b25d0... 1c92a6037289c69c...
*-powerpc64-linux-gnu-debug.tar.gz ccb1fdb2c7d966b6...
*-powerpc64-linux-gnu.tar.gz 2df6d7ff9f92dc1d...
*-powerpc64le-linux-gnu-debug.tar.gz 4b2b837a7020aca9...
*-powerpc64le-linux-gnu.tar.gz 429133693e740b1b...
bitcoin-core-linux-22-res.yml.diff 8c2d77155b8c213e...
bitcoin-core-osx-22-res.yml.diff b64967946ae06812...
bitcoin-core-win-22-res.yml.diff 7fabc826b3573afa...
linux-build.log.diff 7a0bf66c7060f547...
osx-build.log.diff a3941fc2d80ee41d...
win-build.log.diff 343a6f62a2bc65ab...

@laanwj
Copy link
Member Author

laanwj commented Jan 28, 2021

Looks like the gitian build is working, nice.

@@ -118,7 +130,7 @@ script: |
# Extract the git archive into a dir for each host and build
for i in ${HOSTS}; do
export PATH=${BASEPREFIX}/${i}/native/bin:${ORIGPATH}
if [ "${i}" = "riscv64-linux-gnu" ]; then
if [ "${i}" = "riscv64-linux-gnu" ] || [ "${i}" = "powerpc64-linux-gnu" ] || [ "${i}" = "powerpc64le-linux-gnu" ]; then
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why does this remain conditional? We enforce it on all platforms; might as well make it explicit.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because that is unsafe, see rationale here: #20963 (comment)

These lines can go away completely after gcc upgrade that fixes the underlying issue. But no sooner.

Copy link
Member

@luke-jr luke-jr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utACK

@laanwj laanwj merged commit 6a726cb into bitcoin:master Jan 28, 2021
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Aug 18, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants