build, macos: Fix `qt` package build with new Xcode 15 linker #28543

hebasto · 2023-09-27T11:51:46Z

Fixes #28541 by backporting an upstream patch.

Guix build:

x86_64
b37713bc8a526662eac3d9535924f4a4d2893c58f9c12d3c7599e761e6ff677c  guix-build-79ef528511f0/output/arm64-apple-darwin/SHA256SUMS.part
0befb524181aa10e1635a2616a8bed53f51beafa4f0d495d3bf52a64cbd2d977  guix-build-79ef528511f0/output/arm64-apple-darwin/bitcoin-79ef528511f0-arm64-apple-darwin-unsigned.tar.gz
9cba170f2ffe542c33fdd1ac52b7684dd6301e91d32aa45af7b4ce8769d88d4a  guix-build-79ef528511f0/output/arm64-apple-darwin/bitcoin-79ef528511f0-arm64-apple-darwin-unsigned.zip
04556309266c791ae4d7409359222c88cd7aeb569566f7ef4d29816148a5b7e4  guix-build-79ef528511f0/output/arm64-apple-darwin/bitcoin-79ef528511f0-arm64-apple-darwin.tar.gz
51229df8e104a2ffcd5c5b3f81f7585e1258ef10461d136948ea2a2d690a920d  guix-build-79ef528511f0/output/dist-archive/bitcoin-79ef528511f0.tar.gz
3fe216a05561f2fe7229ddf186ff495b29a5cc31b6f35f407187573d072c5743  guix-build-79ef528511f0/output/x86_64-apple-darwin/SHA256SUMS.part
961d71104e61a2baf727576eb2da630697bb4f109f66e73be5c96add25378d12  guix-build-79ef528511f0/output/x86_64-apple-darwin/bitcoin-79ef528511f0-x86_64-apple-darwin-unsigned.tar.gz
5598f514d065756ac376e2f3c4f8e758bfba53a43ddef778f106456de1536073  guix-build-79ef528511f0/output/x86_64-apple-darwin/bitcoin-79ef528511f0-x86_64-apple-darwin-unsigned.zip
5360ae1f1b7d96a44a33b2c87708b466e4a7bf3f9de0fc58bccbbcdb21ee254e  guix-build-79ef528511f0/output/x86_64-apple-darwin/bitcoin-79ef528511f0-x86_64-apple-darwin.tar.gz

DrahtBot · 2023-09-27T11:51:48Z

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Reviews

See the guideline for information on the review process.
A summary of reviews will appear here.

Conflicts

Reviewers, this pull request conflicts with the following ones:

#28532 (qt: enable -ltcg for windows under LTO by fanquake)
#25391 (guix: Use LTO to build releases by fanquake)
#21778 (build: LLD based macOS toolchain by fanquake)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

DrahtBot · 2023-09-27T22:57:19Z

Guix builds (on x86_64)

File	commit `c9f2882` (master)	commit `37a4a10` (master and this pull)
SHA256SUMS.part	`42de1160de03cfb0...`	`4b73e5a3205dbdf6...`
*-aarch64-linux-gnu-debug.tar.gz	`71b1f5c8d9e9bfd4...`	`0aa92abd4334efeb...`
*-aarch64-linux-gnu.tar.gz	`dd653e6ecc0dc1ac...`	`cfa9a0f8a667e7eb...`
*-arm-linux-gnueabihf-debug.tar.gz	`35390cc2d589cdc0...`	`2c8d3ac542624d0a...`
*-arm-linux-gnueabihf.tar.gz	`02a7542f7e313f41...`	`a56d7bc90d4e32a4...`
*-arm64-apple-darwin-unsigned.tar.gz	`308c0e7f1def2cf0...`	`e7365125faa922b0...`
*-arm64-apple-darwin-unsigned.zip	`5a4b00190fa0cdde...`	`43a93557c41c9b33...`
*-arm64-apple-darwin.tar.gz	`63c9b9705f9a954c...`	`53f293aa868d8d0c...`
*-powerpc64-linux-gnu-debug.tar.gz	`8f6b18c067004964...`	`5f48fd6ff87c4d98...`
*-powerpc64-linux-gnu.tar.gz	`acdf08bef308c148...`	`0c01997bd3661a34...`
*-powerpc64le-linux-gnu-debug.tar.gz	`e98727cac73fb47e...`	`ec0701b87f73ce2d...`
*-powerpc64le-linux-gnu.tar.gz	`1071de351bc4c847...`	`9f133b52e52837cf...`
*-riscv64-linux-gnu-debug.tar.gz	`d86c33975d7ab5f0...`	`6410d740f972a9d4...`
*-riscv64-linux-gnu.tar.gz	`61b80a4e2b1e67ae...`	`875a6cd8f6b86eb8...`
*-x86_64-apple-darwin-unsigned.tar.gz	`c5c551d7ddf2c952...`	`f0af85213cb6b929...`
*-x86_64-apple-darwin-unsigned.zip	`ddb1cced452553b0...`	`13a2ac47b35618bb...`
*-x86_64-apple-darwin.tar.gz	`4fb40e3742708fb4...`	`bdc7b3d12defeb7b...`
*-x86_64-linux-gnu-debug.tar.gz	`f0f09c5b962632cd...`	`215ef5e9eea5e4ee...`
*-x86_64-linux-gnu.tar.gz	`504f1fdc02b601df...`	`2293aeacca58e7d8...`
*.tar.gz	`2099c1570f5ec761...`	`bf11fba7b4b9d9fd...`
guix_build.log	`60656ee77aef272c...`	`bb085752c705e787...`
guix_build.log.diff		`e0a2d91ba2b80577...`

maaku · 2023-09-27T23:54:29Z

Confirmed this fixes the issue described in #28541 for me. Thank you!

fanquake · 2023-10-02T09:13:25Z

Looks like this might still fail, if you aren't running macOS 14 (maybe aarch64 only)? I see:

compiling .moc/moc_qfbscreen_p.cpp
compiling .moc/moc_qfbvthandler_p.cpp
gmake[4]: Entering directory '/bitcoin/depends/work/build/aarch64-apple-darwin22.6.0/qt/5.15.5-8525c199952/qtbase/src/widgets'
uic dialogs/qfiledialog.ui
dyld[55501]: Symbol not found: __ZTVNSt3__13pmr25monotonic_buffer_resourceE
  Referenced from: <577DB68A-5C1A-38A3-8285-AC3E88B934FE> /bitcoin/depends/work/build/aarch64-apple-darwin22.6.0/qt/5.15.5-8525c199952/qtbase/bin/uic
  Expected in:     <3EE92404-8FC3-374B-A598-D5C9A8CD64B5> /usr/lib/libc++.1.dylib
/bin/sh: line 1: 55501 Abort trap: 6           /bitcoin/depends/work/build/aarch64-apple-darwin22.6.0/qt/5.15.5-8525c199952/qtbase/bin/uic dialogs/qfiledialog.ui -o .uic/ui_qfiledialog.h
compiling kernel/qt_widgets_pch.h
gmake[4]: *** [Makefile:72588: .uic/ui_qfiledialog.h] Error 134

This seems to be the same as https://forum.qt.io/topic/149842/qt-6-5-2-build-crash-after-update-to-macos14-sdk.

fanquake · 2023-10-02T09:50:44Z

I wonder if a better (short-term) solution here, would be to invoke the deprecated linker using -ld_classic. Although I'm not sure if this will behave with the older ld64.

hebasto · 2023-10-02T10:44:21Z

Looks like this might still fail, if you aren't running macOS 14 (maybe aarch64 only)? I see:

compiling .moc/moc_qfbscreen_p.cpp
compiling .moc/moc_qfbvthandler_p.cpp
gmake[4]: Entering directory '/bitcoin/depends/work/build/aarch64-apple-darwin22.6.0/qt/5.15.5-8525c199952/qtbase/src/widgets'
uic dialogs/qfiledialog.ui
dyld[55501]: Symbol not found: __ZTVNSt3__13pmr25monotonic_buffer_resourceE
  Referenced from: <577DB68A-5C1A-38A3-8285-AC3E88B934FE> /bitcoin/depends/work/build/aarch64-apple-darwin22.6.0/qt/5.15.5-8525c199952/qtbase/bin/uic
  Expected in:     <3EE92404-8FC3-374B-A598-D5C9A8CD64B5> /usr/lib/libc++.1.dylib
/bin/sh: line 1: 55501 Abort trap: 6           /bitcoin/depends/work/build/aarch64-apple-darwin22.6.0/qt/5.15.5-8525c199952/qtbase/bin/uic dialogs/qfiledialog.ui -o .uic/ui_qfiledialog.h
compiling kernel/qt_widgets_pch.h
gmake[4]: *** [Makefile:72588: .uic/ui_qfiledialog.h] Error 134

This seems to be the same as forum.qt.io/topic/149842/qt-6-5-2-build-crash-after-update-to-macos14-sdk.

That is another issue different from #28541 which is supposed to be fixed by this PR.

fanquake · 2023-10-02T12:54:33Z

Two other thoughts:

Is it worth bumping to qt 5.15.10, before applying (more) patches. I haven't looked if there have been other relevant bug fixes since 5.15.5.
We could also just merge this as-is, as it's fixing the build in at least some cases, and the case above is already broken on master (with the new tools), and just less broken after the patch. I'm just unsure if this could break other still-working configurations.

hebasto · 2023-10-02T15:36:49Z

Is it worth bumping to qt 5.15.10, before applying (more) patches. I haven't looked if there have been other relevant bug fixes since 5.15.5.

When I started to work on this PR, I verified whether an update to 5.15.10 allows us to drop any of our Qt patches. All of them are still required. And we have no reported Qt-related bugs that might be fixed in 5.15.10.

Suggesting to wait for Qt 5.15.16 (for Bitcoin Core v27.0 or v28.0), which will allow us to drop the patch introduced in this PR.

fanquake · 2023-10-02T15:55:54Z

Suggesting to wait for Qt 5.15.16 (for Bitcoin Core v27.0 or v28.0),

We need to bump ealier than that, otherwise Qt (and it's lack of proper support for c++20) is going to be a blocker for c++20, which we'll want to do very soon after branch off.

hebasto · 2023-10-02T19:08:20Z

When I started to work on this PR, I verified whether an update to 5.15.10 allows us to drop any of our Qt patches. All of them are still required.

I was wrong. Regardless that on https://codereview.qt-project.org/q/I70db7e4c27f0d3da5d0af33cb491d72c312d3fa8 the backport to Qt 5.15 has status "Deferred", the change from dont_hardcode_x86_64.patch has been backported and available since Qt 5.15.8.

maaku · 2023-10-02T20:09:27Z

Bitcoin Core currently does not build on a platform used by many developers, and you suggest waiting until v27 or v28?

hebasto · 2023-10-02T21:34:38Z

Bitcoin Core currently does not build on a platform used by many developers, and you suggest waiting until v27 or v28?

If you are referring to a fix of #28541, then this PR is ready for reviewing and merging.

If you are quoting my comment, please consider the full context. It was an answer to "bumping to qt 5.15.10". However, I've changed my opinion and submitted #28561.

maaku · 2023-10-02T23:05:27Z

The context was:

Is it worth bumping to qt 5.15.10, before applying (more) patches.

This PR is "applying (more) patches" so that would have meant waiting for 5.15.10.

This patch is trivial and the impact large enough to warrant merging ASAP.

maflcko · 2023-10-03T08:11:25Z

Suggesting to wait for Qt 5.15.16 (for Bitcoin Core v27.0 or v28.0), which will allow us to drop the patch introduced in this PR.

How would that work? Isn't 5.15.11 (and later) commercial-only?

fanquake · 2023-10-03T08:29:06Z

then this PR is ready for reviewing and merging

Is it? Isn't this meant to also fix this issue: #28543 (comment) ? Or is that not being fixed here anymore?

hebasto · 2023-10-03T08:46:12Z

then this PR is ready for reviewing and merging

Is it? Isn't this meant to also fix this issue: #28543 (comment) ? Or is that not being fixed here anymore?

I apologize for my poor English. By this

That is another issue different from #28541 which is supposed to be fixed by this PR.

I meant that this PR aims to fix the bug described in #28541, not in #28543 (comment).

And I have no macOS 13 at my disposal right now to work on and test the other bugfix.

fanquake · 2023-10-03T09:19:28Z

Ok. Going to merge this now then, as it's still required, regardless of #28561, and the additional build issues will have to be fixed with different patches.

We should probably also update one of the macOS jobs in the CI to be using the Xcode 15 tools. Maybe we can do that with an M1 machine now: github/roadmap#528.

maflcko · 2023-10-03T09:21:47Z

We should probably also update one of the macOS jobs in the CI to be using the Xcode 15 tools. Maybe we can do that with an M1 machine now: github/roadmap#528.

How would that work, given that the release is commercial-only? "With today’s launch, our macOS larger runners will be priced at $0.16/minute for XL and $0.12/minute for large."

fanquake · 2023-10-03T09:23:29Z

How would that work, given that the release is commercial-only?

I didn't read the blog, and assumed they would be free to use, similar to x86_64. In that case, I guess we wont use them. Just another downside to GitHub CI.

fanquake · 2023-10-03T09:32:25Z

Opened an issue to keep track of the other failure: #28566.

Github-Pull: bitcoin#28543 Rebased-From: 79ef528

fanquake · 2023-10-03T09:34:23Z

Added to #28487 for backporting to 25.x.

Github-Pull: bitcoin#28543 Rebased-From: 79ef528

fanquake · 2023-10-03T09:36:23Z

Added to #28535 for backporting to 24.x as well.

45a5fcb http: bugfix: track closed connection (stickies-v) 752a456 http: log connection instead of request count (stickies-v) ae86ada http: refactor: use encapsulated HTTPRequestTracker (stickies-v) f31899d gui: macOS, make appMenuBar part of the main app window (furszy) 64ffa94 gui: macOS, do not process dock icon actions during shutdown (furszy) e270f3f depends: fix unusable memory_resource in macos qt build (fanquake) a668394 build, macos: Fix `qt` package build with new Xcode 15 linker (Hennadii Stepanov) b3517cb test: Test loading wallets with conflicts without a chain (Andrew Chow) d63478c wallet: Check last block and conflict height are valid in MarkConflicted (Andrew Chow) 5e51a9c ci: Nuke Android APK task, Use credits for tsan (MarcoFalke) 910c362 test: ensure old fee_estimate.dat not read on restart and flushed (ismaelsadeeq) 37764d3 tx fees, policy: read stale fee estimates with a regtest-only option (ismaelsadeeq) 16bb916 tx fees, policy: do not read estimates of old fee_estimates.dat (ismaelsadeeq) c4dd598 tx fees, policy: periodically flush fee estimates to fee_estimates.dat (ismaelsadeeq) c36770c test: wallet, verify migration doesn't crash for an invalid script (furszy) 0d2a33e wallet: disallow migration of invalid or not-watched scripts (furszy) 2c51a07 Do not use std::vector = {} to release memory (Pieter Wuille) Pull request description: Further backports for the `25.x` branch. Currently: * #27622 * #27834 * #28125 * #28452 * #28542 * #28543 * #28551 * #28571 * bitcoin-core/gui#751 ACKs for top commit: hebasto: re-ACK 45a5fcb, only #28551 has been backported with since my recent [review](#28487 (review)). dergoegge: reACK 45a5fcb willcl-ark: reACK 45a5fcb Tree-SHA512: 0f5807aa364b7c2a2039fef11d5cd5e168372c3bf5b0e941350fcd92e7db4a1662801b97bb4f68e29788c77d24bbf97385a483c4501ca72d93fa25327d5694fa

9077f21 depends: fix unusable memory_resource in macos qt build (fanquake) dccacf0 build, macos: Fix `qt` package build with new Xcode 15 linker (Hennadii Stepanov) 4359649 ci: Switch to `amd64` container in "ARM" task (Hennadii Stepanov) 805f98b ci: Nuke Android APK task, Use credits for tsan (MarcoFalke) cb5512d test: ensure old fee_estimate.dat not read on restart and flushed (ismaelsadeeq) 01f8ee4 tx fees, policy: read stale fee estimates with a regtest-only option (ismaelsadeeq) 1c98029 tx fees, policy: do not read estimates of old fee_estimates.dat (ismaelsadeeq) 77979e0 tx fees, policy: periodically flush fee estimates to fee_estimates.dat (ismaelsadeeq) 67b6d99 Do not use std::vector = {} to release memory (Pieter Wuille) defdc15 ci: Use podman stop over podman kill (MarcoFalke) 7f1357d ci: Use podman for persistent workers (MarcoFalke) 0db69a3 ci: Prune dangling images on RESTART_CI_DOCKER_BEFORE_RUN (MarcoFalke) Pull request description: Backports to the 24.x branch. Currently: * #27622 * #27777 * #27834 * #27844 * #27886 * #28452 * #28543 * #28571 ACKs for top commit: stickies-v: ACK 9077f21 Tree-SHA512: abaafc9a048b67b494993134fd332457ea52695ec007b963c283f962ec40c3b6b3a7e98407481be55d3271a595088a0281cc84b79dad4f24d260381ea0153076

…ode 15 linker 79ef528 build, macos: Fix `qt` package build with new Xcode 15 linker (Hennadii Stepanov) Pull request description: Fixes bitcoin#28541 by backporting an upstream [patch](qt/qtbase@cdf64b0). Guix build: ``` x86_64 b37713bc8a526662eac3d9535924f4a4d2893c58f9c12d3c7599e761e6ff677c guix-build-79ef528511f0/output/arm64-apple-darwin/SHA256SUMS.part 0befb524181aa10e1635a2616a8bed53f51beafa4f0d495d3bf52a64cbd2d977 guix-build-79ef528511f0/output/arm64-apple-darwin/bitcoin-79ef528511f0-arm64-apple-darwin-unsigned.tar.gz 9cba170f2ffe542c33fdd1ac52b7684dd6301e91d32aa45af7b4ce8769d88d4a guix-build-79ef528511f0/output/arm64-apple-darwin/bitcoin-79ef528511f0-arm64-apple-darwin-unsigned.zip 04556309266c791ae4d7409359222c88cd7aeb569566f7ef4d29816148a5b7e4 guix-build-79ef528511f0/output/arm64-apple-darwin/bitcoin-79ef528511f0-arm64-apple-darwin.tar.gz 51229df8e104a2ffcd5c5b3f81f7585e1258ef10461d136948ea2a2d690a920d guix-build-79ef528511f0/output/dist-archive/bitcoin-79ef528511f0.tar.gz 3fe216a05561f2fe7229ddf186ff495b29a5cc31b6f35f407187573d072c5743 guix-build-79ef528511f0/output/x86_64-apple-darwin/SHA256SUMS.part 961d71104e61a2baf727576eb2da630697bb4f109f66e73be5c96add25378d12 guix-build-79ef528511f0/output/x86_64-apple-darwin/bitcoin-79ef528511f0-x86_64-apple-darwin-unsigned.tar.gz 5598f514d065756ac376e2f3c4f8e758bfba53a43ddef778f106456de1536073 guix-build-79ef528511f0/output/x86_64-apple-darwin/bitcoin-79ef528511f0-x86_64-apple-darwin-unsigned.zip 5360ae1f1b7d96a44a33b2c87708b466e4a7bf3f9de0fc58bccbbcdb21ee254e guix-build-79ef528511f0/output/x86_64-apple-darwin/bitcoin-79ef528511f0-x86_64-apple-darwin.tar.gz ``` Top commit has no ACKs. Tree-SHA512: e3a0f7a578b30a216cc84c8ac6a0eeac3f59b02525e1eb5a9f5512bc9a049a1b17d3feb140259ffe5d2197279c74594126b85112aa596df9013f74bb1047c298

Summary: Backport of [[bitcoin/bitcoin#28561 | core#28561]], [[bitcoin/bitcoin#28571 | core#28571]] and [[bitcoin/bitcoin#28543 | core#28543]]. Test Plan: Run the GUIX builds. Reviewers: #bitcoin_abc, PiRK Reviewed By: #bitcoin_abc, PiRK Differential Revision: https://reviews.bitcoinabc.org/D17204

build, macos: Fix qt package build with new Xcode 15 linker

79ef528

hebasto added macOS Build system labels Sep 27, 2023

hebasto mentioned this pull request Sep 27, 2023

[depends] qt: "Project ERROR: failed to parse default search paths from compiler output" on macOS 14.0 #28541

Closed

1 task

maflcko added the DrahtBot Guix build requested label Sep 27, 2023

DrahtBot mentioned this pull request Sep 27, 2023

qt: enable -ltcg for windows under LTO #28532

Merged

fanquake added the Needs backport (25.x) label Sep 27, 2023

This was referenced Sep 27, 2023

guix: Use LTO to build releases #25391

Closed

build: LLD based macOS toolchain #21778

Merged

DrahtBot removed the DrahtBot Guix build requested label Sep 27, 2023

0xETHengineer approved these changes Sep 28, 2023

View reviewed changes

fanquake mentioned this pull request Oct 2, 2023

build: Require C++20 compiler #28349

Merged

fanquake merged commit 8113ac0 into bitcoin:master Oct 3, 2023

fanquake mentioned this pull request Oct 3, 2023

build: depends qt failure with macos 13 & Xcode (tools) 15 #28566

Closed

fanquake pushed a commit to fanquake/bitcoin that referenced this pull request Oct 3, 2023

build, macos: Fix qt package build with new Xcode 15 linker

a668394

Github-Pull: bitcoin#28543 Rebased-From: 79ef528

fanquake removed the Needs backport (25.x) label Oct 3, 2023

This was referenced Oct 3, 2023

build: Update qt package up to 5.15.10 #28561

Merged

[25.1] Final backports #28487

Merged

fanquake pushed a commit to fanquake/bitcoin that referenced this pull request Oct 3, 2023

build, macos: Fix qt package build with new Xcode 15 linker

dccacf0

Github-Pull: bitcoin#28543 Rebased-From: 79ef528

fanquake mentioned this pull request Oct 3, 2023

[24.x] Further backports #28535

Merged

hebasto deleted the 230927-qt-sonoma branch October 3, 2023 09:43

bitcoin locked and limited conversation to collaborators Dec 3, 2024

build, macos: Fix qt package build with new Xcode 15 linker #28543

build, macos: Fix qt package build with new Xcode 15 linker #28543

Uh oh!

Conversation

hebasto commented Sep 27, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DrahtBot commented Sep 27, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reviews

Conflicts

Uh oh!

DrahtBot commented Sep 27, 2023

Guix builds (on x86_64)

Uh oh!

maaku commented Sep 27, 2023

Uh oh!

fanquake commented Oct 2, 2023

Uh oh!

fanquake commented Oct 2, 2023

Uh oh!

hebasto commented Oct 2, 2023

Uh oh!

fanquake commented Oct 2, 2023

Uh oh!

hebasto commented Oct 2, 2023

Uh oh!

fanquake commented Oct 2, 2023

Uh oh!

hebasto commented Oct 2, 2023

Uh oh!

maaku commented Oct 2, 2023

Uh oh!

hebasto commented Oct 2, 2023

Uh oh!

maaku commented Oct 2, 2023

Uh oh!

maflcko commented Oct 3, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

fanquake commented Oct 3, 2023

Uh oh!

hebasto commented Oct 3, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

fanquake commented Oct 3, 2023

Uh oh!

maflcko commented Oct 3, 2023

Uh oh!

fanquake commented Oct 3, 2023

Uh oh!

fanquake commented Oct 3, 2023

Uh oh!

fanquake commented Oct 3, 2023

Uh oh!

fanquake commented Oct 3, 2023

Uh oh!

Uh oh!

build, macos: Fix `qt` package build with new Xcode 15 linker #28543

build, macos: Fix `qt` package build with new Xcode 15 linker #28543

hebasto commented Sep 27, 2023 •

edited

Loading

DrahtBot commented Sep 27, 2023 •

edited

Loading

maflcko commented Oct 3, 2023 •

edited

Loading

hebasto commented Oct 3, 2023 •

edited

Loading