depends: fix osx build with clang 16 #27328

theuni · 2023-03-24T20:48:50Z

Current build (using forced system clang as a test) results in:

error: unknown argument: '-internal-externc-isystem/opt/clang+llvm-16.0.0-x86_64-linux-gnu-ubuntu-18.04/lib/clang/16/include'

For some reason the previous syntax worked with clang 15 and below, but clang 16 requires that the option and value are properly separated.

See here for an example of upstream using this syntax.

There is no change in behavior for previous versions.

I'm seeing an additional unrelated problem with linking with system clang, but I'll PR the solution to that separately as it's not as straightforward as this.

For some reason the previous syntax worked with clang 15 and below, but clang 16 requires that the option and value are properly separated.

DrahtBot · 2023-03-24T20:48:54Z

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Reviews

See the guideline for information on the review process.

Type	Reviewers
ACK	TheCharlatan, hebasto

If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update.

hebasto · 2023-03-24T20:50:31Z

~~#27314?~~ Not related. Sorry.

theuni · 2023-03-25T00:47:48Z

#27314?

I'm not sure I see the connection?

hebasto · 2023-03-25T10:50:53Z

Current build (using forced system clang as a test) results in:

error: unknown argument: '-internal-externc-isystem/opt/clang+llvm-16.0.0-x86_64-linux-gnu-ubuntu-18.04/lib/clang/16/include'

~~Mind providing steps to reproduce it, including your build environment details?~~

UPD. On Ubuntu 23.04, I've got:

$ clang --version
Ubuntu clang version 16.0.0 (1~exp1ubuntu2)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/lib/llvm-16/bin
$ cd depends
$ make libevent HOST=x86_64-apple-darwin FORCE_USE_SYSTEM_CLANG=1
Configuring libevent...
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for x86_64-apple-darwin-strip... x86_64-apple-darwin-strip
checking for a thread-safe mkdir -p... /usr/bin/mkdir -p
checking for gawk... no
checking for mawk... mawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking whether make supports nested variables... (cached) yes
checking whether make supports the include directive... yes (GNU style)
checking for x86_64-apple-darwin-gcc... env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /usr/lib/llvm-16/bin/clang --target=x86_64-apple-darwin -mmacosx-version-min=10.15 -B/home/hebasto/bitcoin/depends/x86_64-apple-darwin/native/bin -mlinker-version=609 -isysroot/home/hebasto/bitcoin/depends/SDKs/Xcode-12.2-12B45b-extracted-SDK-with-libcxx-headers -Xclang -internal-externc-isystem/usr/lib/llvm-16/lib/clang/16/include -Xclang -internal-externc-isystem/home/hebasto/bitcoin/depends/SDKs/Xcode-12.2-12B45b-extracted-SDK-with-libcxx-headers/usr/include
checking whether the C compiler works... no
configure: error: in `/home/hebasto/bitcoin/depends/work/build/x86_64-apple-darwin/libevent/2.1.12-stable-e8d8ba5ae39':
configure: error: C compiler cannot create executables
See `config.log' for more details
make: *** [funcs.mk:292: /home/hebasto/bitcoin/depends/x86_64-apple-darwin/.libevent_stamp_configured] Error 77

hebasto

Tested 87afcb0 on Ubuntu 23.04.

It fixes building make -C depends HOST=x86_64-apple-darwin FORCE_USE_SYSTEM_CLANG=1 with clang-16.

hebasto · 2023-03-25T14:44:59Z

I'm seeing an additional unrelated problem with linking with system clang, but I'll PR the solution to that separately as it's not as straightforward as this.

FWIW, with the following diff:

--- a/depends/hosts/darwin.mk
+++ b/depends/hosts/darwin.mk
@@ -38,7 +38,9 @@ clangxx_prog=$(shell $(SHELL) $(.SHELLFLAGS) "command -v clang++")
 llvm_config_prog=$(shell $(SHELL) $(.SHELLFLAGS) "command -v llvm-config")
 
 clang_resource_dir=$(shell clang -print-resource-dir)
+llvm_bin_dir=$(shell $(llvm_config_prog) --bindir)
 llvm_lib_dir=$(shell $(llvm_config_prog) --libdir)
+darwin_LDFLAGS += -fuse-ld=$(llvm_bin_dir)/ld64.lld -Wl,-platform_version,macos,10.15,12.2
 endif
 
 cctools_TOOLS=AR RANLIB STRIP NM LIBTOOL OTOOL INSTALL_NAME_TOOL DSYMUTIL

I'm able to build as follows:

$ make -C depends HOST=x86_64-apple-darwin FORCE_USE_SYSTEM_CLANG=1 NO_QT=1 NO_WALLET=1 NO_ZMQ=1 NO_UPNP=1 NO_NATPMP=1
$ ./configure CONFIG_SITE=$PWD/depends/x86_64-apple-darwin/share/config.site --without-libs
$ make

The lld package is required to be installed.

theuni · 2023-03-26T20:27:04Z

It fixes building make -C depends HOST=x86_64-apple-darwin FORCE_USE_SYSTEM_CLANG=1 with clang-16.

Yes, exactly this thanks.

This can be verified very simply outside of depends:
Broken:

$ /opt/clang+llvm-16.0.0-x86_64-linux-gnu-ubuntu-18.04/bin/clang -c test.c -o test.o -Xclang -internal-externc-isystem/tmp
error: unknown argument: '-internal-externc-isystem/tmp'

Working:

$ /opt/clang+llvm-16.0.0-x86_64-linux-gnu-ubuntu-18.04/bin/clang -c test.c -o test.o -Xclang -internal-externc-isystem -Xclang /tmp

I'm seeing an additional unrelated problem with linking with system clang, but I'll PR the solution to that separately as it's not as straightforward as this.

FWIW, with the following diff:

Thanks, but the linking issue is totally unrelated to this one so I'd rather not get into that here.

TheCharlatan

ACK 87afcb0

hebasto

ACK 87afcb0

My Guix builds:

0b0f331d84a16aa9b69227a0e6a59ded8087932b4131e003dbac1e15326b99f5  guix-build-87afcb0029b8/output/arm64-apple-darwin/SHA256SUMS.part
569257f2141e7c03cb2619a5b3baef81210f745d0f80c7531fc5b2e868fe7ade  guix-build-87afcb0029b8/output/arm64-apple-darwin/bitcoin-87afcb0029b8-arm64-apple-darwin-unsigned.dmg
55d61448d3eb607bb607c14ecabd0109556fc8ecb769eb63c43479a6f7ef0d86  guix-build-87afcb0029b8/output/arm64-apple-darwin/bitcoin-87afcb0029b8-arm64-apple-darwin-unsigned.tar.gz
41f68c6ceacfaa2b9d5c4c3acc698812052c8d7e9da9762994b1a22d5c42b9f1  guix-build-87afcb0029b8/output/arm64-apple-darwin/bitcoin-87afcb0029b8-arm64-apple-darwin.tar.gz
201e8ad783b0d4d5ea78ca3b331a90edd48c7b0f664daf0968fcb7614877a1a1  guix-build-87afcb0029b8/output/dist-archive/bitcoin-87afcb0029b8.tar.gz
1f9e24641bebcad9865be0c3d0b78d089add38362711d07f4071a283cf9a78ac  guix-build-87afcb0029b8/output/x86_64-apple-darwin/SHA256SUMS.part
a8a1972128f30ba2e5815c50c6303528a010b7a03f73e9443ac875bd3e5ada08  guix-build-87afcb0029b8/output/x86_64-apple-darwin/bitcoin-87afcb0029b8-x86_64-apple-darwin-unsigned.dmg
2d1bf1392cbd7a26b48f0b21f2c406746463a79ac2b27dea6296bf1ce1f69790  guix-build-87afcb0029b8/output/x86_64-apple-darwin/bitcoin-87afcb0029b8-x86_64-apple-darwin-unsigned.tar.gz
819e5c9d9f17e57ce36ed93ce0db50bb8d2a526667c8e3823b01104db24fe208  guix-build-87afcb0029b8/output/x86_64-apple-darwin/bitcoin-87afcb0029b8-x86_64-apple-darwin.tar.gz

87afcb0 depends: fix osx build with clang 16 (Cory Fields) Pull request description: Current build (using forced system clang as a test) results in: > error: unknown argument: '-internal-externc-isystem/opt/clang+llvm-16.0.0-x86_64-linux-gnu-ubuntu-18.04/lib/clang/16/include' For some reason the previous syntax worked with clang 15 and below, but clang 16 requires that the option and value are properly separated. See [here for an example of upstream using this syntax](https://github.com/llvm/llvm-project/blob/main/clang/test/Driver/crash-report-with-asserts.c#L9). There is no change in behavior for previous versions. I'm seeing an additional unrelated problem with linking with system clang, but I'll PR the solution to that separately as it's not as straightforward as this. ACKs for top commit: TheCharlatan: ACK 87afcb0 hebasto: ACK 87afcb0 Tree-SHA512: 127037c888c37c6ccd9679e96da34037cc43ccdc07915865a0a5494edb62633e83fc1bd6b1c4bb7a0322f5b59622e10090a31987f38496fb6b306488e9941594

d742be3 ci: Switch native macOS CI job to Xcode 15.0 (Hennadii Stepanov) 8decc5c build: Fix `-Xclang -internal-isystem` option (Hennadii Stepanov) Pull request description: This PR: - addresses #29165 (comment) - fixes #29174 ACKs for top commit: fanquake: ACK d742be3. The same as what was done in #27328. Tree-SHA512: 4788a0511e9fac638edab8e4f7ec62c5e08aeb07e518ab62fd53074ab3dd4eca1f62dc17c2af2b535bad12e77a7437e5c1c714cd03ce711e5d5e5c87d4620358

Summary: > depends: fix osx build with clang 16 > > For some reason the previous syntax worked with clang 15 and below, but > clang 16 requires that the option and value are properly separated. > depends: remove redundant stdlib option > > Use of -stdlib++-isystem gets rid of any system c++ header include paths and > negates the need for this option. In newer versions of clangs the combo > produces a warning. > depends: modernize clang flags > > Fixes builds with llvm >= 11 in guix by working around the problem. As a bonus, > this is much cleaner and more maintainable than what we had before. This is a backport of [[bitcoin/bitcoin#27328 | core#27328]], [[bitcoin/bitcoin#27721 | core#27721]] and [[bitcoin/bitcoin#27798 | core#27798]] The change to ci/test/00_setup_env_mac.sh in [[bitcoin/bitcoin#27798 | core#27798]] to ignore -Wunused-command-line-argument warnings seems equivalent to D3792 Test Plan: check that gitian-osx still works Reviewers: #bitcoin_abc, Fabien Reviewed By: #bitcoin_abc, Fabien Differential Revision: https://reviews.bitcoinabc.org/D15579

depends: fix osx build with clang 16

87afcb0

For some reason the previous syntax worked with clang 15 and below, but clang 16 requires that the option and value are properly separated.

DrahtBot added the Build system label Mar 24, 2023

hebasto reviewed Mar 25, 2023

View reviewed changes

TheCharlatan approved these changes Mar 26, 2023

View reviewed changes

hebasto approved these changes Mar 27, 2023

View reviewed changes

fanquake merged commit 3e835ca into bitcoin:master Mar 27, 2023

fanquake mentioned this pull request Jan 9, 2024

build: Fix -Xclang -internal-isystem option #29195

Merged

bitcoin locked and limited conversation to collaborators Mar 26, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

depends: fix osx build with clang 16 #27328

depends: fix osx build with clang 16 #27328

Uh oh!

theuni commented Mar 24, 2023

Uh oh!

DrahtBot commented Mar 24, 2023 •

edited

Loading

Uh oh!

hebasto commented Mar 24, 2023 •

edited

Loading

Uh oh!

theuni commented Mar 25, 2023

Uh oh!

hebasto commented Mar 25, 2023 •

edited

Loading

Uh oh!

hebasto left a comment

Uh oh!

hebasto commented Mar 25, 2023

Uh oh!

theuni commented Mar 26, 2023

Uh oh!

TheCharlatan left a comment

Uh oh!

hebasto left a comment

Uh oh!

Uh oh!

depends: fix osx build with clang 16 #27328

depends: fix osx build with clang 16 #27328

Uh oh!

Conversation

theuni commented Mar 24, 2023

Uh oh!

DrahtBot commented Mar 24, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reviews

Uh oh!

hebasto commented Mar 24, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

theuni commented Mar 25, 2023

Uh oh!

hebasto commented Mar 25, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hebasto left a comment

Choose a reason for hiding this comment

Uh oh!

hebasto commented Mar 25, 2023

Uh oh!

theuni commented Mar 26, 2023

Uh oh!

TheCharlatan left a comment

Choose a reason for hiding this comment

Uh oh!

hebasto left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

DrahtBot commented Mar 24, 2023 •

edited

Loading

hebasto commented Mar 24, 2023 •

edited

Loading

hebasto commented Mar 25, 2023 •

edited

Loading