depends: Add --sysroot option to mac os native compile flags #18743
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Catalina SDK clang stopped automatically searching the SDK include paths when invoked without --sysroot: bitcoin#16367 (comment) Homebrew/homebrew-core#45061 This hasn't been a problem for current native depends packages because are passing their own --sysroot values, and hasn't been a problem for current host packages because they use `darwin_` commands instead of `build_darwin_` commands. But the current `build_darwin_CC` and `build_darwin_CXX` commands are still unnecessarily fragile, and incompatible with new native depends packages added in bitcoin#18677. Cory Fields <cory-nospam-@coryfields.com> suggested in bitcoin#16367 (comment) switching compiler from SDK clang to native clang (from $PATH) to avoid this problem. This is easy and makes a certain amount of sense for building native packages, as opposed to host packages. But fanquake <fanquake@gmail.com> pointed out in bitcoin#18677 (comment) that it would be inconsistent use switch to non-SDK compilers while still using other SDK tools like ranlib and install_name_tool. So simplest, minimal fix seems to be just adding the missing --sysroot option.
|
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. ConflictsNo conflicts as of last run. |
This was referenced Apr 23, 2020
|
Thanks for splitting this out. Will review. |
Gitian builds
|
|
ACK 1e94a2b - I think this change is ok, and I prefer it to the previous patch. Thanks for the summary in the PR description. I played around with Xcode and the CLT; I think previously I didn't fully grok the slight differences between the two. This output was produced from a test.cpp containing just CLT Clang vs CLT Clang with
--- a.txt
+++ b.txt
@@ -1,17 +1,17 @@
Apple clang version 11.0.3 (clang-1103.0.32.59)
Target: x86_64-apple-darwin19.4.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin
- "/Library/Developer/CommandLineTools/usr/bin/clang" -cc1 -triple x86_64-apple-macosx10.15.0 -Wdeprecated-objc-isa-usage -Werror=deprecated-objc-isa-usage -emit-obj -mrelax-all -disable-free -disable-llvm-verifier -discard-value-names -main-file-name test.cpp -mrelocation-model pic -pic-level 2 -mthread-model posix -mframe-pointer=all -fno-strict-return -masm-verbose -munwind-tables -target-cpu penryn -dwarf-column-info -debugger-tuning=lldb -target-linker-version 556.6 -v -coverage-notes-file /Users/michael/Desktop/clt/test.gcno -resource-dir /Library/Developer/CommandLineTools/usr/lib/clang/11.0.3 -stdlib=libc++ -internal-isystem /Library/Developer/CommandLineTools/usr/bin/../include/c++/v1 -internal-isystem /usr/include/c++/v1 -internal-isystem /usr/local/include -internal-isystem /Library/Developer/CommandLineTools/usr/lib/clang/11.0.3/include -internal-externc-isystem /usr/include -internal-externc-isystem /Library/Developer/CommandLineTools/usr/include -Wno-objc-signed-char-bool-implicit-int-conversion -Wno-extra-semi-stmt -Wno-quoted-include-in-framework-header -fdeprecated-macro -fdebug-compilation-dir /Users/michael/Desktop/clt -ferror-limit 19 -fmessage-length 131 -stack-protector 1 -fstack-check -mdarwin-stkchk-strong-link -fblocks -fencode-extended-block-signature -fregister-global-dtors-with-atexit -fobjc-runtime=macosx-10.15.0 -fcxx-exceptions -fexceptions -fmax-type-align=16 -fdiagnostics-show-option -fcolor-diagnostics -o test.o -x c++ test.cpp
+ "/Library/Developer/CommandLineTools/usr/bin/clang" -cc1 -triple x86_64-apple-macosx10.15.0 -Wdeprecated-objc-isa-usage -Werror=deprecated-objc-isa-usage -emit-obj -mrelax-all -disable-free -disable-llvm-verifier -discard-value-names -main-file-name test.cpp -mrelocation-model pic -pic-level 2 -mthread-model posix -mframe-pointer=all -fno-strict-return -masm-verbose -munwind-tables -target-sdk-version=10.15.4 -target-cpu penryn -dwarf-column-info -debugger-tuning=lldb -target-linker-version 556.6 -v -coverage-notes-file /Users/michael/Desktop/clt/test.gcno -resource-dir /Library/Developer/CommandLineTools/usr/lib/clang/11.0.3 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -stdlib=libc++ -internal-isystem /Library/Developer/CommandLineTools/usr/bin/../include/c++/v1 -internal-isystem /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1 -internal-isystem /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/local/include -internal-isystem /Library/Developer/CommandLineTools/usr/lib/clang/11.0.3/include -internal-externc-isystem /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -internal-externc-isystem /Library/Developer/CommandLineTools/usr/include -Wno-objc-signed-char-bool-implicit-int-conversion -Wno-extra-semi-stmt -Wno-quoted-include-in-framework-header -fdeprecated-macro -fdebug-compilation-dir /Users/michael/Desktop/clt -ferror-limit 19 -fmessage-length 131 -stack-protector 1 -fstack-check -mdarwin-stkchk-strong-link -fblocks -fencode-extended-block-signature -fregister-global-dtors-with-atexit -fobjc-runtime=macosx-10.15.0 -fcxx-exceptions -fexceptions -fmax-type-align=16 -fdiagnostics-show-option -fcolor-diagnostics -o test.o -x c++ test.cpp
clang -cc1 version 11.0.3 (clang-1103.0.32.59) default target x86_64-apple-darwin19.4.0
-ignoring nonexistent directory "/usr/include/c++/v1"
-ignoring nonexistent directory "/usr/include"
+ignoring nonexistent directory "/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1"
+ignoring nonexistent directory "/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/local/include"
+ignoring nonexistent directory "/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/Library/Frameworks"
#include "..." search starts here:
#include <...> search starts here:
/Library/Developer/CommandLineTools/usr/bin/../include/c++/v1
- /usr/local/include
/Library/Developer/CommandLineTools/usr/lib/clang/11.0.3/include
+ /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include
/Library/Developer/CommandLineTools/usr/include
- /System/Library/Frameworks (framework directory)
- /Library/Frameworks (framework directory)
+ /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks (framework directory)
End of search list.When you pass CLT Clang Invoking CLT Clang with
-InstalledDir: /Library/Developer/CommandLineTools/usr/bin
- "/Library/Developer/CommandLineTools/usr/bin/clang" -cc1 -triple x86_64-apple-macosx10.15.0 -Wdeprecated-objc-isa-usage -Werror=deprecated-objc-isa-usage -emit-obj -mrelax-all -disable-free -disable-llvm-verifier -discard-value-names -main-file-name test.cpp -mrelocation-model pic -pic-level 2 -mthread-model posix -mframe-pointer=all -fno-strict-return -masm-verbose -munwind-tables -target-sdk-version=10.15.4 -target-cpu penryn -dwarf-column-info -debugger-tuning=lldb -target-linker-version 556.6 -v -coverage-notes-file /Users/michael/Desktop/clt/test.gcno -resource-dir /Library/Developer/CommandLineTools/usr/lib/clang/11.0.3 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -stdlib=libc++ -internal-isystem /Library/Developer/CommandLineTools/usr/bin/../include/c++/v1 -internal-isystem /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1 -internal-isystem /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/local/include -internal-isystem /Library/Developer/CommandLineTools/usr/lib/clang/11.0.3/include -internal-externc-isystem /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -internal-externc-isystem /Library/Developer/CommandLineTools/usr/include -Wno-objc-signed-char-bool-implicit-int-conversion -Wno-extra-semi-stmt -Wno-quoted-include-in-framework-header -fdeprecated-macro -fdebug-compilation-dir /Users/michael/Desktop/clt -ferror-limit 19 -fmessage-length 131 -stack-protector 1 -fstack-check -mdarwin-stkchk-strong-link -fblocks -fencode-extended-block-signature -fregister-global-dtors-with-atexit -fobjc-runtime=macosx-10.15.0 -fcxx-exceptions -fexceptions -fmax-type-align=16 -fdiagnostics-show-option -fcolor-diagnostics -o test.o -x c++ test.cpp
+InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
+ "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang" -cc1 -triple x86_64-apple-macosx10.15.0 -Wdeprecated-objc-isa-usage -Werror=deprecated-objc-isa-usage -emit-obj -mrelax-all -disable-free -disable-llvm-verifier -discard-value-names -main-file-name test.cpp -mrelocation-model pic -pic-level 2 -mthread-model posix -mframe-pointer=all -fno-strict-return -masm-verbose -munwind-tables -target-sdk-version=10.15.4 -target-cpu penryn -dwarf-column-info -debugger-tuning=lldb -target-linker-version 556.6 -v -coverage-notes-file /Users/michael/Desktop/clt/test.gcno -resource-dir /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/11.0.3 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -I/usr/local/include -stdlib=libc++ -internal-isystem /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1 -internal-isystem /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/v1 -internal-isystem /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/local/include -internal-isystem /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/11.0.3/include -internal-externc-isystem /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include -internal-externc-isystem /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include -Wno-objc-signed-char-bool-implicit-int-conversion -Wno-extra-semi-stmt -Wno-quoted-include-in-framework-header -fdeprecated-macro -fdebug-compilation-dir /Users/michael/Desktop/clt -ferror-limit 19 -fmessage-length 131 -stack-protector 1 -fstack-check -mdarwin-stkchk-strong-link -fblocks -fencode-extended-block-signature -fregister-global-dtors-with-atexit -fobjc-runtime=macosx-10.15.0 -fcxx-exceptions -fexceptions -fmax-type-align=16 -fdiagnostics-show-option -fcolor-diagnostics -o test.o -x c++ test.cpp
clang -cc1 version 11.0.3 (clang-1103.0.32.59) default target x86_64-apple-darwin19.4.0
-ignoring nonexistent directory "/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1"
-ignoring nonexistent directory "/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/local/include"
-ignoring nonexistent directory "/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/Library/Frameworks"
+ignoring nonexistent directory "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/v1"
+ignoring nonexistent directory "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/local/include"
+ignoring nonexistent directory "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/Library/Frameworks"
#include "..." search starts here:
#include <...> search starts here:
- /Library/Developer/CommandLineTools/usr/bin/../include/c++/v1
- /Library/Developer/CommandLineTools/usr/lib/clang/11.0.3/include
- /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include
- /Library/Developer/CommandLineTools/usr/include
- /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks (framework directory)
+ /usr/local/include
+ /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1
+ /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/11.0.3/include
+ /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include
+ /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include
+ /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks (framework directory)
End of search list. |
|
Is |
No, they are standalone, and there are decent differences between the content of either. Xcode has a bunch more Swift modules, Apple simulator components and dylibs bundled with it. For example, you can see a diff of The macOS SDKs inside either seem to be identical. i.e: |
The current change 1e94a2b is the original fix I made for the issue before Cory's suggestion in #16367 (comment). It seems safe and good. But Cory's suggestion to just use clang and presumably ld, nm, etc directly instead of going through an SDK for native packages (which are internal build & code generation tools not distributed or used at runtime) seems it should be a nice simplification and cleanup and more durable fix. I could be wrong but I'd assume a more typical way of building non-gui unix tools on mac is: and not: My reason for going back to the direct bugfix 1e94a2b instead of my half-implementation fba09de of Cory suggestion is just that I know the direct fix works and I don't have a great way of testing other alternatives. But probably an extension of fba09de would be a better way to go in the long run. |
|
@dongcarl did you have something specific you wanted to follow up with here in regards to the symlink query? |
|
@fanquake No not really. My initial impression here is that we should remove all the |
sidhujag
pushed a commit
to syscoin/syscoin
that referenced
this pull request
May 12, 2020
…ompile flags 1e94a2b depends: Add --sysroot option to mac os native compile flags (Russell Yanofsky) Pull request description: Catalina SDK clang stopped automatically searching the SDK include paths when invoked without `--sysroot`: - bitcoin#16367 (comment) - Homebrew/homebrew-core#45061 This hasn't been a problem for current native depends packages because are passing their own `--sysroot` values, and hasn't been a problem for current host packages because they use `darwin_` commands instead of `build_darwin_` commands. But the current `build_darwin_CC` and `build_darwin_CXX` commands are still unnecessarily fragile, and incompatible with new native depends packages added in bitcoin#18677. Cory Fields (theuni) suggested in bitcoin#16367 (comment) switching compiler from SDK clang to native clang (from $PATH) to avoid this problem. This is easy and makes a certain amount of sense for building native packages, as opposed to host packages. But Michael (fanquake) pointed out in bitcoin#18677 (comment) that it would be inconsistent to switch to non-SDK compilers while still using other SDK tools like `ranlib` and `install_name_tool`. So simplest, minimal fix seems to be just adding the missing `--sysroot` option. ACKs for top commit: ryanofsky: > ACK [1e94a2b](bitcoin@1e94a2b) - I think this change is ok, and I prefer it to the previous patch. fanquake: ACK 1e94a2b - I think this change is ok, and I prefer it to the previous patch. Thanks for the summary in the PR description. I played around with Xcode and the CLT; I think previously I didn't fully grok the slight differences between the two. Tree-SHA512: 4d4bbb7f49acb76d934a872a15b4e14f36290b508cb9e728815f959767ec174bcfb6d2ca7dcd995cc550d86980d64d4247ea5ecfca2301f0953006e50744fdb4
jasonbcox
pushed a commit
to Bitcoin-ABC/bitcoin-abc
that referenced
this pull request
Sep 23, 2020
Summary: ``` Catalina SDK clang stopped automatically searching the SDK include paths when invoked without --sysroot: bitcoin/bitcoin#16367 (comment) Homebrew/homebrew-core#45061 This hasn't been a problem for current native depends packages because are passing their own --sysroot values, and hasn't been a problem for current host packages because they use `darwin_` commands instead of `build_darwin_` commands. But the current `build_darwin_CC` and `build_darwin_CXX` commands are still unnecessarily fragile, and incompatible with new native depends packages added in bitcoin/bitcoin#18677. Cory Fields <cory-nospam-@coryfields.com> suggested in bitcoin/bitcoin#16367 (comment) switching compiler from SDK clang to native clang (from $PATH) to avoid this problem. This is easy and makes a certain amount of sense for building native packages, as opposed to host packages. But fanquake <fanquake@gmail.com> pointed out in bitcoin/bitcoin#18677 (comment) that it would be inconsistent use switch to non-SDK compilers while still using other SDK tools like ranlib and install_name_tool. So simplest, minimal fix seems to be just adding the missing --sysroot option. ``` Backport of core [[bitcoin/bitcoin#18743 | PR18743]]. Note that it doesn't fix any real life problem for now. Test Plan: Run the OSX Gitian build. Reviewers: #bitcoin_abc, deadalnix Reviewed By: #bitcoin_abc, deadalnix Differential Revision: https://reviews.bitcoinabc.org/D7522
This was referenced Feb 27, 2021
random-zebra
added a commit
to PIVX-Project/PIVX
that referenced
this pull request
Jul 2, 2021
07f08d4 build: fix mutex detection when building bdb on macOS (fanquake) 07185c3 build: use -isysroot over --sysroot on macOS (fanquake) 424ea09 depends: Add --sysroot option to mac os native compile flags (Russell Yanofsky) Pull request description: Fixing macOS depends builds, solving #2456 issue. Back porting: * bitcoin#18743. * bitcoin#20195. * bitcoin#21793. ACKs for top commit: Fuzzbawls: ACK 07f08d4 random-zebra: utACK 07f08d4 and merging... Tree-SHA512: 2de72a30e8bdfad582b303993eff8384e74d3699992695d6d7a6ec14f0e40898e7e1d70fc6e6a74318eaf21d5e312c346748a4a5986f1f29aa149150e30bdb3f
kwvg
added a commit
to kwvg/dash
that referenced
this pull request
Jul 15, 2021
kwvg
added a commit
to kwvg/dash
that referenced
this pull request
Jul 15, 2021
UdjinM6
pushed a commit
to UdjinM6/dash
that referenced
this pull request
Jul 19, 2021
…ompile flags 1e94a2b depends: Add --sysroot option to mac os native compile flags (Russell Yanofsky) Pull request description: Catalina SDK clang stopped automatically searching the SDK include paths when invoked without `--sysroot`: - bitcoin#16367 (comment) - Homebrew/homebrew-core#45061 This hasn't been a problem for current native depends packages because are passing their own `--sysroot` values, and hasn't been a problem for current host packages because they use `darwin_` commands instead of `build_darwin_` commands. But the current `build_darwin_CC` and `build_darwin_CXX` commands are still unnecessarily fragile, and incompatible with new native depends packages added in bitcoin#18677. Cory Fields (theuni) suggested in bitcoin#16367 (comment) switching compiler from SDK clang to native clang (from $PATH) to avoid this problem. This is easy and makes a certain amount of sense for building native packages, as opposed to host packages. But Michael (fanquake) pointed out in bitcoin#18677 (comment) that it would be inconsistent to switch to non-SDK compilers while still using other SDK tools like `ranlib` and `install_name_tool`. So simplest, minimal fix seems to be just adding the missing `--sysroot` option. ACKs for top commit: ryanofsky: > ACK [1e94a2b](bitcoin@1e94a2b) - I think this change is ok, and I prefer it to the previous patch. fanquake: ACK 1e94a2b - I think this change is ok, and I prefer it to the previous patch. Thanks for the summary in the PR description. I played around with Xcode and the CLT; I think previously I didn't fully grok the slight differences between the two. Tree-SHA512: 4d4bbb7f49acb76d934a872a15b4e14f36290b508cb9e728815f959767ec174bcfb6d2ca7dcd995cc550d86980d64d4247ea5ecfca2301f0953006e50744fdb4
UdjinM6
added a commit
to dashpay/dash
that referenced
this pull request
Jul 19, 2021
Merge bitcoin#18743: depends: Add --sysroot option to mac os native compile flags
kwvg
added a commit
to kwvg/dash
that referenced
this pull request
Jul 20, 2021
kwvg
added a commit
to kwvg/dash
that referenced
this pull request
Jul 20, 2021
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Catalina SDK clang stopped automatically searching the SDK include paths when invoked without
--sysroot:This hasn't been a problem for current native depends packages because are passing their own
--sysrootvalues, and hasn't been a problem for current host packages because they usedarwin_commands instead ofbuild_darwin_commands. But the currentbuild_darwin_CCandbuild_darwin_CXXcommands are still unnecessarily fragile, and incompatible with new native depends packages added in #18677.Cory Fields (theuni) suggested in #16367 (comment) switching compiler from SDK clang to native clang (from $PATH) to avoid this problem. This is easy and makes a certain amount of sense for building native packages, as opposed to host packages. But Michael (fanquake) pointed out in #18677 (comment) that it would be inconsistent to switch to non-SDK compilers while still using other SDK tools like
ranlibandinstall_name_tool. So simplest, minimal fix seems to be just adding the missing--sysrootoption.