Skip to content

Conversation

dongcarl
Copy link
Contributor

@dongcarl dongcarl commented Apr 13, 2021

Closes #21347

Note: this is probably not absolutely necessary for Guix release, but it's nice to have!

@fanquake
Copy link
Member

Concept ACK. We should also update the HOSTS listing in the Guix README.

@dongcarl dongcarl force-pushed the 2021-03-guix-canonicalization-HOST branch from c9ea4c7 to 9e4e638 Compare April 14, 2021 14:10
@dongcarl
Copy link
Contributor Author

@laanwj
Copy link
Member

laanwj commented Apr 14, 2021

Concept ACK. Using canonicalized architecture tuples makes sense, I guess, it's good to rule out ambiguity when the architecture (at least the part that we care about) is the same but have slightly different tuples. I think the whole thing (architecture tuples in general, I mean) is a mess but that's how it is.

@DrahtBot
Copy link
Contributor

DrahtBot commented Apr 29, 2021

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@dongcarl dongcarl force-pushed the 2021-03-guix-canonicalization-HOST branch 2 times, most recently from db4a71c to 69d4d9c Compare May 4, 2021 15:50
@dongcarl
Copy link
Contributor Author

dongcarl commented May 4, 2021

Updated to be based on: #21799

dongcarl added 2 commits May 5, 2021 13:12
1. Bump time-machine to a commit that includes:

   commit 58452d08ff3e0044e9a32e6d5b3663cf185d8b33
   Author: Carl Dong <contact@carldong.me>
   Date:   Sun Mar 28 10:51:16 2021 -0700

       gnu: cross-base: Relax check for powerpc64le.

       * gnu/packages/cross-base.scm (cross-gcc-arguments): When conditionally adding
       "--with-long-double-128", check for "powerpc64le-" prefix instead of matching
       full target.

       Signed-off-by: Chris Marusich <cmmarusich@gmail.com>

2. Explicitly supply CC_BUILD to freetype's configure invocation,
   otherwise it erroneously use CC for the target build if it thinks
   that host=build. However, when Guix-building, even if host=build, we
   need to use a different CC for the target, as the Guix's default
   GCC's specfile (which inserts a lot of rpaths) is not suitable for
   our purposes.
We need to specify the qt_config_opts_aarch64_linux because the default
Qt platform profile for aarch64 assumes that gcc is named
aarch64-linux-gnu-gcc, but it's actually named
aarch64-unknown-linx-gnu-gcc if platform triples are correctly
canonicalized.
@dongcarl dongcarl force-pushed the 2021-03-guix-canonicalization-HOST branch from 69d4d9c to aa0f832 Compare May 5, 2021 17:13
@dongcarl dongcarl marked this pull request as ready for review May 5, 2021 17:13
@dongcarl
Copy link
Contributor Author

dongcarl commented May 7, 2021

Pushed 69d4d9c -> aa0f832

@laanwj
Copy link
Member

laanwj commented Jun 3, 2021

Code review ACK aa0f832

@fanquake
Copy link
Member

Tested that hosts are properly passed through, i.e x86_64-linux-gnu becomes x86_64-pc-linux-gnu. However this doesn't build the x86_64-pc-linux-gnu host for me (aa0f832):

time HOSTS="x86_64-pc-linux-gnu" BASE_CACHE="/guix/base_cache" SOURCES_PATH="/guix/sources" SDK_PATH="/guix/SDKs" ./contrib/guix/guix-build
...
INFO: Building aa0f83283504 for platform triple x86_64-pc-linux-gnu:
      ...using reference timestamp: 1618345007
      ...running at most 8 jobs
      ...from worktree directory: '/bitcoin'
          ...bind-mounted in container to: '/bitcoin'
      ...in build directory: '/bitcoin/guix-build-aa0f83283504/distsrc-aa0f83283504-x86_64-pc-linux-gnu'
          ...bind-mounted in container to: '/distsrc-base/distsrc-aa0f83283504-x86_64-pc-linux-gnu'
      ...outputting in: '/bitcoin/guix-build-aa0f83283504/output/x86_64-pc-linux-gnu'
          ...bind-mounted in container to: '/outdir-base/x86_64-pc-linux-gnu'
Required environment variables as seen inside the container:
    DIST_ARCHIVE_BASE: /outdir-base/dist-archive
    DISTNAME: bitcoin-aa0f83283504
    HOST: x86_64-pc-linux-gnu
    SOURCE_DATE_EPOCH: 1618345007
    JOBS: 8
    DISTSRC: /distsrc-base/distsrc-aa0f83283504-x86_64-pc-linux-gnu
    OUTDIR: /outdir-base/x86_64-pc-linux-gnu
make: Entering directory '/bitcoin/depends'
...
checking whether we are cross compiling... configure: error: in `/bitcoin/depends/work/build/x86_64-pc-linux-gnu/libevent/2.1.11-stable-3f710cc1119':
configure: error: cannot run C compiled programs.
If you meant to cross compile, use `--host'.
See `config.log' for more details
make: *** [funcs.mk:282: /bitcoin/depends/work/build/x86_64-pc-linux-gnu/libevent/2.1.11-stable-3f710cc1119/./.stamp_configured] Error 1
make: Leaving directory '/bitcoin/depends'

config.log

configure:2676: checking for a BSD-compatible install
configure:2744: result: /root/.guix-profile/bin/install -c
configure:2755: checking whether build environment is sane
configure:2810: result: yes
configure:2959: checking for a thread-safe mkdir -p
configure:2998: result: /root/.guix-profile/bin/mkdir -p
configure:3005: checking for gawk
configure:3021: found /root/.guix-profile/bin/gawk
configure:3032: result: gawk
configure:3043: checking whether make sets $(MAKE)
configure:3065: result: yes
configure:3094: checking whether make supports nested variables
configure:3111: result: yes
configure:3248: checking whether make supports nested variables
configure:3265: result: yes
configure:3291: checking whether make supports the include directive
configure:3306: make -f confmf.GNU && cat confinc.out
make[1]: Entering directory '/bitcoin/depends/work/build/x86_64-pc-linux-gnu/libevent/2.1.11-stable-3f710cc1119'
make[1]: Leaving directory '/bitcoin/depends/work/build/x86_64-pc-linux-gnu/libevent/2.1.11-stable-3f710cc1119'
this is the am__doit target
configure:3309: $? = 0
configure:3328: result: yes (GNU style)
configure:3358: checking for x86_64-pc-linux-gnu-gcc
configure:3385: result: x86_64-pc-linux-gnu-gcc
configure:3654: checking for C compiler version
configure:3663: x86_64-pc-linux-gnu-gcc --version >&5
x86_64-pc-linux-gnu-gcc (GCC) 8.4.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

configure:3674: $? = 0
configure:3663: x86_64-pc-linux-gnu-gcc -v >&5
Using built-in specs.
COLLECT_GCC=x86_64-pc-linux-gnu-gcc
COLLECT_LTO_WRAPPER=/gnu/store/jyg0s2g2gil3w8d7ym2i6lhz45cj5ncx-gcc-cross-x86_64-pc-linux-gnu-8.4.0/libexec/gcc/x86_64-pc-linux-gnu/8.4.0/lto-wrapper
Target: x86_64-pc-linux-gnu
Configured with: 
Thread model: posix
gcc version 8.4.0 (GCC) 
configure:3674: $? = 0
configure:3663: x86_64-pc-linux-gnu-gcc -V >&5
x86_64-pc-linux-gnu-gcc: error: unrecognized command line option '-V'
x86_64-pc-linux-gnu-gcc: fatal error: no input files
compilation terminated.
configure:3674: $? = 1
configure:3663: x86_64-pc-linux-gnu-gcc -qversion >&5
x86_64-pc-linux-gnu-gcc: error: unrecognized command line option '-qversion'; did you mean '--version'?
x86_64-pc-linux-gnu-gcc: fatal error: no input files
compilation terminated.
configure:3674: $? = 1
configure:3694: checking whether the C compiler works
configure:3716: x86_64-pc-linux-gnu-gcc -pipe -O2   -I/bitcoin/depends/x86_64-pc-linux-gnu/include   -L/bitcoin/depends/x86_64-pc-linux-gnu/lib conftest.c  >&5
configure:3720: $? = 0
configure:3768: result: yes
configure:3771: checking for C compiler default output file name
configure:3773: result: a.out
configure:3779: checking for suffix of executables
configure:3786: x86_64-pc-linux-gnu-gcc -o conftest -pipe -O2   -I/bitcoin/depends/x86_64-pc-linux-gnu/include   -L/bitcoin/depends/x86_64-pc-linux-gnu/lib conftest.c  >&5
configure:3790: $? = 0
configure:3812: result: 
configure:3834: checking whether we are cross compiling
configure:3842: x86_64-pc-linux-gnu-gcc -o conftest -pipe -O2   -I/bitcoin/depends/x86_64-pc-linux-gnu/include   -L/bitcoin/depends/x86_64-pc-linux-gnu/lib conftest.c  >&5
configure:3846: $? = 0
configure:3853: ./conftest
./conftest: error while loading shared libraries: libgcc_s.so.1: cannot open shared object file: No such file or directory
configure:3857: $? = 127
configure:3864: error: in `/bitcoin/depends/work/build/x86_64-pc-linux-gnu/libevent/2.1.11-stable-3f710cc1119':
configure:3866: error: cannot run C compiled programs.
If you meant to cross compile, use `--host'.
See `config.log' for more details

All other hosts seem to build fine.

@dongcarl
Copy link
Contributor Author

dongcarl commented Jun 15, 2021

Very curious as I did not encounter this before (or perhaps I did and did not notice?)

I will investigate further but the following patch fixes the problem (not in any way that we'd want in our repo), which is interesting.

diff --git a/depends/funcs.mk b/depends/funcs.mk
index 34a030fab7..e5fcb352a8 100644
--- a/depends/funcs.mk
+++ b/depends/funcs.mk
@@ -146,7 +146,7 @@ $(1)_stage_env+=PATH=$(build_prefix)/bin:$(PATH)
 # config.guess, which is what we set it too here. This also quells autoconf
 # warnings, "If you wanted to set the --build type, don't use --host.",
 # when using versions older than 2.70.
-$(1)_autoconf=./configure --build=$(BUILD) --host=$($($(1)_type)_host) --prefix=$($($(1)_type)_prefix) $$($(1)_config_opts) CC="$$($(1)_cc)" CXX="$$($(1)_cxx)"
+$(1)_autoconf=./configure --build=$(BUILD) --host=$($($(1)_type)_host) --prefix=$($($(1)_type)_prefix) $$($(1)_config_opts) CC="$$($(1)_cc)" CXX="$$($(1)_cxx)" cross_compiling=yes
 ifneq ($($(1)_nm),)
 $(1)_autoconf += NM="$$($(1)_nm)"
 endif

dongcarl added 2 commits June 18, 2021 14:07
Abstractly, our Guix builds do not treat native-builds and cross-builds
differently. We always build 2 toolchains:

1. A native toolchain (in depends terms: build) (namely, the
   `gcc-toolchain` package, with the command line name being simply
   `gcc`), and
2. A cross toolchain (in depends terms: host) (namely, packages returned
   by the `make-cross-toolchain` procedure in `manifest.scm`, with the
   command line name being something like `x86_64-pc-linux-gnu-gcc`)

This means that even if we are technically not cross
compiling (`build==host`), we do not coalesce or reuse the toolchains in
any way. This is necessary because our release binaries (produced by the
cross toolchain) are built to:

1. Have a dependency on a minimal number of shared libraries
2. Expect the dynamic linker/loader `ld-linux.so(8)` to be somewhere
   like `/lib{,64}/ld*`

This is not the case for binaries produced by Guix's native
`gcc-toolchain` package. They are built to work best in Guix containers,
where the binaries reach for libraries and `ld-linux.so(8)` in
`/gnu/store` and have dependencies recorded as RUNPATHs in their dynamic
section (since paths are deterministic).

In other words, Guix's native `gcc-toolchain` package can be thought of
as a different platform triple: `x86_64-guix-linux-gnu` (PLEASE don't
actually use this triple this is not standard just a way to think about
it).

We can therefore think of performing a Guix build on `x86_64` for
`x86_64` as a cross-compilation with build=`x86_64-guix-linux-gnu` and
host=`x86_64-pc-linux-gnu` since even when the architecture and OS
matches, binaries built with the native toolchain will not run on normal
distros, and binaries built with the cross toolchain will not run in
Guix environments.

---

With this context in mind, we can now discuss our particular issue.

Prior to canonicalizing `$HOST` (this PR), autoconf's barely-working
cross compilation detection (which is marked `# FIXME: To remove some
day.`) set `cross_compiling=yes` if `$host` is not the exact same string
as `$build`:
http://git.savannah.gnu.org/gitweb/?p=autoconf.git;a=blob;f=lib/autoconf/general.m4;h=053130b3d0ba7af8b9177da6ff850d8c20ac6cbe;hb=HEAD#l972.
Which actually worked in our favor since we were not canonicalizing
`host` and specifying `build=x86_64-pc-linux-gnu` and
`host=x86_64-linux-gnu`, and the missing `-pc-` made autoconf assume
that it was a cross-compilation.

After canonicalizing `$HOST` autoconf instead sees that `$build==$host`
and sets `cross_compiling=no`, which leads to later configure tests to
compile `conftest` binaries using `CC` (namely, our cross-toolchain
`x86_64-pc-linux-gnu-gcc`) and attempt to run them inside the Guix
container. This leads to the problems fanquake identified here:
bitcoin#21671 (comment)

The solution is to be explicit and supply `cross_compiling=yes` to
`./configure`, which is us saying: "yes i'm on the same architecture,
OS, and userspace as the target, but there are important differences in
what we want in our dynamic sections which lead to binaries not being
able to find the right libraries to load if they were copied to the
other environment so just assume that I'm cross compiling and don't try
to be smart"
@dongcarl
Copy link
Contributor Author

Pushed aa0f832..d0e2378

Wrote up a description of my investigation


Abstractly, our Guix builds do not treat native-builds and cross-builds differently. We always build 2 toolchains:

  1. A native toolchain (in depends terms: build) (namely, the gcc-toolchain package, with the command line name being simply gcc), and
  2. A cross toolchain (in depends terms: host) (namely, packages returned by the make-cross-toolchain procedure in manifest.scm, with the command line name being something like x86_64-pc-linux-gnu-gcc)

This means that even if we are technically not cross compiling (build==host), we do not coalesce or reuse the toolchains in any way. This is necessary because our release binaries (produced by the cross toolchain) are built to:

  1. Have a dependency on a minimal number of shared libraries
  2. Expect the dynamic linker/loader ld-linux.so(8) to be somewhere like /lib{,64}/ld*

This is not the case for binaries produced by Guix's native gcc-toolchain package. They are built to work best in Guix containers, where the binaries reach for libraries and ld-linux.so(8) in /gnu/store and have dependencies recorded as RUNPATHs in their dynamic section (since paths are deterministic).

In other words, Guix's native gcc-toolchain package can be thought of as a different platform triple: x86_64-guix-linux-gnu (PLEASE don't actually use this triple this is not standard just a way to think about it).

We can therefore think of performing a Guix build on x86_64 for x86_64 as a cross-compilation with build=x86_64-guix-linux-gnu and host=x86_64-pc-linux-gnu since even when the architecture and OS matches, binaries built with the native toolchain will not run on normal distros, and binaries built with the cross toolchain will not run in Guix environments.


With this context in mind, we can now discuss our particular issue.

Prior to canonicalizing $HOST (this PR), autoconf's barely-working cross compilation detection (which is marked # FIXME: To remove some day.) set cross_compiling=yes if $host is not the exact same string as $build: http://git.savannah.gnu.org/gitweb/?p=autoconf.git;a=blob;f=lib/autoconf/general.m4;h=053130b3d0ba7af8b9177da6ff850d8c20ac6cbe;hb=HEAD#l972. Which actually worked in our favor since we were not canonicalizing host and specifying build=x86_64-pc-linux-gnu and host=x86_64-linux-gnu, and the missing -pc- made autoconf assume that it was a cross-compilation.

After canonicalizing $HOST autoconf instead sees that $build==$host and sets cross_compiling=no, which leads to later configure tests to compile conftest binaries using CC (namely, our cross-toolchain x86_64-pc-linux-gnu-gcc) and attempt to run them inside the Guix container. This leads to the problems fanquake identified here: #21671 (comment)

The solution is to be explicit and supply cross_compiling=yes to ./configure, which is us saying: "yes i'm on the same architecture, OS, and userspace as the target, but there are important differences in what we want in our dynamic sections which lead to binaries not being able to find the right libraries to load if they were copied to the other environment so just assume that I'm cross compiling and don't try to be smart"

@DrahtBot
Copy link
Contributor

DrahtBot commented Jul 5, 2021

🐙 This pull request conflicts with the target branch and needs rebase.

Want to unsubscribe from rebase notifications on this pull request? Just convert this pull request to a "draft".

@DrahtBot
Copy link
Contributor

There hasn't been much activity lately and the patch still needs rebase. What is the status here?
  • Is it still relevant? ➡️ Please solve the conflicts to make it ready for review and to ensure the CI passes.
  • Is it no longer relevant? ➡️ Please close.
  • Did the author lose interest or time to work on this? ➡️ Please close it and mark it 'Up for grabs' with the label, so that it can be picked up in the future.

fanquake pushed a commit to fanquake/bitcoin that referenced this pull request Aug 4, 2022
Abstractly, our Guix builds do not treat native-builds and cross-builds
differently. We always build 2 toolchains:

1. A native toolchain (in depends terms: build) (namely, the
   `gcc-toolchain` package, with the command line name being simply
   `gcc`), and
2. A cross toolchain (in depends terms: host) (namely, packages returned
   by the `make-cross-toolchain` procedure in `manifest.scm`, with the
   command line name being something like `x86_64-pc-linux-gnu-gcc`)

This means that even if we are technically not cross
compiling (`build==host`), we do not coalesce or reuse the toolchains in
any way. This is necessary because our release binaries (produced by the
cross toolchain) are built to:

1. Have a dependency on a minimal number of shared libraries
2. Expect the dynamic linker/loader `ld-linux.so(8)` to be somewhere
   like `/lib{,64}/ld*`

This is not the case for binaries produced by Guix's native
`gcc-toolchain` package. They are built to work best in Guix containers,
where the binaries reach for libraries and `ld-linux.so(8)` in
`/gnu/store` and have dependencies recorded as RUNPATHs in their dynamic
section (since paths are deterministic).

In other words, Guix's native `gcc-toolchain` package can be thought of
as a different platform triple: `x86_64-guix-linux-gnu` (PLEASE don't
actually use this triple this is not standard just a way to think about
it).

We can therefore think of performing a Guix build on `x86_64` for
`x86_64` as a cross-compilation with build=`x86_64-guix-linux-gnu` and
host=`x86_64-pc-linux-gnu` since even when the architecture and OS
matches, binaries built with the native toolchain will not run on normal
distros, and binaries built with the cross toolchain will not run in
Guix environments.

---

With this context in mind, we can now discuss our particular issue.

Prior to canonicalizing `$HOST` (this PR), autoconf's barely-working
cross compilation detection (which is marked `# FIXME: To remove some
day.`) set `cross_compiling=yes` if `$host` is not the exact same string
as `$build`:
http://git.savannah.gnu.org/gitweb/?p=autoconf.git;a=blob;f=lib/autoconf/general.m4;h=053130b3d0ba7af8b9177da6ff850d8c20ac6cbe;hb=HEAD#l972.
Which actually worked in our favor since we were not canonicalizing
`host` and specifying `build=x86_64-pc-linux-gnu` and
`host=x86_64-linux-gnu`, and the missing `-pc-` made autoconf assume
that it was a cross-compilation.

After canonicalizing `$HOST` autoconf instead sees that `$build==$host`
and sets `cross_compiling=no`, which leads to later configure tests to
compile `conftest` binaries using `CC` (namely, our cross-toolchain
`x86_64-pc-linux-gnu-gcc`) and attempt to run them inside the Guix
container. This leads to the problems fanquake identified here:
bitcoin#21671 (comment)

The solution is to be explicit and supply `cross_compiling=yes` to
`./configure`, which is us saying: "yes i'm on the same architecture,
OS, and userspace as the target, but there are important differences in
what we want in our dynamic sections which lead to binaries not being
able to find the right libraries to load if they were copied to the
other environment so just assume that I'm cross compiling and don't try
to be smart"
fanquake pushed a commit to fanquake/bitcoin that referenced this pull request Aug 11, 2022
Abstractly, our Guix builds do not treat native-builds and cross-builds
differently. We always build 2 toolchains:

1. A native toolchain (in depends terms: build) (namely, the
   `gcc-toolchain` package, with the command line name being simply
   `gcc`), and
2. A cross toolchain (in depends terms: host) (namely, packages returned
   by the `make-cross-toolchain` procedure in `manifest.scm`, with the
   command line name being something like `x86_64-pc-linux-gnu-gcc`)

This means that even if we are technically not cross
compiling (`build==host`), we do not coalesce or reuse the toolchains in
any way. This is necessary because our release binaries (produced by the
cross toolchain) are built to:

1. Have a dependency on a minimal number of shared libraries
2. Expect the dynamic linker/loader `ld-linux.so(8)` to be somewhere
   like `/lib{,64}/ld*`

This is not the case for binaries produced by Guix's native
`gcc-toolchain` package. They are built to work best in Guix containers,
where the binaries reach for libraries and `ld-linux.so(8)` in
`/gnu/store` and have dependencies recorded as RUNPATHs in their dynamic
section (since paths are deterministic).

In other words, Guix's native `gcc-toolchain` package can be thought of
as a different platform triple: `x86_64-guix-linux-gnu` (PLEASE don't
actually use this triple this is not standard just a way to think about
it).

We can therefore think of performing a Guix build on `x86_64` for
`x86_64` as a cross-compilation with build=`x86_64-guix-linux-gnu` and
host=`x86_64-pc-linux-gnu` since even when the architecture and OS
matches, binaries built with the native toolchain will not run on normal
distros, and binaries built with the cross toolchain will not run in
Guix environments.

---

With this context in mind, we can now discuss our particular issue.

Prior to canonicalizing `$HOST` (this PR), autoconf's barely-working
cross compilation detection (which is marked `# FIXME: To remove some
day.`) set `cross_compiling=yes` if `$host` is not the exact same string
as `$build`:
http://git.savannah.gnu.org/gitweb/?p=autoconf.git;a=blob;f=lib/autoconf/general.m4;h=053130b3d0ba7af8b9177da6ff850d8c20ac6cbe;hb=HEAD#l972.
Which actually worked in our favor since we were not canonicalizing
`host` and specifying `build=x86_64-pc-linux-gnu` and
`host=x86_64-linux-gnu`, and the missing `-pc-` made autoconf assume
that it was a cross-compilation.

After canonicalizing `$HOST` autoconf instead sees that `$build==$host`
and sets `cross_compiling=no`, which leads to later configure tests to
compile `conftest` binaries using `CC` (namely, our cross-toolchain
`x86_64-pc-linux-gnu-gcc`) and attempt to run them inside the Guix
container. This leads to the problems fanquake identified here:
bitcoin#21671 (comment)

The solution is to be explicit and supply `cross_compiling=yes` to
`./configure`, which is us saying: "yes i'm on the same architecture,
OS, and userspace as the target, but there are important differences in
what we want in our dynamic sections which lead to binaries not being
able to find the right libraries to load if they were copied to the
other environment so just assume that I'm cross compiling and don't try
to be smart"
@bitcoin bitcoin locked and limited conversation to collaborators Mar 21, 2023
@fanquake
Copy link
Member

I'm going to remove "Up for grabs" here. Not entirely convinvced that we should end up doing this.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

guix: guix doesn't like x86_64-pc-linux-gnu
4 participants