Skip to content

[POC] guix: produce a fully -static-pie bitcoind #25573

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 4 commits into
base: master
Choose a base branch
from
Draft

[POC] guix: produce a fully -static-pie bitcoind #25573

wants to merge 4 commits into from
+161 −10

Conversation

fanquake
Copy link
Member

@fanquake fanquake commented Jul 8, 2022 •
edited
Loading

Use Guix to produce a fully -static-pie bitcoind using GCC 13.3.0 and glibc 2.35. The binary is ~18mb stripped, includes wallet/upnp/natpmp/zmq etc. This is currently done for x86_64-linux-gnu, aarch64-linux-gnu.

# file bitcoind
	ELF 64-bit LSB pie executable, x86-64, version 1 (GNU/Linux), static-pie linked, for GNU/Linux 3.2.0, stripped

# ldd bitcoind
	statically linked

-static-pie

Produce a static position independent executable on targets that support it.
A static position independent executable is similar to a static executable,
but can be loaded at any address without a dynamic linker.

Using glibc for static builds, would be less of a leap compared to switching to musl libc (POC in #23203).

@fanquake fanquake marked this pull request as draft July 8, 2022 16:49
@DrahtBot
Copy link
Contributor

DrahtBot commented Jul 8, 2022 •
edited
Loading

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/25573.

Reviews

See the guideline for information on the review process.
A summary of reviews will appear here.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #33185 (guix: update time-machine to 5cb84f2013c5b1e48a7d0e617032266f1e6059e2 by fanquake)
  • #33181 (guix: build for Linux HOSTS with -static-libgcc by fanquake)
  • #24123 (guix: Pointer Authentication and Branch Target Identification for aarch64 Linux by fanquake)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

This was referenced Jul 8, 2022
Merged
Closed
Closed
Closed
Draft
@fanquake fanquake force-pushed the fully_static_bitcoind branch from da9f731 to 26c437e Compare July 13, 2022 17:44
@fanquake
Copy link
Member Author

Rebased, and made small changes to fixup the linter and improve the security/symbol checks.

@theuni
Copy link
Member

theuni commented Jul 13, 2022

This is amazing!

Using glibc for static builds, would be less of a leap compared to switching to musl libc (POC in #23203).

Hard agree. Static musl builds would be very cool but tricky to audit for parity and thus pretty unlikely to ever become official/supported imo. This is a MUCH more palatable change.

@fanquake fanquake force-pushed the fully_static_bitcoind branch from 26c437e to 8cccf45 Compare July 14, 2022 10:41
@fanquake
Copy link
Member Author

Added an additional patch to disable building various features/libs in GCC that we don't need. i.e libquadmath, libsanitizer, gcov, libada and native language support. This should speed up compiling GCC, and slightly reduces the size of the pthread patch. I'm not currently disabling these features via the available configure flags, because they get passed through to GMP, which cannot handle them.

I've also updated -flto usage to be -flto=auto, as that seems to now be the preferred invocation, and it mitigates GCC warning output. i.e:

lto-wrapper: warning: using serial compilation of 84 LTRANS jobs
lto-wrapper: note: see the '-flto' option documentation for more information

I've also split the ELF PIE check into it's own func, so we can special-case x86_64, and re-added the list of allowed ELF libs, but special-cased x86_64 to only pass if the list is empty.

@luke-jr
Copy link
Member

luke-jr commented Jul 16, 2022

How does this handle resolvers? eg, static glibc 2.33 applications break on systems with glibc 2.34...

This was referenced Jul 18, 2022
Merged
Merged
@fanquake fanquake force-pushed the fully_static_bitcoind branch from 8cccf45 to caa0664 Compare July 19, 2022 16:27
@fanquake fanquake mentioned this pull request Jul 27, 2022
Closed
@Frank-GER
Copy link

Which OS did you build on?

On Ubuntu 20.04 after build:
file bitcoind
bitcoind: ELF 64-bit LSB shared object, x86-64, version 1 (GNU/Linux), dynamically linked, for GNU/Linux 3.2.0, stripped

Trying to run it, it crashes after seconds:
addcon thread start
Loading addresses from DNS seed dnsseed.emzy.de.
bitcoind: dl-call-libc-early-init.c:37: _dl_call_libc_early_init: Assertion `sym != NULL' failed.
Aborted (core dumped)

Any ideas?

@fanquake
Copy link
Member Author

fanquake commented Aug 5, 2022

Which OS did you build on?

The base OS shouldn't matter, because the build is performed with Guix. However I've been performing these builds on Ubuntu 22.04.

Any ideas?

How did you actually build bitcoind? Something like HOSTS="x86_64-linux-gnu" ./contrib/guix/guix-build?

@fanquake fanquake force-pushed the fully_static_bitcoind branch from caa0664 to ef0edd5 Compare August 5, 2022 08:35
@fanquake
Copy link
Member Author

fanquake commented Aug 5, 2022

Rebased on master. Added a commit to use LIEF master, which reduces the number of changes in our scripts, as support for -static-pie ELF binaries has been added: lief-project/LIEF#747. Will switch to a new release when one is available.

@Frank-GER
Copy link

Frank-GER commented Aug 5, 2022 •
edited
Loading

How did you actually build bitcoind? Something like HOSTS="x86_64-linux-gnu" ./contrib/guix/guix-build?

Yes, that's what I used to build it.
https://github.com/fanquake/bitcoin/commits/fully_static_bitcoind produces the exact same problem when building on Ubuntu 20.04

@fanquake
Copy link
Member Author

fanquake commented Aug 5, 2022

https://github.com/fanquake/bitcoin/commits/fully_static_bitcoind produces the exact same problem when building on Ubuntu 20.04

I have just re-tested building the changes here (ef0edd5), and this produces a -static-pie binary as expected:

guix-build-ef0edd55f3bc/distsrc-ef0edd55f3bc-x86_64-linux-gnu/src/bitcoind: ELF 64-bit LSB pie executable, x86-64, version 1 (GNU/Linux), static-pie linked, for GNU/Linux 3.2.0, with debug_info, not stripped

It's not clear to me that you are actually running a Guix build.

@Frank-GER
Copy link

Frank-GER commented Aug 5, 2022 •
edited
Loading

the build I was testing:
~/bitcoin/guix-build-caa066465973/distsrc-caa066465973-x86_64-linux-gnu/src$ file bitcoind
bitcoind: ELF 64-bit LSB shared object, x86-64, version 1 (GNU/Linux), dynamically linked, for GNU/Linux 3.2.0, with debug_info, not stripped

just building again, fresh cloned with commit ef0edd5
export HOSTS='x86_64-linux-gnu'
./contrib/guix/guix-build

$ file guix-build-ef0edd55f3bc/distsrc-ef0edd55f3bc-x86_64-linux-gnu/src/bitcoind
guix-build-ef0edd55f3bc/distsrc-ef0edd55f3bc-x86_64-linux-gnu/src/bitcoind: ELF 64-bit LSB shared object, x86-64, version 1 (GNU/Linux), dynamically linked, for GNU/Linux 3.2.0, with debug_info, not stripped

$ ldd ./guix-build-ef0edd55f3bc/distsrc-ef0edd55f3bc-x86_64-linux-gnu/src/bitcoind
statically linked

$ ./bitcoind
...
2022-08-05T19:10:46Z init message: Starting network threads…
2022-08-05T19:10:46Z net thread start
2022-08-05T19:10:46Z dnsseed thread start
2022-08-05T19:10:46Z Loading addresses from DNS seed seed.bitcoinstats.com.
2022-08-05T19:10:46Z opencon thread start
2022-08-05T19:10:46Z msghand thread start
2022-08-05T19:10:46Z addcon thread start
2022-08-05T19:10:46Z init message: Done loading
bitcoind: dl-call-libc-early-init.c:37: _dl_call_libc_early_init: Assertion `sym != NULL' failed.
Aborted (core dumped)

build and run on Ubuntu 20.04
there is no other bitcoind on that OS

@Frank-GER
Copy link

building the same on Ubuntu 22.04:
file guix-build-ef0edd55f3bc/distsrc-ef0edd55f3bc-x86_64-linux-gnu/src/bitcoind
guix-build-ef0edd55f3bc/distsrc-ef0edd55f3bc-x86_64-linux-gnu/src/bitcoind: ELF 64-bit LSB pie executable, x86-64, version 1 (GNU/Linux), static-pie linked, for GNU/Linux 3.2.0, with debug_info, not stripped

ldd guix-build-ef0edd55f3bc/distsrc-ef0edd55f3bc-x86_64-linux-gnu/src/bitcoind
statically linked

Running it on Ubuntu22 it works.

Taking this bitcoind to Ubuntu 20 it shows:
file ./bitcoind
./bitcoind: ELF 64-bit LSB shared object, x86-64, version 1 (GNU/Linux), dynamically linked, for GNU/Linux 3.2.0, with debug_info, not stripped

and crashes with the above mentioned message:
bitcoind: dl-call-libc-early-init.c:37: _dl_call_libc_early_init: Assertion `sym != NULL' failed.
Aborted (core dumped)

Checksums are the same for both builds (on Ubuntu 20 and 22)
Ubuntu 20 sees it as dynamic linked, Ubuntu 22 as static-pie linked.
On Ubuntu 20 it crashes, on Ubuntu 22 it works.

Same bitcoind on Debian 10:
file ./bitcoind
./bitcoind: ELF 64-bit LSB shared object, x86-64, version 1 (GNU/Linux), dynamically linked, for GNU/Linux 3.2.0, with debug_info, not stripped
ldd ./bitcoind
statically linked
Gets past that point where it crashes on Ubuntu 20 and starts syncing

@fanquake fanquake force-pushed the fully_static_bitcoind branch from 0d34d13 to 80a9ee0 Compare February 19, 2025 20:40
@fanquake fanquake force-pushed the fully_static_bitcoind branch from 80a9ee0 to 41720e2 Compare February 27, 2025 14:40
@fanquake fanquake force-pushed the fully_static_bitcoind branch from 41720e2 to 550bb0d Compare March 12, 2025 03:36
@fanquake fanquake force-pushed the fully_static_bitcoind branch from 550bb0d to d657be0 Compare March 20, 2025 06:21
@DrahtBot DrahtBot mentioned this pull request Mar 29, 2025
Open
@fanquake fanquake force-pushed the fully_static_bitcoind branch from d657be0 to e881c8c Compare April 11, 2025 13:53
This was referenced Apr 11, 2025
Closed
Open
Merged
This was referenced May 7, 2025
Merged
Merged
@fanquake fanquake force-pushed the fully_static_bitcoind branch from e881c8c to 9c7154a Compare May 12, 2025 18:52
@fanquake fanquake force-pushed the fully_static_bitcoind branch 2 times, most recently from 74e927f to 5959093 Compare May 14, 2025 16:08
@fanquake fanquake force-pushed the fully_static_bitcoind branch from 5959093 to e7bdb2b Compare May 28, 2025 10:03
fanquake added 4 commits June 11, 2025 14:52
@fanquake
guix: add fortified glibc 2.40
677de2e 
https://sourceware.org/git/?p=glibc.git;a=shortlog;h=refs/heads/release/2.40/master
@fanquake
guix: patch around riscv issue with newer (2.40+) binutils
a89446d 
See https://sourceware.org/bugzilla/show_bug.cgi?id=28509.
https://sourceware.org/git/?p=glibc.git;a=commit;h=68389203832ab39dd0dbaabbc4059e7fff51c29b.
@fanquake
guix: update to c5265b90b055ee15908298c5d463301f1aae2eb1
8525ec4 
Package updates:
file 5.44 -> 5.45
(base) glibc 2.35 -> 2.39
grep 3.8 -> 3.11
diffutils 3.8 -> 3.10
gawk 5.2.1 -> 5.3.0
gzip 1.12 -> 1.13
xz 5.2.8 -> 5.4.5
make 4.3 -> 4.4.1
git-minimal 2.45.2 -> 2.48.1
binutils 2.38 -> 2.41
linux-headers 6.1.106 -> 6.1.128

CMake-3.30 becomes available.
Clang/LLVM 19 & 20 become available.
@fanquake
guix: build x86_64-linux bitcoind fully statically
fa0d534 
-static-pie
Produce a static position independent executable on targets that support it.
A static position independent executable is similar to a static executable,
but can be loaded at any address without a dynamic linker.

See https://gcc.gnu.org/onlinedocs/gcc/Link-Options.html
@fanquake fanquake force-pushed the fully_static_bitcoind branch from e7bdb2b to fa0d534 Compare June 11, 2025 14:03
This was referenced Aug 14, 2025
Open
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
Build system
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@fanquake @DrahtBot @theuni @luke-jr @Frank-GER