Skip to content

Conversation

fanquake
Copy link
Member

There was a reproducibility issue (IRC logs) with the osx 0.19.1rc1 gitian builds. The build-id embedded into the dmg tool was mismatching. It's possible that differing versions of binutils/ld were the cause.

While it was resolved after rebuilding the base gitian image, whether an upstream package issue or fluke, we can remove the possibility of it happening in future by just not embedding a build-id into the dmg tool at all. Can close if it's not deemed worth it.

You can test this change using the following:

# build libdmg
make native_libdmg-hfsplus_built -C depends/ HOST=x86_64-apple-darwin16 -j6 V=1

# master
readelf --string-dump .note.gnu.build-id /bitcoin/depends/work/build/x86_64-apple-darwin16/native_libdmg-hfsplus/7ac55ec64c96f7800d9818ce64c79670e7f02b67-3830944ef98/build/dmg/dmg 

String dump of section '.note.gnu.build-id':
  [     c]  GNU
  [    11]  CjRa?]?^V8?v?;%n??

# this pr
readelf --string-dump  .note.gnu.build-id /bitcoin/depends/work/build/x86_64-apple-darwin16/native_libdmg-hfsplus/7ac55ec64c96f7800d9818ce64c79670e7f02b67-a72f53ab110/build/dmg/dmg 
readelf: Warning: Section '.note.gnu.build-id' was not dumped because it does not exist!

@practicalswift
Copy link
Contributor

Concept ACK: reproducibility is good

@laanwj
Copy link
Member

laanwj commented Feb 5, 2020

ACK 2b990a1

What I don't really understand yet is why this affected the build of one binary (a tool for MacOS DMG generation) but not the -linux build. But this can't hurt determinism, at least.

@laanwj
Copy link
Member

laanwj commented Feb 5, 2020

BTW: it might be possible to provide -DCMAKE_C_FLAGS="-Wl,--build-id=none" on the cmake command line, passing it in directly, avoiding the need for a patch.

@fanquake
Copy link
Member Author

fanquake commented Feb 6, 2020

BTW: it might be possible to provide -DCMAKE_C_FLAGS="-Wl,--build-id=none"

Sure, lets just do that rather than maintaining a patch. Updated.

@laanwj
Copy link
Member

laanwj commented Feb 10, 2020

re-code-ACK
going to test this in gitian locally cb9e88e

@laanwj
Copy link
Member

laanwj commented Feb 10, 2020

tested ACK cb9e88e

checked bitcoin-0.19.99-osx-unsigned.tar.gz:

$ readelf --string-dump .note.gnu.build-id  dmg
readelf: Warning: Section '.note.gnu.build-id' was not dumped because it does not exist!

however, the same archive also contains genisoimage which still contains the section

$ readelf --string-dump .note.gnu.build-id  genisoimage

String dump of section '.note.gnu.build-id':
  [     c]  GNU
  [    13]  sm^^X

is this a problem ?

@bitcoin bitcoin deleted a comment from DrahtBot Feb 10, 2020
@fanquake
Copy link
Member Author

is this a problem ?

I'd be happy to leave that as is for now, given we haven't seen any issues with it.

fanquake added a commit that referenced this pull request Feb 12, 2020
cb9e88e build: don't embed a build-id when building libdmg-hfsplus (fanquake)

Pull request description:

  There was a [reproducibility issue (IRC logs)](http://www.erisian.com.au/bitcoin-core-dev/log-2020-01-25.html) with the osx `0.19.1rc1` gitian builds. The `build-id` embedded into the `dmg` tool was mismatching. It's possible that differing versions of binutils/ld were the cause.

  While it was resolved after rebuilding the base gitian image, whether an upstream package issue or fluke, we can remove the possibility of it happening in future by just not embedding a build-id into the `dmg` tool at all. Can close if it's not deemed worth it.

  You can test this change using the following:

  ```bash
  # build libdmg
  make native_libdmg-hfsplus_built -C depends/ HOST=x86_64-apple-darwin16 -j6 V=1

  # master
  readelf --string-dump .note.gnu.build-id /bitcoin/depends/work/build/x86_64-apple-darwin16/native_libdmg-hfsplus/7ac55ec64c96f7800d9818ce64c79670e7f02b67-3830944ef98/build/dmg/dmg

  String dump of section '.note.gnu.build-id':
    [     c]  GNU
    [    11]  CjRa?]?^V8?v?;%n??

  # this pr
  readelf --string-dump  .note.gnu.build-id /bitcoin/depends/work/build/x86_64-apple-darwin16/native_libdmg-hfsplus/7ac55ec64c96f7800d9818ce64c79670e7f02b67-a72f53ab110/build/dmg/dmg
  readelf: Warning: Section '.note.gnu.build-id' was not dumped because it does not exist!
  ```

ACKs for top commit:
  laanwj:
    tested ACK cb9e88e

Tree-SHA512: 191eed32ed0a04a908f9c1b22188180b2db2f35bae0281940f0f9da2450c5c6807cd6ff5bbcce7c933a9133387b127c3478f7a39a2918c14f17be19fd9ea19b4
@fanquake fanquake merged commit cb9e88e into bitcoin:master Feb 12, 2020
@fanquake fanquake deleted the libdmg_no_build_id branch February 12, 2020 00:51
fanquake added a commit to fanquake/bitcoin that referenced this pull request Feb 28, 2020
@fanquake
Copy link
Member Author

Being backported in 18218.

maflcko pushed a commit that referenced this pull request Mar 4, 2020
48fef5e gui: Fix race in WalletModel::pollBalanceChanged (Russell Yanofsky)
1964561 build: don't embed a build-id when building libdmg-hfsplus (fanquake)

Pull request description:

  These are not blockers for the 0.19.1 release, as per [IRC discussion this morning](http://www.erisian.com.au/bitcoin-core-dev/log-2020-02-27.html#l-331), doesn't look like there will be an rc3. This PR can collect further backports for the 0.19 branch.

  Currently backports:
  * #18004 - build: don't embed a build-id when building libdmg-hfsplus
  * #18123 - gui: Fix race in WalletModel::pollBalanceChanged

ACKs for top commit:
  promag:
    ACK 48fef5e.
  laanwj:
    ACK 48fef5e
  luke-jr:
    utACK 48fef5e

Tree-SHA512: c7e7ddda9ee7b8015f16d39aab000e0595f85fe073f79abc1a57b3e2adb0dedc4e07e5fd918e1df5e88b7f3fbc39b57ab3382233c4354b9c2196f65fa1fa6c04
MarkLTZ added a commit to litecoinz-core/litecoinz that referenced this pull request Mar 13, 2020
[0.19] Backports bitcoin#17858
Unbreak build with Boost 1.72.0 bitcoin#17654
cli: fix Fatal LevelDB error when specifying -blockfilterindex=basic twice bitcoin#17687
rpc: require second argument only for scantxoutset start action bitcoin#17728
wallet: Fix origfee return for bumpfee with feerate arg bitcoin#17643
test: fix "bitcoind already running" warnings on macOS bitcoin#17488
net: Log to net category for exceptions in ProcessMessages bitcoin#17762
Updates to appveyor config for VS2019 and Qt5.9.8 + msvc project fixes bitcoin#17364
Appveyor improvement - text file for vcpkg package list bitcoin#17416
Update msvc build for Visual Studio 2019 v16.4 bitcoin#17736
scripts: fix symbol-check & security-check argument passing bitcoin#17857
qt: Periodic translations update for 0.19 branch
IsUsedDestination should count any known single-key address bitcoin#17621
init: Stop indexes on shutdown after ChainStateFlushed callback. bitcoin#17897
qt: Translations update pre-rc1
wallet: Reset reused transactions cache bitcoin#17843
Squashed 'src/univalue/' changes from 7890db9..98261b1
0.19: Update univalue subtree bitcoin#18100
qt: Pre-rc2 translations update
[0.19] Further 0.19 backports bitcoin#18218
build: don't embed a build-id when building libdmg-hfsplus bitcoin#18004
deadalnix pushed a commit to Bitcoin-ABC/bitcoin-abc that referenced this pull request Apr 3, 2020
Summary:
```
There was a reproducibility issue (IRC logs) with the osx 0.19.1rc1
gitian builds. The build-id embedded into the dmg tool was mismatching.
It's possible that differing versions of binutils/ld were the cause.

While it was resolved after rebuilding the base gitian image, whether an
upstream package issue or fluke, we can remove the possibility of it
happening in future by just not embedding a build-id into the dmg tool
at all.
```

Backport of core [[bitcoin/bitcoin#18004 | PR18004]].

Test Plan: Run the Gitian OSX build twice and check the build is deterministic.

Reviewers: #bitcoin_abc, deadalnix

Reviewed By: #bitcoin_abc, deadalnix

Differential Revision: https://reviews.bitcoinabc.org/D5665
ftrader pushed a commit to bitcoin-cash-node/bitcoin-cash-node that referenced this pull request Aug 17, 2020
Summary:
```
There was a reproducibility issue (IRC logs) with the osx 0.19.1rc1
gitian builds. The build-id embedded into the dmg tool was mismatching.
It's possible that differing versions of binutils/ld were the cause.

While it was resolved after rebuilding the base gitian image, whether an
upstream package issue or fluke, we can remove the possibility of it
happening in future by just not embedding a build-id into the dmg tool
at all.
```

Backport of core [[bitcoin/bitcoin#18004 | PR18004]].

Test Plan: Run the Gitian OSX build twice and check the build is deterministic.

Reviewers: #bitcoin_abc, deadalnix

Reviewed By: #bitcoin_abc, deadalnix

Differential Revision: https://reviews.bitcoinabc.org/D5665
kwvg added a commit to kwvg/dash that referenced this pull request Nov 1, 2021
kwvg added a commit to kwvg/dash that referenced this pull request Nov 1, 2021
kwvg added a commit to kwvg/dash that referenced this pull request Nov 1, 2021
kwvg added a commit to kwvg/dash that referenced this pull request Nov 1, 2021
kwvg added a commit to kwvg/dash that referenced this pull request Nov 1, 2021
UdjinM6 added a commit to dashpay/dash that referenced this pull request Nov 2, 2021
pravblockc pushed a commit to pravblockc/dash that referenced this pull request Nov 18, 2021
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Feb 15, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants