build: mac deployment unification #20422

fanquake · 2020-11-19T04:13:56Z

This consolidates our macOS build code so that .DS_Store generation is the same when running make deploy for macOS when building on Linux and macOS, rather than maintaining two version of code that essentially do the same thing (just slightly differently).

It also removes unused code and any AppleScript usage, automates finding translation files and generally simplifies macdeployqtplus. It also gets rid of the annoying "popping up" behaviour during DMG generation, names the created image Bitcoin-Core.dmg rather than Bitcoin-Qt.dmg.

laanwj · 2020-11-19T09:27:40Z

Concept ACK

DrahtBot · 2020-11-19T12:19:41Z

🐙 This pull request conflicts with the target branch and needs rebase.

_{Want to unsubscribe from rebase notifications on this pull request? Just convert this pull request to a "draft".}

laanwj · 2020-11-23T14:17:59Z

One outstanding issue is that there is either a bug, or a backwards incompatibility with the latest (2.1.0) version of mac_alias:

It's a bug. Looks like they tried to spread the expression over multiple lines but forgot to either add parenthesis around the tuple or \s.

https://github.com/al45tair/mac_alias/blob/master/mac_alias/alias.py#L282

Edit: created an upstream issue dmgbuild/mac_alias#9

RandyMcMillan · 2020-11-27T22:55:03Z

Concept ACK!

Rather than using OSX_QT_TRANSLATIONS which must be manually updated, and we forget to update anyway, i.e: bitcoin#19059, automatically find and copy available translations from the translations directory.

4 different levels of verbosity is overkill for a fairly simple script, which was always being run at 2 in any case.

We already require Python 3.5 or later

Rather than two lots of logic doing roughly the same thing, dependent on if you're compiling on Linux or macOS, combine the .DS store generation into macdeployqtplus. This also removes the -fancy and -volname options.

fanquake · 2020-11-30T07:40:37Z

Edit: created an upstream issue dmgbuild/mac_alias#9

Great, thanks. The upstream issue has been resolved with the release of mac_alias 2.1.1. I've added an additional commit to this PR to switch to that version in depends. I've also updated the PR description to remove no-longer-relevant info.

This PR should now be ready for review.

dergoegge

ACK b685f60 - Less and cleaner code looks good. I tested this with make deploy and everything still works + the popup during DMG generation is gone.

dergoegge · 2020-12-02T14:27:31Z

contrib/macdeploy/macdeployqtplus

@@ -16,9 +16,13 @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 #

-import subprocess, sys, re, os, shutil, stat, os.path, time
-from string import Template
+import plistlib


This import was only used by the -fancy option and could also be removed since it is now unused.

Good catch. However I'm going to merge this now, and we'll take care of the leftover import shortly.

willcl-ark

utACK. I don't have an Apple Developer account so can't get the SDK to test with I'm afraid.

Sjors · 2020-12-10T09:43:21Z

Tested (via master) on macOS and it seems to work.

We should add an instruction before make deploy to install the Python dependency:

pip3 install ds_store

Unused since bitcoin#20422, see: bitcoin#20422 (comment)

a68c7d5 depends: mac_alias 2.2.0 (fanquake) d8e2baf doc: Add explicit macdeployqtplus dependencies install step (Hennadii Stepanov) 013305d macdeploy: use Python 3.6 (fanquake) faf77c3 macdeploy: remove runHDIUtil in favor of directly calling subprocess.run (fanquake) 8bcfd58 macdeploy: remove existing PIVX-Core.dmg if present (fanquake) 023d3ca macdeploy: move qt_conf to where it's used (fanquake) 7cdb5bb macdeploy: consolidate .DS_Store generation (fanquake) 4da04d7 macdeploy: assume plistlib is available (fanquake) 56ab77a macdeploy: have a single level of logging output (fanquake) d111cdf macdeploy: remove add-resources argument (fanquake) 4312410 macdeploy: remove codesigning argument (fanquake) c2ee635 build: automatically determine macOS translations (fanquake) 1c44ecf scripts: filter more qt plugins we don't use in macdeployqtplus (fanquake) c854f78 scripts: misc cleanups in macdeployqtplus (fanquake) a3873ea scripts: use format() in macdeployqtplus (fanquake) a65bea5 scripts: add type annotations to macdeployqtplus (fanquake) ba179e5 build: Drop macports support (Ben Woosley) Pull request description: This is a companion to PIVX-Project#2272 that focuses on on the `.dmg` creation aspect of macOS builds (ie, `make deploy`). The following upstream PRs are backported here: - bitcoin#15175 - bitcoin#16477 - bitcoin#20422 - bitcoin#20890 - bitcoin#21658 Also worth mentioning: This drops support for MacPorts entirely, which has been antiquated and un-maintained for quite some time, and never actually used by any PIVX macOS build doc. ACKs for top commit: furszy: Tested using depends, ACK a68c7d5. random-zebra: utACK a68c7d5 and merging... Tree-SHA512: 3e9fa81a905ca3e90f07ff1213ec69dd1220a19a6a215f256ab67f2594476dc95e8fe88f15a1c9f3314b1757a7a2e5d8e6d7a790d85c117bf4236a3833757430

0a5723b macdeploy: cleanup .temp.dmg if present (fanquake) ecffe86 macdeploy: remove qt4 related code (fanquake) 639f064 macdeploy: select the plugins we need, rather than excluding those we don't (fanquake) 3d26b6b macdeploy: fix framework printing when passing -verbose (fanquake) dca6c90 macdeploy: remove unused plistlib import (fanquake) Pull request description: This includes [one followup](#20422 (comment)) and [one bug fix](3d26b6b) from #20422, as well as some simplifications to the `macdeployqtplus` code. ACKs for top commit: hebasto: ACK 0a5723b, tested on macOS Big Sur 11.4 (20F71, x86_64) + Homebrew's Qt 5.15.2. Tree-SHA512: cfad9505eacd32fe3a9d06eb13b2de0b6d2cad7b17778e90b503501cbf922e53d4e7f7f74952d1aed58410bdae9b0bb3248098583ef5b85689cb27d4dc06c029

Unused since bitcoin#20422, see: bitcoin#20422 (comment)

Unused since #20422, see: bitcoin/bitcoin#20422 (comment)

0a5723b macdeploy: cleanup .temp.dmg if present (fanquake) ecffe86 macdeploy: remove qt4 related code (fanquake) 639f064 macdeploy: select the plugins we need, rather than excluding those we don't (fanquake) 3d26b6b macdeploy: fix framework printing when passing -verbose (fanquake) dca6c90 macdeploy: remove unused plistlib import (fanquake) Pull request description: This includes [one followup](bitcoin#20422 (comment)) and [one bug fix](bitcoin@3d26b6b) from bitcoin#20422, as well as some simplifications to the `macdeployqtplus` code. ACKs for top commit: hebasto: ACK 0a5723b, tested on macOS Big Sur 11.4 (20F71, x86_64) + Homebrew's Qt 5.15.2. Tree-SHA512: cfad9505eacd32fe3a9d06eb13b2de0b6d2cad7b17778e90b503501cbf922e53d4e7f7f74952d1aed58410bdae9b0bb3248098583ef5b85689cb27d4dc06c029

hebasto · 2021-09-14T10:13:28Z

It appears this PR breaks make deploy on macOS 10.14.6 (18G9323):

$ make deploy
...
created: /Users/hebasto/bitcoin/Bitcoin-Core.temp.dmg
+ Applying fancy settings +
+ Finalizing .dmg disk image +
"disk23" ejected.
hdiutil: convert failed - Resource temporarily unavailable
Traceback (most recent call last):
  File "/Users/hebasto/bitcoin/./contrib/macdeploy/macdeployqtplus", line 687, in <module>
    run(["hdiutil", "convert", tempname, "-format", "UDZO", "-o", appname, "-imagekey", "zlib-level=9"], check=True, universal_newlines=True)
  File "/usr/local/Cellar/python@3.9/3.9.7/Frameworks/Python.framework/Versions/3.9/lib/python3.9/subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['hdiutil', 'convert', 'Bitcoin-Core.temp.dmg', '-format', 'UDZO', '-o', 'Bitcoin-Core', '-imagekey', 'zlib-level=9']' returned non-zero exit status 1.
make: *** [Makefile:1282: Bitcoin-Core.dmg] Error 1

Newer macOS versions—10.15 and 11.x—are not affected.

fanquake · 2021-09-16T09:40:29Z

hdiutil: convert failed - Resource temporarily unavailable

This looks like an intermittent issue, and I would be somewhat surprised if the changes here worked on macOS 10.15 but not 10.14. I don't think there's anything we're doing that would be specific to the OS, or a certain version of a tool, like hdiutil etc.

Does make deploy work on 10.14 with this change reverted? Without any more info there's not a lot to go on here.

hebasto · 2021-09-16T09:45:17Z

Does make deploy work on 10.14 with this change reverted?

Yes, it does.

I think @jarolrod could also confirm such behavior?

I would be somewhat surprised if the changes here worked on macOS 10.15 but not 10.14.

Me too.

Unused since #20422, see: bitcoin/bitcoin#20422 (comment)

fanquake added macOS Build system labels Nov 19, 2020

DrahtBot added the Needs rebase label Nov 19, 2020

fanquake force-pushed the macdeploy_unify branch from 35bb751 to cdc84c0 Compare November 19, 2020 14:44

fanquake removed the Needs rebase label Nov 19, 2020

laanwj added the Upstream label Nov 23, 2020

bitcoin deleted a comment Nov 28, 2020

fanquake added 10 commits November 30, 2020 14:54

build: automatically determine macOS translations

4d70d3d

Rather than using OSX_QT_TRANSLATIONS which must be manually updated, and we forget to update anyway, i.e: bitcoin#19059, automatically find and copy available translations from the translations directory.

macdeploy: remove codesigning argument

464b34d

macdeploy: remove add-resources argument

827d382

macdeploy: have a single level of logging output

0ab4018

4 different levels of verbosity is overkill for a fairly simple script, which was always being run at 2 in any case.

macdeploy: assume plistlib is available

32347cd

We already require Python 3.5 or later

macdeploy: consolidate .DS_Store generation

6390a04

Rather than two lots of logic doing roughly the same thing, dependent on if you're compiling on Linux or macOS, combine the .DS store generation into macdeployqtplus. This also removes the -fancy and -volname options.

macdeploy: move qt_conf to where it's used

ccb0325

macdeploy: remove existing Bitcoin-Core.dmg if present

adaa262

macdeploy: remove runHDIUtil in favor of directly calling subprocess.run

a42aa94

macdeploy: use Python 3.6

5d2cbdf

fanquake force-pushed the macdeploy_unify branch from cdc84c0 to 5d2cbdf Compare November 30, 2020 07:37

build: mac_alias 2.1.1

b685f60

dergoegge approved these changes Dec 2, 2020

View reviewed changes

willcl-ark reviewed Dec 2, 2020

View reviewed changes

fanquake merged commit 16b31cc into bitcoin:master Dec 8, 2020

sidhujag pushed a commit to syscoin/syscoin that referenced this pull request Dec 8, 2020

Merge bitcoin#20422: build: mac deployment unification

c3efae2

fanquake mentioned this pull request Dec 10, 2020

build: Replace genisoimage with xorriso #20470

Merged

Sjors mentioned this pull request Dec 11, 2020

guix: Build support for macOS #17920

Merged

fanquake deleted the macdeploy_unify branch February 9, 2021 06:35

Fuzzbawls mentioned this pull request May 25, 2021

[Build] [Depends] Unify macOS deploy target PIVX-Project/PIVX#2400

Merged

fanquake added a commit to fanquake/bitcoin that referenced this pull request Jun 9, 2021

macdeploy: remove unused plistlib import

dca6c90

Unused since bitcoin#20422, see: bitcoin#20422 (comment)

fanquake mentioned this pull request Jun 9, 2021

macdeploy: minor fixups and simplifications #22199

Merged

josibake pushed a commit to josibake/bitcoin that referenced this pull request Jul 21, 2021

macdeploy: remove unused plistlib import

7cf7fd0

Unused since bitcoin#20422, see: bitcoin#20422 (comment)

hebasto pushed a commit to hebasto/gui-qml that referenced this pull request Jul 22, 2021

macdeploy: remove unused plistlib import

2d5ce10

Unused since #20422, see: bitcoin/bitcoin#20422 (comment)

hebasto mentioned this pull request Sep 16, 2021

build: set OSX_MIN_VERSION to 10.15 #22993

Merged

janus pushed a commit to BitgesellOfficial/bitgesell that referenced this pull request Oct 29, 2021

macdeploy: remove unused plistlib import

d74124b

Unused since #20422, see: bitcoin/bitcoin#20422 (comment)

lyricidal mentioned this pull request Nov 1, 2021

[Upstream] [Build] [Depends] Unify macOS deploy target PRCYCoin/PRCYCoin#138

Merged

hebasto mentioned this pull request Dec 13, 2021

Cannot make deploy Mac OS dmg #23754

Closed

bitcoin locked and limited conversation to collaborators Oct 30, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

build: mac deployment unification #20422

build: mac deployment unification #20422

Uh oh!

fanquake commented Nov 19, 2020 •

edited

Loading

Uh oh!

laanwj commented Nov 19, 2020

Uh oh!

DrahtBot commented Nov 19, 2020

Uh oh!

laanwj commented Nov 23, 2020 •

edited

Loading

Uh oh!

RandyMcMillan commented Nov 27, 2020

Uh oh!

fanquake commented Nov 30, 2020

Uh oh!

dergoegge left a comment

Uh oh!

dergoegge Dec 2, 2020

Uh oh!

fanquake Dec 8, 2020

Uh oh!

willcl-ark left a comment

Uh oh!

Sjors commented Dec 10, 2020 •

edited

Loading

Uh oh!

hebasto commented Sep 14, 2021

Uh oh!

fanquake commented Sep 16, 2021

Uh oh!

hebasto commented Sep 16, 2021 •

edited

Loading

Uh oh!

Uh oh!

build: mac deployment unification #20422

build: mac deployment unification #20422

Uh oh!

Conversation

fanquake commented Nov 19, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

laanwj commented Nov 19, 2020

Uh oh!

DrahtBot commented Nov 19, 2020

Uh oh!

laanwj commented Nov 23, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

RandyMcMillan commented Nov 27, 2020

Uh oh!

fanquake commented Nov 30, 2020

Uh oh!

dergoegge left a comment

Choose a reason for hiding this comment

Uh oh!

dergoegge Dec 2, 2020

Choose a reason for hiding this comment

Uh oh!

fanquake Dec 8, 2020

Choose a reason for hiding this comment

Uh oh!

willcl-ark left a comment

Choose a reason for hiding this comment

Uh oh!

Sjors commented Dec 10, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hebasto commented Sep 14, 2021

Uh oh!

fanquake commented Sep 16, 2021

Uh oh!

hebasto commented Sep 16, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

fanquake commented Nov 19, 2020 •

edited

Loading

laanwj commented Nov 23, 2020 •

edited

Loading

Sjors commented Dec 10, 2020 •

edited

Loading

hebasto commented Sep 16, 2021 •

edited

Loading