Switch Travis to Trusty #7920

theuni · 2016-04-21T03:35:27Z

I worked with Travis to get the cache flag on my personal repo for testing, and I'm happy to report that all works great (after a day of churning builds).

For some background: Travis has enabled caching for their new infrastructure, but the feature is brand new and not yet generally available. For now it requires a special flag (set on their end) on your repo. Once they're satisfied with its stability, the flag will be granted to everyone. Note that the flag comes with the caveat of invalidating all current caches. That's not a problem for us, other than slowing things down for a few hours. I was primarily worried about unintended side-effects of opting in, so I added it on my own repo for testing before requesting it here.

If there's no reason to delay our move, I'll request the flag for this repo tomorrow (4/21).

Once that's take care of, we can merge this for Trusty goodness. On the day that Xenial is scheduled to be released :p.

Next step is c++11, which should be painless to enable now.

Note that this includes #7792, otherwise the OSX build fails. @laanwj I'm happy to rebase if you'd prefer to merge that first.

theuni · 2016-04-21T04:52:00Z

Note that the build failed because we currently have a (different) special flag on our repo that force-routes us to the legacy infrastructure for caching. This change will disable that in favor of the new testing flag.

(It may sound convoluted, but this change moves us one step closer towards being a normal repo. Once the caching feature is out of beta, there will be nothing special about us.)

maflcko · 2016-04-21T06:02:03Z

.travis.yml

    - compiler: ": bitcoind"
-      env: HOST=x86_64-unknown-linux-gnu PACKAGES="bc python-zmq" PPA="ppa:chris-lea/zeromq" DEP_OPTS="NO_QT=1 NO_UPNP=1 DEBUG=1" RUN_TESTS=true GOAL="install" BITCOIN_CONFIG="--enable-zmq --enable-glibc-back-compat --enable-reduce-exports CPPFLAGS=-DDEBUG_LOCKORDER"
+      env: HOST=x86_64-unknown-linux-gnu PACKAGES="bc python-zmq python3-zmq" DEP_OPTS="NO_QT=1 NO_UPNP=1 DEBUG=1" RUN_TESTS=true GOAL="install" BITCOIN_CONFIG="--enable-zmq --enable-glibc-back-compat --enable-reduce-exports CPPFLAGS=-DDEBUG_LOCKORDER"


Mind to share why the py2 and py3 packages of zmq are needed?

@theuni Jup, let's keep it py2 only for now, if possible.

Sure, will remove.

maflcko · 2016-04-21T06:02:16Z

Awesome! Concept ACK!

theuni · 2016-04-21T06:08:14Z

@MarcoFalke Just for the sake of being future-proof. It only uses the python2 one at the moment, but I believe the (your?) intention is to move the rpc tests to python3. Just doing python2 for now is fine by me if that's what you'd prefer. I was just saving some future head-scratching when builds fail after the switch.

Edit: Looking at the diff, I now see why you asked. Python was a headache when testing the migration from precise->trusty. I had a theory at one point that python3 was being forced for the rpc tests, which obviously turned out to not be the case. But I left it there since that's where we're headed.

gmaxwell · 2016-04-21T17:14:15Z

Thanks for working on this.

This fixes the gitian MacOSX build, it was broken in bitcoin#7723. The patch to `native_mac_alias` should probably make it upstream.

theuni · 2016-04-21T17:41:45Z

Dropped the python3-zmq packages.

theuni · 2016-04-23T05:22:03Z

This was delayed because there's a little extra work needed in switching away from our old setup. ETA given from Travis is Monday.

All trusty compilers work ok with this now, and it shaves a few minutes off of build time

These were only in place because of the old precice mingw toolchain

This was doing more harm than good. The original intention was to speed up builds, since a PR's ccache results will be thrown away anyway. However, each PR maintains its own cache, so disabling writes means that subsequent pushes don't benefit from the fresh cache. This is significant when (for example) many headers are touched in a PR, then the PR is updated. With this change, the updated PR will take advantage of the cache generated during the PR's previous build.

These builds take longer than they did on precise. Disable them now to keep things working, we can re-enable them after experimentation

theuni · 2016-04-26T06:56:35Z

Ok, finally all green and cache-enabled.

I disabled a few qt builds because they seem to take longer on this infrastructure than before. We can investigate and turn them back on asap.

For now, until this is merged, PRs will be unusually slow, so it's probably best to go ahead with this now and work through the regressions.

theuni · 2016-04-26T07:22:39Z

To expand on the "PRs will be unusually slow" bit above:

In removing the special bits from our repo, all caches have been invalidated. In theory, precise caches will be regenerated for subsequent PRs, but I'm nervous about that actually happening seamlessly on the deprecated platform.

Even if it does work without issue, PRs will be slow until something is pushed into master. When a PR is created, the build cache is taken from its source branch (master), and copied into the PR. From there, the PR updates its own cache with each update.

Right now we have no master cache, and no PR caches. So until there's something to pull from master, each PR will be rebuilding from scratch. Also, once we do pull the trigger on this, it will essentially invalidate all caches again, because all dependencies will be rebuilt for the new toolchains.

tl;dr: Need to push something into master asap to avoid slow builds all day. May as well push this one, to avoid a repeat.

maflcko · 2016-04-26T08:01:37Z

ACK on merging today.

btcdrak · 2016-04-26T11:15:10Z

ACK

jonasschnelli · 2016-04-26T11:18:56Z

ACK 174023c

a33b7c9 travis: temporarily disable qt to avoid timeouts (Cory Fields) 174023c travis: Don't disable writing ccache for pull-requests (Cory Fields) cf77fcd travis: drop MAKEJOBS=2 for windows compilers (Cory Fields) 9267a47 depends: enable pre-compiled headers for qt (Cory Fields) 06fdffd travis: switch to Trusty (Cory Fields) a6666b2 depends: mac deploy Py3 compatibility (Wladimir J. van der Laan)

btcdrak · 2016-04-26T11:27:29Z

@theuni Does this have to be backported as well?

theuni · 2016-04-26T15:59:25Z

@btcdrak Yes, the same issues apply to the other branches.

Github-Pull: bitcoin#7920 Rebased-From: 06fdffd, 9267a47, cf77fcd, 174023c, a33b7c9

Github-Pull: bitcoin#7920 Rebased-From: 06fdffd, 9267a47, cf77fcd, 174023c, a33b7c9 Cherry-pick-From: 564aaa2

Github-Pull: bitcoin#7920 Rebased-From: 06fdffd, 9267a47, cf77fcd, 174023c, a33b7c9

Github-Pull: bitcoin#7920 Rebased-From: 06fdffd, 9267a47, cf77fcd, 174023c, a33b7c9 Cherry-pick-From: 564aaa2

maflcko reviewed Apr 21, 2016
View reviewed changes

maflcko mentioned this pull request Apr 21, 2016

[qa] Switch to py3 #7814

Merged

1 task

jonasschnelli added the Build system label Apr 21, 2016

theuni force-pushed the travis-trusty branch from 7797435 to 06fdffd Compare April 21, 2016 17:35

laanwj and others added 2 commits April 21, 2016 13:40

depends: mac deploy Py3 compatibility

a6666b2

This fixes the gitian MacOSX build, it was broken in bitcoin#7723. The patch to `native_mac_alias` should probably make it upstream.

travis: switch to Trusty

06fdffd

laanwj mentioned this pull request Apr 25, 2016

depends: mac deploy Py3 compatibility (fixes macosx gitian build) #7792

Closed

depends: enable pre-compiled headers for qt

9267a47

All trusty compilers work ok with this now, and it shaves a few minutes off of build time

theuni force-pushed the travis-trusty branch from 7023e72 to cf77fcd Compare April 26, 2016 03:43

travis: drop MAKEJOBS=2 for windows compilers

cf77fcd

These were only in place because of the old precice mingw toolchain

theuni force-pushed the travis-trusty branch 2 times, most recently from 0eb0a04 to a33b7c9 Compare April 26, 2016 06:19

theuni added 2 commits April 26, 2016 02:25

travis: temporarily disable qt to avoid timeouts

a33b7c9

These builds take longer than they did on precise. Disable them now to keep things working, we can re-enable them after experimentation

theuni changed the title ~~Do not merge yet: Switch Travis to Trusty~~ Switch Travis to Trusty Apr 26, 2016

theuni mentioned this pull request Apr 26, 2016

Intermittent travis issues win32/win64 #7924

Closed

laanwj merged commit a33b7c9 into bitcoin:master Apr 26, 2016

maflcko pushed a commit to maflcko/bitcoin-core that referenced this pull request Apr 27, 2016

travis: switch to Trusty

564aaa2

Github-Pull: bitcoin#7920 Rebased-From: 06fdffd, 9267a47, cf77fcd, 174023c, a33b7c9

braydonf pushed a commit to braydonf/bitcoin that referenced this pull request May 12, 2016

travis: switch to Trusty

43b9b4e

Github-Pull: bitcoin#7920 Rebased-From: 06fdffd, 9267a47, cf77fcd, 174023c, a33b7c9 Cherry-pick-From: 564aaa2

braydonf pushed a commit to braydonf/bitcoin that referenced this pull request Jun 2, 2016

travis: switch to Trusty

6c44620

Github-Pull: bitcoin#7920 Rebased-From: 06fdffd, 9267a47, cf77fcd, 174023c, a33b7c9 Cherry-pick-From: 564aaa2

thokon00 pushed a commit to faircoin/faircoin that referenced this pull request Jun 28, 2016

travis: switch to Trusty

83e80c4

Github-Pull: bitcoin#7920 Rebased-From: 06fdffd, 9267a47, cf77fcd, 174023c, a33b7c9

maflcko mentioned this pull request Sep 22, 2016

[travis] cross-mac: explicitly enable gui #8791

Merged

braydonf pushed a commit to braydonf/bitcoin that referenced this pull request Oct 4, 2016

travis: switch to Trusty

d53ca25

Github-Pull: bitcoin#7920 Rebased-From: 06fdffd, 9267a47, cf77fcd, 174023c, a33b7c9 Cherry-pick-From: 564aaa2

nomnombtc pushed a commit to nomnombtc/bitcoin that referenced this pull request Nov 22, 2016

travis: switch to Trusty

f179536

Github-Pull: bitcoin#7920 Rebased-From: 06fdffd, 9267a47, cf77fcd, 174023c, a33b7c9 Cherry-pick-From: 564aaa2

bitcoin locked as resolved and limited conversation to collaborators Sep 8, 2021

Switch Travis to Trusty #7920

Switch Travis to Trusty #7920

Uh oh!

Conversation

theuni commented Apr 21, 2016

Uh oh!

theuni commented Apr 21, 2016

Uh oh!

maflcko Apr 21, 2016

Choose a reason for hiding this comment

Uh oh!

maflcko Apr 21, 2016

Choose a reason for hiding this comment

Uh oh!

theuni Apr 21, 2016

Choose a reason for hiding this comment

Uh oh!

maflcko commented Apr 21, 2016

Uh oh!

theuni commented Apr 21, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gmaxwell commented Apr 21, 2016

Uh oh!

theuni commented Apr 21, 2016

Uh oh!

theuni commented Apr 23, 2016

Uh oh!

theuni commented Apr 26, 2016

Uh oh!

theuni commented Apr 26, 2016

Uh oh!

maflcko commented Apr 26, 2016

Uh oh!

btcdrak commented Apr 26, 2016

Uh oh!

jonasschnelli commented Apr 26, 2016

Uh oh!

btcdrak commented Apr 26, 2016

Uh oh!

theuni commented Apr 26, 2016

Uh oh!

Uh oh!

theuni commented Apr 21, 2016 •

edited

Loading