Skip to content

Conversation

maflcko
Copy link
Member

@maflcko maflcko commented Jan 4, 2016

No description provided.

@maflcko
Copy link
Member Author

maflcko commented Jan 4, 2016

@jonasschnelli I don't want to break your debian box, so would you mind testing this after updating your gitian-builder to the latest version?

@laanwj
Copy link
Member

laanwj commented Jan 4, 2016

Ping @theuni

Concept ACK however:

There's a potential determinism problem with making the datetime depend on the bitcoin repository: the dependencies are built with datetime set to a single commit date, then cached, then next time they'll be reused. If the dependencies embed the datetime anywhere, different builders can get different result depending on when they built the dependencies.

@maflcko
Copy link
Member Author

maflcko commented Jan 4, 2016

Appears to be working:

$ curl https://bitcoin.jonasschnelli.ch/pulls/7256/bitcoin-0.12.99-linux64.tar.gz -s | tar ztv | head -3
drwxr-xr-x root/root         0 2015-06-01 02:00 bitcoin-0.12.99/
drwxr-xr-x root/root         0 2015-06-01 02:00 bitcoin-0.12.99/bin/
-rwxr-xr-x root/root   2470520 2015-06-01 02:00 bitcoin-0.12.99/bin/bench_bitcoin
$ curl https://bitcoin.jonasschnelli.ch/pulls/7283/bitcoin-0.12.99-linux64.tar.gz -s | tar ztv | head -3
drwxr-xr-x root/root         0 2016-01-04 12:11 bitcoin-0.12.99/
drwxr-xr-x root/root         0 2016-01-04 12:11 bitcoin-0.12.99/bin/
-rwxr-xr-x root/root   2470520 2016-01-04 12:11 bitcoin-0.12.99/bin/bench_bitcoin

@jonasschnelli
Copy link
Contributor

Agree with @laanwj: would it be possible to build and cache the dependencies still with a fixed time?
Maybe first someone needs to check if the datetime results/gets included somehow in the dependencies.

@maflcko maflcko force-pushed the MarcoFalke-2016-gitianTimeDefault branch 2 times, most recently from af1b8b2 to 2222962 Compare January 4, 2016 13:48
@laanwj
Copy link
Member

laanwj commented Jan 5, 2016

BTW: Ugh. I think it was a mistake to add the build date there. It's worthless at most, and deceptive at the least.
If we want the date of last commit in there, would make sense to add it after the "Client version".

@theuni
Copy link
Member

theuni commented Jan 5, 2016

@laanwj I'm all for removing the build date and git commit from tagged releases. That would also mean that we could merge the signing steps, since the only reason they're staggered is the changed commit hash in the binary.

@jonasschnelli I believe we've already removed all time sources from the depends.

@laanwj I believe that would already be the case?

@laanwj
Copy link
Member

laanwj commented Jan 7, 2016

@laanwj I believe that would already be the case?

Faketime prevents that at the moment by setting the date/time stamp to a fixed value

@maflcko
Copy link
Member Author

maflcko commented Jan 7, 2016

Then, wouldn't stuff like be65628 break this as well?

Anyway, depends could use a different (static) date other than reference_datetime.

@laanwj
Copy link
Member

laanwj commented Jan 18, 2016

You're right, but be65628 is a one-time change. So dependencies built before and after that may differ. The problem with this is that the time/date will change every time.

Not sure it is an issue, but as this is a non-critical detail I'd like to be sure it doesn't cause problems down the line before merging it.

@maflcko maflcko force-pushed the MarcoFalke-2016-gitianTimeDefault branch from 2222962 to fa7dc3b Compare February 1, 2016 15:16
@laanwj
Copy link
Member

laanwj commented Feb 4, 2016

Needs rebase

@maflcko maflcko force-pushed the MarcoFalke-2016-gitianTimeDefault branch 2 times, most recently from 297fd3f to fa34926 Compare February 5, 2016 21:59
@maflcko maflcko force-pushed the MarcoFalke-2016-gitianTimeDefault branch from fa34926 to fa6127f Compare February 26, 2016 15:45
@maflcko
Copy link
Member Author

maflcko commented Feb 27, 2016

@laanwj I've been running gitian builder to see if the results match:

yml source target commit cache populated by result
e7ea5db e7ea5db e7ea5db 041a0f82a0c4... bitcoin-0.12.99-linux32.tar.gz
this pull's commit this pull's commit e7ea5db 49fd8bbb3fcc... bitcoin-0.12.99-linux32.tar.gz
this pull's commit this pull's commit this pull's commit 49fd8bbb3fcc... bitcoin-0.12.99-linux32.tar.gz
e7ea5db e7ea5db this pull's commit 041a0f82a0c4... bitcoin-0.12.99-linux32.tar.gz

So this appears to indicate that the result is independent from the cache of which commit created the cache.

@maflcko maflcko force-pushed the MarcoFalke-2016-gitianTimeDefault branch from fa6127f to fa58c76 Compare March 1, 2016 18:47
@maflcko
Copy link
Member Author

maflcko commented Mar 14, 2016

@theuni Do you think this is ready to merge?

@theuni
Copy link
Member

theuni commented Apr 4, 2016

Sorry for the delay, testing this.

@theuni
Copy link
Member

theuni commented Apr 4, 2016

Yes, after taking a closer look, unfortunately this will not play nice with depends. Mostly because of the timestamps inside of tarballs, but also .a's and other problematic formats. Note that the end-results may or may not be the same, but the intermediates would differ.

We can revisit this later though, as options for tar/binutils/ld/etc have been added recently that reliably purge this data.

The only realistic thing I see that we can do here for now without a major overhaul would be to set the faketime once to a hard-coded date for depends, so that it remains the same regardless of the commit used, then reset for the bitcoin build itself.

@maflcko
Copy link
Member Author

maflcko commented Apr 5, 2016

set the faketime once to a hard-coded date for depends

@theuni Something like the last commit I pushed (untested)?

@theuni
Copy link
Member

theuni commented Apr 5, 2016

@MarcoFalke Yep, I think that approach should be fine.

@maflcko maflcko force-pushed the MarcoFalke-2016-gitianTimeDefault branch from 218029a to fad0d5a Compare April 8, 2016 18:39
@maflcko maflcko force-pushed the MarcoFalke-2016-gitianTimeDefault branch from fad0d5a to fa42a67 Compare April 10, 2016 20:58
@maflcko
Copy link
Member Author

maflcko commented Apr 10, 2016

Not sure why gitian fails on the windows build...

[gitian-builder]$ tail -8 var/build.log
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking whether to enable maintainer-specific portions of Makefiles... yes
checking whether make supports nested variables... (cached) yes
checking whether the C++ compiler works... no
configure: error: in `/home/ubuntu/build/bitcoin':
configure: error: C++ compiler cannot create executables
See `config.log' for more details

[gitian-builder]$ ./libexec/on-target tail -5 ./build/bitcoin/config.log
#define PACKAGE_STRING "Bitcoin Core 0.12.99"
#define PACKAGE_BUGREPORT "https://github.com/bitcoin/bitcoin/issues"
#define PACKAGE_URL "https://bitcoincore.org/"

configure: exit 77

@arowser
Copy link
Contributor

arowser commented May 25, 2016

Can one of the admins verify this patch?

@laanwj
Copy link
Member

laanwj commented May 30, 2016

I think it would be better to remove use of the build date in our binary completely, making the use of faketime unnecessary. We don't have control over that for the dependencies but we do for our own code.

If you want the date/time of the last commit somewhere in the binary there are better ways to do that (e.g. through genbuild.sh) than a faketime.

@maflcko
Copy link
Member Author

maflcko commented May 30, 2016

The motivation for this pull was to avoid regular "bump date" pulls such as #7251 (comment).

The build date was removed in #7732 and I don't think we should add the commit date to the gui.

@maflcko
Copy link
Member Author

maflcko commented May 30, 2016

If this pull gets closes, it might be worth to update https://github.com/bitcoin/bitcoin/blob/950be19727a581970591d8f8138dfe4725750382/doc/release-process.md#bitcoin-maintainersrelease-engineers-update-version-in-sources and mention to bump the date in the same go.

@maflcko maflcko force-pushed the MarcoFalke-2016-gitianTimeDefault branch 2 times, most recently from 68146f6 to fa42a67 Compare June 5, 2016 13:02
@laanwj
Copy link
Member

laanwj commented Jun 9, 2016

There's another place where this matters: the times/dates in the .zip files and .tar.gz archives of the binary distribution. I guess that makes it worth doing this.

The build date was removed in #7732

Only from the GUI though. It's still logged in two places:

src/init.cpp:    LogPrintf("Bitcoin version %s (%s)\n", FormatFullVersion(), CLIENT_DATE);
src/wallet/rpcdump.cpp:    file << strprintf("# Wallet dump created by Bitcoin %s (%s)\n", CLIENT_BUILD, CLIENT_DATE);

@laanwj
Copy link
Member

laanwj commented Jun 9, 2016

If this pull gets closes, it might be worth to update

Well my point was more 'why would we care about this'. If there is no reason to care about this date at all, there's no point in bumping it either. E.g. after making sure it doesn't get shown in the GUI or log anymore.
But I suppose the dates in archives are a reason to care that it is somewhat up to date. It looks stupid to have files from 1970.

@laanwj laanwj merged commit fa42a67 into bitcoin:master Jun 9, 2016
laanwj added a commit that referenced this pull request Jun 9, 2016
fa42a67 [gitian] hardcode datetime for depends (MarcoFalke)
fa58c76 [gitian] Default reference_datetime to commit author date (MarcoFalke)
@maflcko maflcko deleted the MarcoFalke-2016-gitianTimeDefault branch June 9, 2016 11:02
zkbot pushed a commit to zcash/zcash that referenced this pull request Oct 17, 2016
Upstream gitian updates

This PR pulls in all gitian-related PRs that have been merged upstream since 0.11.2. The only ones I left out were documentation-only PRs, because we removed `doc/gitian-building.md` at some point. Here are the commits applied here, in the order shown in `git log` (ie. last to first):

- bitcoin/bitcoin#7283
  - fa42a67
  - fa58c76
- bitcoin/bitcoin#8175
  - 74c1347
- bitcoin/bitcoin#8167
  - 7e7eb27
  - ad38204
  - b676f38
- bitcoin/bitcoin#7776
  - f063863
- bitcoin/bitcoin#7424
  - a81c87f ~ we already partly applied
  - a8ce872
  - f3d3eaf ~ we already partly applied
  - 475813b
  - ~~cd27bf5~~ X we already applied
- bitcoin/bitcoin#7060
  - 3b468a0 ~ we removed doc/gitian-building.md
  - ~~99fda26~~ X we removed doc/gitian-building.md
- bitcoin/bitcoin#7251
  - fa09562
- bitcoin/bitcoin#6900
  - ~~2cecb24~~ X we removed doc/gitian-building.md
  - 957c0fd
  - 2e31d74
  - ~~0b416c6~~ X we removed QT
  - 9f251b7
- bitcoin/bitcoin#6854
  - 579b863 ~ we already partly applied

Part of #540
codablock pushed a commit to codablock/dash that referenced this pull request Dec 22, 2017
…hor date

fa42a67 [gitian] hardcode datetime for depends (MarcoFalke)
fa58c76 [gitian] Default reference_datetime to commit author date (MarcoFalke)
andvgal pushed a commit to energicryptocurrency/gen2-energi that referenced this pull request Jan 6, 2019
…hor date

fa42a67 [gitian] hardcode datetime for depends (MarcoFalke)
fa58c76 [gitian] Default reference_datetime to commit author date (MarcoFalke)
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Sep 8, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants