Create reproducible artifacts #6322
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Enable class file timestamp normalization by bumping to latest version of maven-bundle-plugin - See changes at apache/felix-dev@957e5c9, available from plugin 5.1.3 and later - Make build artifacts bit-for-bit reproducible by normalizing timestamp - maven-bundle-plugin reacts to this value with desirable side-effect of removing timestampts from OSGi bundle/jar manifests and pom.properties files - Uses the same normalization schemes as Gradle and Bnd: - https://github.com/gradle/gradle/blob/v7.6.0/subprojects/core/src/main/java/org/gradle/api/internal/file/archive/ZipCopyAction.java#L57 - https://github.com/bndtools/bnd/blame/a6aed533455f43e8a3802f847e8f6548a868f922/biz.aQute.bndlib/src/aQute/bnd/osgi/Jar.java#L71-L87
1247939 to
ade03d2
Compare
|
@eamonnmcmanus |
|
Looking good! Thanks very much for doing this. Because the Source Of Truth for Guava is actually Google's internal repository, we'll make the change there and it will appear as a separate PR that will close this one. You will be duly credited, of course. |
copybara-service bot
pushed a commit
that referenced
this pull request
Feb 9, 2023
As with Gradle and Bnd, timestamps in jars are [normalized](https://github.com/gradle/gradle/blob/v7.6.0/subprojects/core/src/main/java/org/gradle/api/internal/file/archive/ZipCopyAction.java#L57) to 1980 February 1st CET. Closes #6322. RELNOTES=n/a PiperOrigin-RevId: 508455012
9 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Configures Maven to create bit-by-bit reproducible artifacts.
As with Gradle and Bnd, timestamps in jars are normalized to 1980 February 1st CET. This is not without controversy, as shown by this example issue: FasterXML/jackson-databind#3680
An alternative albeit slightly more complex approach is to update
project.build.outputTimestampat release time, as shown in this FAQ: https://maven.apache.org/guides/mini/guide-reproducible-builds.html#faqFinally, for reference purposes I've used the excellent
diffoscopetool to create a visual diff ofguava-31.1-jrerelease vs. the same code built with this change. Openindex.htmlin this archive: guava-diff.zipSteps to reproduce the above:
$ git checkout v31.1$ git cherry-pick #### the commit of this PR$ mvn clean install -U -DskipTests=true$ curl https://repo1.maven.org/maven2/com/google/guava/guava/31.1-jre/guava-31.1-jre.jar --output ~/Downloads/guava-31.1-jre.jar # sha1sum should be 60458f877d055d0c9114d9e1a2efb737b4bc282c$ diffoscope --exclude \*.class --html-dir guava-diff ~/Downloads/guava-31.1-jre.jar guava/target/guava-31.1-jre.jarThe important things to notice in the diff:
zipinfooutput shows normalized timestamps in the zip catalog on RHSMETA-INF/MANIFEST.MF: ProblematicBnd-LastModifiedandBuilt-Byattributes are gone. DifferentToolversion is used bymaven-bundle-plugin.META-INF/**/pom.properties: Problematic comment with timestamp is goneNote that in step 5 I excluded class files from analysis; that's because
diffoscopedelegates to a decompiler which seemed confused about inlined Strings and lambdas, creating false-positives. I suspect this is a difference between my local JDK and/or vendor vs. the released artifact.NB: There appear to be other past changes related to reproducible builds: #3534, #3681 and #3686