Skip to content

Followup for #18331 "build: Use git archive as source tarball" #18547

New issue
New issue
Closed
#18556
Closed
Followup for #18331 "build: Use git archive as source tarball"#18547
#18556
Labels
Build system
@dongcarl

Description

@dongcarl
dongcarl
opened on Apr 6, 2020

After the merge of #18331, there are now two ways to create source tarballs.

  1. make dist
  2. git archive

It would seem that both methods are being used in our Gitian descriptor.

First, we run make dist to create a bootstrapped tarball:

bitcoin/contrib/gitian-descriptors/gitian-linux.yml

Lines 143 to 148 in c5966a8

# Create the release tarball using (arbitrarily) the first host
./autogen.sh
CONFIG_SITE=${BASEPREFIX}/$(echo "${HOSTS}" | awk '{print $1;}')/share/config.site ./configure --prefix=/
make dist
SOURCEDIST=$(echo bitcoin-*.tar.gz)
DISTNAME=${SOURCEDIST/%.tar.gz}

Which is used by each host to build the binary tarballs in the for loop:

bitcoin/contrib/gitian-descriptors/gitian-linux.yml

Lines 154 to 191 in c5966a8

# Extract the release tarball into a dir for each host and build
for i in ${HOSTS}; do
export PATH=${BASEPREFIX}/${i}/native/bin:${ORIGPATH}
if [ "${i}" = "riscv64-linux-gnu" ]; then
# Workaround for https://bugs.launchpad.net/ubuntu/+source/gcc-8-cross-ports/+bug/1853740
# TODO: remove this when no longer needed
HOST_LDFLAGS="${HOST_LDFLAGS_BASE} -Wl,-z,noexecstack"
else
HOST_LDFLAGS="${HOST_LDFLAGS_BASE}"
fi
mkdir -p distsrc-${i}
cd distsrc-${i}
INSTALLPATH="${PWD}/installed/${DISTNAME}"
mkdir -p ${INSTALLPATH}
tar --strip-components=1 -xf ../$SOURCEDIST
# Workaround for tarball not building with the bare tag version
echo '#!/bin/true' >share/genbuild.sh
mkdir src/obj
cp ../src/obj/build.h src/obj/
CONFIG_SITE=${BASEPREFIX}/${i}/share/config.site ./configure --prefix=/ --disable-ccache --disable-maintainer-mode --disable-dependency-tracking ${CONFIGFLAGS} CFLAGS="${HOST_CFLAGS}" CXXFLAGS="${HOST_CXXFLAGS}" LDFLAGS="${HOST_LDFLAGS}"
make ${MAKEOPTS}
make ${MAKEOPTS} -C src check-security
make ${MAKEOPTS} -C src check-symbols
make install DESTDIR=${INSTALLPATH}
cd installed
find . -name "lib*.la" -delete
find . -name "lib*.a" -delete
rm -rf ${DISTNAME}/lib/pkgconfig
find ${DISTNAME}/bin -type f -executable -print0 | xargs -0 -n1 -I{} ../contrib/devtools/split-debug.sh {} {} {}.dbg
find ${DISTNAME}/lib -type f -print0 | xargs -0 -n1 -I{} ../contrib/devtools/split-debug.sh {} {} {}.dbg
cp ../../README.md ${DISTNAME}/
find ${DISTNAME} -not -name "*.dbg" | sort | tar --mtime="$REFERENCE_DATETIME" --no-recursion --mode='u+rw,go+r-w,a+X' --owner=0 --group=0 -c -T - | gzip -9n > ${OUTDIR}/${DISTNAME}-${i}.tar.gz
find ${DISTNAME} -name "*.dbg" | sort | tar --mtime="$REFERENCE_DATETIME" --no-recursion --mode='u+rw,go+r-w,a+X' --owner=0 --group=0 -c -T - | gzip -9n > ${OUTDIR}/${DISTNAME}-${i}-debug.tar.gz
cd ../../
rm -rf distsrc-${i}
done

Then, we use git archive to create the final source tarball:

bitcoin/contrib/gitian-descriptors/gitian-linux.yml

Line 194 in c5966a8

git archive --output=${OUTDIR}/src/${DISTNAME}.tar.gz HEAD

It seems to me that perhaps it'd be good to reinstate the behaviour whereby our binary tarballs are built from the source tarball, just for robustness's sake.

A simple fix might be:

  1. switch the earlier make dist to git archive
  2. ./autogen.sh for each host
  3. Remove the last call to git archive

Perhaps this can even allow us to specify the no-dist automake option and clean up dist related targets in Makefile.am.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Build system

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions