Skip to content

Conversation

hebasto
Copy link
Member

@hebasto hebasto commented Nov 3, 2019

This PR extracts shell scripts from Gitian descriptors (contrib/gitian-descriptors/) and checks for ShellCheck warnings as any other one.

Some non-controversial warnings are fixed.

Copy link
Contributor

@promag promag left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concept ACK.

@@ -0,0 +1,22 @@
#!/usr/bin/env python3
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this really necessary? Couldn't use some YAML processor like yq?

Copy link
Member Author

@hebasto hebasto Nov 4, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no straightforward way to install yq on Travis, unfortunately.

Or did you mean that yq?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couldn't use some YAML processor like yq?

Done.
TIL, thank you ;)

@hebasto hebasto force-pushed the 20191103-lint-gitian-scripts branch 3 times, most recently from d82b77a to 7e8b40e Compare November 4, 2019 06:40
@practicalswift
Copy link
Contributor

Concept ACK: good idea!

Could perhaps check for yq and skip the Gitian checking if it is not installed (with a warning message) in order to not break current setups?

@hebasto hebasto force-pushed the 20191103-lint-gitian-scripts branch from 7e8b40e to 93faddf Compare November 4, 2019 07:21
@hebasto
Copy link
Member Author

hebasto commented Nov 4, 2019

@practicalswift

Could perhaps check for yq and skip the Gitian checking if it is not installed (with a warning message) in order to not break current setups?

Done. Thank you.

@hebasto hebasto force-pushed the 20191103-lint-gitian-scripts branch from 93faddf to a953da7 Compare November 4, 2019 17:30
@hebasto
Copy link
Member Author

hebasto commented Nov 4, 2019

@dongcarl
Thank you for your review. All your comments have been addressed.

Copy link
Contributor

@dongcarl dongcarl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For all the places where you had to split export DIR=$(pwd), just use export DIR="$PWD", which doesn't require the splitting

do
echo
echo "$descriptor"
SCRIPT=$'#!/bin/bash\n'$(yq -r .script "$descriptor")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
SCRIPT=$'#!/bin/bash\n'$(yq -r .script "$descriptor")
SCRIPT=$'#!/usr/bin/env bash\n'$(yq -r .script "$descriptor")

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It just mimics the way gitian-builder/bin/gbuild makes shell script:

  File.open("var/build-script", "w") do |script|
    script.puts "#!/bin/bash"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah! Makes sense to do it this way, could you add a comment?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@hebasto hebasto force-pushed the 20191103-lint-gitian-scripts branch from a953da7 to 53ff028 Compare November 4, 2019 22:52
@hebasto
Copy link
Member Author

hebasto commented Nov 4, 2019

For all the places where you had to split export DIR=$(pwd), just use export DIR="$PWD", which doesn't require the splitting

Done.

@DrahtBot
Copy link
Contributor

DrahtBot commented Nov 5, 2019

Gitian builds

File commit fba574c
(master)
commit 03cba51
(master and this pull)
bitcoin-0.19.99-aarch64-linux-gnu-debug.tar.gz de237fcbed0428e9... 30695e643ab9f5c8...
bitcoin-0.19.99-aarch64-linux-gnu.tar.gz 37414ac687fca757... bf632b6eb372cc1e...
bitcoin-0.19.99-arm-linux-gnueabihf-debug.tar.gz 15ec11ef799983be... 8eafdee2275e4747...
bitcoin-0.19.99-arm-linux-gnueabihf.tar.gz d2a4b5c5eecf4867... 9c083fd1ca78126e...
bitcoin-0.19.99-i686-pc-linux-gnu-debug.tar.gz 93c7a296c2b1892a... 5e764f885367d3f0...
bitcoin-0.19.99-i686-pc-linux-gnu.tar.gz 6ba72605de448718... 35bd351eb4b6e563...
bitcoin-0.19.99-osx-unsigned.dmg 9f5e8bd0f7b5d596... 037dc66f64c011a8...
bitcoin-0.19.99-osx64.tar.gz e0c9bae97735661b... 5ce605bc2e7eecad...
bitcoin-0.19.99-riscv64-linux-gnu-debug.tar.gz 00eb708d00240949... 3bb47e7fb3adfd13...
bitcoin-0.19.99-riscv64-linux-gnu.tar.gz 01c2cac7151b4b97... d2c2da2bcdc3caed...
bitcoin-0.19.99-win64-debug.zip 39909ee795800bfb...
bitcoin-0.19.99-win64-setup-unsigned.exe c2a4f6aa56e0ae3b...
bitcoin-0.19.99-win64.zip 55d8e692451261b6...
bitcoin-0.19.99-x86_64-linux-gnu-debug.tar.gz 38343844b0191747... 2a7ebe8ad5c2bd4d...
bitcoin-0.19.99-x86_64-linux-gnu.tar.gz 9b7e7f889b59604c... 183f728c9b074532...
bitcoin-0.19.99.tar.gz 468e3d9979c27430... 224f38363c8c26c7...
bitcoin-core-linux-0.20-res.yml 1147a1c215d2cfbd... b0a281cb13e2d3ae...
bitcoin-core-osx-0.20-res.yml 48b30111fb80c772... a1142403eb634cdc...
bitcoin-core-win-0.20-res.yml 84c74d55aeea95e2...
linux-build.log 6898cbc40626a205... 61792b23cc02b876...
osx-build.log 91702333e74cd1f8... b1d5202ba081635f...
win-build.log fefebb4a795de7f7...
bitcoin-core-linux-0.20-res.yml.diff ab3952092e2ea658...
bitcoin-core-osx-0.20-res.yml.diff 44759f80b2f42f19...
linux-build.log.diff 203d923c4540f2aa...
osx-build.log.diff 2c8087914b8df914...

@DrahtBot
Copy link
Contributor

DrahtBot commented Nov 6, 2019

Gitian builds

File commit bdda137
(master)
commit 52a158a
(master and this pull)
bitcoin-0.19.99-aarch64-linux-gnu-debug.tar.gz 338c2453e92576d6... d00f76215eb7cad7...
bitcoin-0.19.99-aarch64-linux-gnu.tar.gz ebfe02ebb327a5fd... 829fca33b72e3e49...
bitcoin-0.19.99-arm-linux-gnueabihf-debug.tar.gz 10685a8364a42850... 809e882222ecdb8b...
bitcoin-0.19.99-arm-linux-gnueabihf.tar.gz 5debad8ecf03f614... bd7f0819847ee09a...
bitcoin-0.19.99-i686-pc-linux-gnu-debug.tar.gz c94274278265de7d... 568b25450ace1558...
bitcoin-0.19.99-i686-pc-linux-gnu.tar.gz 9199c2a5e431eb9a... a0d9500045d0ff3b...
bitcoin-0.19.99-osx-unsigned.dmg 7da5a7f569a4318d... 9e25d1d09f726e61...
bitcoin-0.19.99-osx64.tar.gz a459a9173951314d... 255e95cfc99e80e3...
bitcoin-0.19.99-riscv64-linux-gnu-debug.tar.gz 660347859b4073ee... 09871d8a73d30994...
bitcoin-0.19.99-riscv64-linux-gnu.tar.gz feb185a593b038d9... e0064e2658430257...
bitcoin-0.19.99-win64-debug.zip a58bd2fba33f4e15... 548fc80baf980816...
bitcoin-0.19.99-win64-setup-unsigned.exe 501af3958d794c3d... 52437c0e5bd068d8...
bitcoin-0.19.99-win64.zip fb49f8e495bd1bdc... 81b3b6ffa93ffb2b...
bitcoin-0.19.99-x86_64-linux-gnu-debug.tar.gz 8b1f655dfa7aab99... 7c4742ffc5d1bcff...
bitcoin-0.19.99-x86_64-linux-gnu.tar.gz 21b36fd6521700d8... f5bf832c0c5bf244...
bitcoin-0.19.99.tar.gz 96c42231845ab16d... 93fd881e2c8eacc2...
bitcoin-core-linux-0.20-res.yml 5fa050a02e3cb95c... e72dce94ec5b9090...
bitcoin-core-osx-0.20-res.yml d0d8fb2e491f47a0... 2fe9ef65e4f22aa9...
bitcoin-core-win-0.20-res.yml 6f9be4b4073760a5... 7035f5f17d238e3d...
linux-build.log b31ba24d0a233210... 91def7219b8c52c1...
osx-build.log eaeccfa77df8d98a... 8cea9c3430cca6ab...
win-build.log b9e547f8f55648b3... 20d51977604eee0f...
bitcoin-core-linux-0.20-res.yml.diff 391052be640490b5...
bitcoin-core-osx-0.20-res.yml.diff 96a8469c9b7db345...
bitcoin-core-win-0.20-res.yml.diff 9a5fcddeec8edff2...
linux-build.log.diff 7f52c250016bb7f7...
osx-build.log.diff 85d75ca12c4b66ee...
win-build.log.diff c3fb2c988d5d37bb...

@laanwj
Copy link
Member

laanwj commented Nov 6, 2019

Concept ACK

Copy link
Contributor

@promag promag left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hebasto
Copy link
Member Author

hebasto commented Nov 6, 2019

@promag

Please update https://github.com/bitcoin/bitcoin/blob/master/test/README.md#dependencies-1

It already lacks ShellCheck. Maybe just add yq dependency to #17353?

@promag
Copy link
Contributor

promag commented Nov 6, 2019

@hebasto let's see if that one is merged first so you can update the doc here.

MarkLTZ added a commit to litecoinz-core/litecoinz that referenced this pull request Apr 10, 2020
sidhujag pushed a commit to syscoin-core/syscoin that referenced this pull request Nov 10, 2020
17f81e9 script: Enable SC2001 rule for Gitian scripts (Hennadii Stepanov)
61bb21b script: Enable SC2155 rule for Gitian scripts (Hennadii Stepanov)
577682d script: Enable SC2006 rule for Gitian scripts (Hennadii Stepanov)
14aded4 script: Lint Gitian descriptors with ShellCheck (Hennadii Stepanov)

Pull request description:

  This PR extracts shell scripts from Gitian descriptors (`contrib/gitian-descriptors/`) and checks for ShellCheck warnings as any other one.

  Some non-controversial warnings are fixed.

ACKs for top commit:
  practicalswift:
    ACK 17f81e9 -- diff looks correct

Tree-SHA512: bdfa3d35bbb65ff634c90835d75c3df63e958b558599771d21366724f5cf64da83a68957d926e926a99c3704b9529e96a17697dc8d9ff3adf7154d9cb1999a8d
kwvg added a commit to kwvg/dash that referenced this pull request Jul 15, 2021
kwvg added a commit to kwvg/dash that referenced this pull request Jul 15, 2021
kwvg added a commit to kwvg/dash that referenced this pull request Jul 20, 2021
kwvg added a commit to kwvg/dash that referenced this pull request Jul 20, 2021
kwvg added a commit to kwvg/dash that referenced this pull request Jul 20, 2021
kwvg added a commit to kwvg/dash that referenced this pull request Jul 20, 2021
kwvg added a commit to kwvg/dash that referenced this pull request Aug 1, 2021
kwvg added a commit to kwvg/dash that referenced this pull request Aug 24, 2021
kwvg added a commit to kwvg/dash that referenced this pull request Aug 24, 2021
kwvg added a commit to kwvg/dash that referenced this pull request Aug 25, 2021
kwvg added a commit to kwvg/dash that referenced this pull request Aug 25, 2021
kwvg added a commit to kwvg/dash that referenced this pull request Aug 26, 2021
kwvg added a commit to kwvg/dash that referenced this pull request Aug 26, 2021
kwvg added a commit to kwvg/dash that referenced this pull request Aug 26, 2021
kwvg added a commit to kwvg/dash that referenced this pull request Aug 27, 2021
kwvg added a commit to kwvg/dash that referenced this pull request Aug 30, 2021
kwvg added a commit to kwvg/dash that referenced this pull request Sep 1, 2021
kwvg added a commit to kwvg/dash that referenced this pull request Sep 1, 2021
kwvg added a commit to kwvg/dash that referenced this pull request Sep 1, 2021
kwvg added a commit to kwvg/dash that referenced this pull request Sep 1, 2021
kwvg added a commit to kwvg/dash that referenced this pull request Sep 2, 2021
kwvg added a commit to kwvg/dash that referenced this pull request Sep 3, 2021
kwvg added a commit to kwvg/dash that referenced this pull request Sep 3, 2021
kwvg added a commit to kwvg/dash that referenced this pull request Sep 3, 2021
PastaPastaPasta pushed a commit to PastaPastaPasta/dash that referenced this pull request Sep 17, 2021
c966ff1 gitian: fixed SC2001 regex (willyk)

Pull request description:

  Currently the gitian-win-signer.yml produces OUTFILE names without `-unsigned` stripped out
  This is due to regex having an`%` in front of it
  ```
  $ INFILE="bitcoin-0.19.0-win64-setup-unsigned.exe"
  $ echo "${INFILE/%-unsigned}"
  bitcoin-0.19.0-win64-setup-unsigned.exe
  $ echo "${INFILE/-unsigned}"
  bitcoin-0.19.0-win64-setup.exe
  ```

  Fixes bitcoin#17361

ACKs for top commit:
  laanwj:
    ACK c966ff1
  hebasto:
    ACK c966ff1

Tree-SHA512: 954547f9dfa4cab4def5f284d4837c21f0e6fed7454a04e83e6b1b7d3fd3f9661ea657047f0d8162f6591909d32ef2f72e801b2f3a44cbb1131ac344cb913a69
thelazier pushed a commit to thelazier/dash that referenced this pull request Sep 25, 2021
c966ff1 gitian: fixed SC2001 regex (willyk)

Pull request description:

  Currently the gitian-win-signer.yml produces OUTFILE names without `-unsigned` stripped out
  This is due to regex having an`%` in front of it
  ```
  $ INFILE="bitcoin-0.19.0-win64-setup-unsigned.exe"
  $ echo "${INFILE/%-unsigned}"
  bitcoin-0.19.0-win64-setup-unsigned.exe
  $ echo "${INFILE/-unsigned}"
  bitcoin-0.19.0-win64-setup.exe
  ```

  Fixes bitcoin#17361

ACKs for top commit:
  laanwj:
    ACK c966ff1
  hebasto:
    ACK c966ff1

Tree-SHA512: 954547f9dfa4cab4def5f284d4837c21f0e6fed7454a04e83e6b1b7d3fd3f9661ea657047f0d8162f6591909d32ef2f72e801b2f3a44cbb1131ac344cb913a69
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Dec 16, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants