Skip to content

scripts and tools: Update ShellCheck linter #16327

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Jul 5, 2019
Merged

scripts and tools: Update ShellCheck linter #16327

merged 1 commit into from
Jul 5, 2019

Conversation

hebasto
Copy link
Member

@hebasto hebasto commented Jul 2, 2019 •
edited
Loading

Enable some simple ShellCheck rules.

Note for reviewers: bash and shellcheck on macOS are different from ones on Ubuntu.
For local tests the latest shellcheck version 0.6.0 should be used (see #15166).

@hebasto
Copy link
Member Author

hebasto commented Jul 2, 2019

ping @Empact @practicalswift @promag @koalaman

@hebasto
Copy link
Member Author

hebasto commented Jul 2, 2019 •
edited
Loading

Forgot about macOS quirks (#14115). Going to fix right now.

@hebasto hebasto force-pushed the 20190702-shellcheck branch from cef55d3 to 077b62e Compare July 2, 2019 18:54
@hebasto
Copy link
Member Author

hebasto commented Jul 2, 2019

Forgot about macOS quirks (#14115). Going to fix right now.

Fixed. OP updated with a note for reviewers.

@DrahtBot
Copy link
Contributor

DrahtBot commented Jul 2, 2019

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #12134 (Build previous releases and run functional tests by Sjors)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@Empact
Copy link
Contributor

Empact commented Jul 2, 2019

ACK 077b62e test/lint/lint-shell.sh passes on macOS 10.14.5

@hebasto
Copy link
Member Author

hebasto commented Jul 3, 2019

I was wrong: the 05fa54d commit is misleading. There is nothing macOS specific in SC1087, SC2206, SC2207, SC2230 and SC2236 warnings.
The point is the default shellcheck version on bionic is:

$ /usr/bin/shellcheck -V | grep version:
version: 0.4.6

But we use version 0.6.0 (see #15166).

The misleading commit will be removed today.

@practicalswift
Copy link
Contributor

practicalswift commented Jul 3, 2019 •
edited
Loading

Strong concept ACK:

As an active reviewer of all shell scripts added to the project I applaud this change: it will make sure more of the bugs introduced due to "language level misunderstandings" are fixed before human review takes place. That is clearly a good thing.

Case in point: This change fixes such a bug/misunderstanding in contrib/verify-commits/gpg.sh.

@hebasto hebasto force-pushed the 20190702-shellcheck branch from 077b62e to e8683bf Compare July 3, 2019 17:19
dongcarl
dongcarl reviewed Jul 3, 2019
View reviewed changes
Copy link
Contributor

@dongcarl dongcarl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utACK e8683bf

Strongly in favor of stricter linting with shellcheck for shell scripts. They can be very hard to debug or fail silently if they're not.

@practicalswift
Copy link
Contributor

When @dongcarl's comments are addressed I think these changes should be ready to go. One last thing:

These tests remain disabled after this PR:

SC2006 # Use $(..) instead of legacy `..`.
SC2046 # Quote this to prevent word splitting.
SC2086 # Double quote to prevent globbing and word splitting.
SC2162 # read without -r will mangle backslashes.

I think it makes sense to keep these disabled with one exception:

@hebasto Would it be feasible to also enable SC2006?

Given the problems with the legacy form and the fact that all of these problems can be avoided by simply switching to $(…) I think it is worth having shellcheck guiding PR authors. That way human reviews don't have to repeat the arguments below :-)

Quoting shellcheck:

Use $(...) notation instead of legacy backticked `...`.

Problematic code

echo "You are running on `uname`"

Correct code

echo "You are running on $(uname)"

Rationale

Backtick command substitution `...` is legacy syntax with several issues.

  1. It has a series of undefined behaviors related to quoting in POSIX.
  2. It imposes a custom escaping mode with surprising results.
  3. It's exceptionally hard to nest.

$(...) command substitution has none of these problems, and is therefore strongly encouraged.

Exceptions

None.

Related resources:

@hebasto hebasto force-pushed the 20190702-shellcheck branch from e8683bf to 4dd3018 Compare July 4, 2019 15:30
@hebasto
Copy link
Member Author

hebasto commented Jul 4, 2019

@practicalswift

Would it be feasible to also enable SC2006?

Done.

@practicalswift
Copy link
Contributor

utACK 4dd3018 modulo squash + @dongcarl's comment about shellcheck disable=SC2181

@hebasto hebasto force-pushed the 20190702-shellcheck branch from 4dd3018 to 9dd9e2c Compare July 4, 2019 16:08
@hebasto
Copy link
Member Author

hebasto commented Jul 4, 2019

@practicalswift

... + @dongcarl's comment about shellcheck disable=SC2181

Done.

... modulo squash...

What commit message would be appropriate after squashing of all commits?

@practicalswift
Copy link
Contributor

@hebasto Title "Enable shellcheck rules" and description could be the squashed messages? :-)

@hebasto hebasto force-pushed the 20190702-shellcheck branch from 9dd9e2c to cb2f23d Compare July 4, 2019 16:30
@hebasto
Copy link
Member Author

hebasto commented Jul 4, 2019

@practicalswift

... modulo squash...

Squashed.

@practicalswift
Copy link
Contributor

Very nice!

utACK cb2f23d

@hebasto
Enable ShellCheck rules
1ac454a 
Enabled ShellCheck rules:
  SC1087
  SC2001
  SC2004
  SC2005
  SC2006
  SC2016
  SC2028
  SC2048
  SC2066 (note that IFS already contains only a line feed)
  SC2116
  SC2166
  SC2181
  SC2206
  SC2207
  SC2230
  SC2236
@hebasto hebasto force-pushed the 20190702-shellcheck branch from cb2f23d to 1ac454a Compare July 4, 2019 16:37
@hebasto
Copy link
Member Author

hebasto commented Jul 4, 2019

@practicalswift the rules in commit message are ordered now.

@practicalswift
Copy link
Contributor

utACK 1ac454a

@dongcarl
Copy link
Contributor

dongcarl commented Jul 4, 2019

utACK 1ac454a

fanquake
fanquake approved these changes Jul 5, 2019
View reviewed changes
Copy link
Member

@fanquake fanquake left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 1ac454a

Tested ./autogen.sh, contrib/install_db4.sh, contrib/devtools/gen-manpages.sh, contrib/verify-commits/gpg.sh, contrib/verifybinaries/verify.sh bitcoin-core-0.18.0 all still work. test/lint/lint-all.sh seems to pass.

@fanquake fanquake merged commit 1ac454a into bitcoin:master Jul 5, 2019
fanquake added a commit that referenced this pull request Jul 5, 2019
@fanquake
Merge #16327: scripts and tools: Update ShellCheck linter
1088b90 
1ac454a Enable ShellCheck rules (Hennadii Stepanov)

Pull request description:

  Enable some simple ShellCheck rules.

  Note for reviewers: `bash` and `shellcheck` on macOS are different from ones on Ubuntu.
  For local tests the latest `shellcheck` version 0.6.0 should be used (see #15166).

ACKs for top commit:
  practicalswift:
    utACK 1ac454a
  dongcarl:
    utACK 1ac454a
  fanquake:
    ACK 1ac454a

Tree-SHA512: 8d0a3a5c09fe1a0c22120178f5e6b80f81f746f8c3356b7701ff301c117acb2edea8fe08f08fb54ed73f94b1617515fb239fa28e7ab4121f74872e6494b6f20e
@hebasto hebasto deleted the 20190702-shellcheck branch July 5, 2019 02:30
This was referenced Oct 26, 2020
Merged
Merged
zkbot added a commit to zcash/zcash that referenced this pull request Oct 27, 2020
@zkbot
Auto merge of #4827 - str4d:commit-script-check, r=str4d
311a079 
Verifier for scriptable changes

Includes changes from the following upstream PRs:
- bitcoin/bitcoin#10189
  - Excluding the `CNode` scripted changes.
- bitcoin/bitcoin#10480
- bitcoin/bitcoin#11390
- bitcoin/bitcoin#13281
  - Only the lint scripts we already have.
- bitcoin/bitcoin#13454
  - Only changes to scripts we already have.
- bitcoin/bitcoin#14864
- bitcoin/bitcoin#16327
  - Includes some portability fixes to other shell scripts.
- bitcoin/bitcoin#20069
UdjinM6 pushed a commit to UdjinM6/dash that referenced this pull request Oct 23, 2021
@fanquake @UdjinM6
partial merge bitcoin#16327: scripts and tools: Update ShellCheck linter
0f14a3b 
1ac454a Enable ShellCheck rules (Hennadii Stepanov)

Pull request description:

  Enable some simple ShellCheck rules.

  Note for reviewers: `bash` and `shellcheck` on macOS are different from ones on Ubuntu.
  For local tests the latest `shellcheck` version 0.6.0 should be used (see bitcoin#15166).

ACKs for top commit:
  practicalswift:
    utACK 1ac454a
  dongcarl:
    utACK 1ac454a
  fanquake:
    ACK 1ac454a

Tree-SHA512: 8d0a3a5c09fe1a0c22120178f5e6b80f81f746f8c3356b7701ff301c117acb2edea8fe08f08fb54ed73f94b1617515fb239fa28e7ab4121f74872e6494b6f20e
UdjinM6 pushed a commit to UdjinM6/dash that referenced this pull request Oct 23, 2021
@fanquake @UdjinM6
partial merge bitcoin#16327: scripts and tools: Update ShellCheck linter
895db4c 
1ac454a Enable ShellCheck rules (Hennadii Stepanov)

Pull request description:

  Enable some simple ShellCheck rules.

  Note for reviewers: `bash` and `shellcheck` on macOS are different from ones on Ubuntu.
  For local tests the latest `shellcheck` version 0.6.0 should be used (see bitcoin#15166).

ACKs for top commit:
  practicalswift:
    utACK 1ac454a
  dongcarl:
    utACK 1ac454a
  fanquake:
    ACK 1ac454a

Tree-SHA512: 8d0a3a5c09fe1a0c22120178f5e6b80f81f746f8c3356b7701ff301c117acb2edea8fe08f08fb54ed73f94b1617515fb239fa28e7ab4121f74872e6494b6f20e
UdjinM6 pushed a commit to UdjinM6/dash that referenced this pull request Dec 4, 2021
@fanquake @UdjinM6
partial merge bitcoin#16327: scripts and tools: Update ShellCheck linter
ffe950b 
1ac454a Enable ShellCheck rules (Hennadii Stepanov)

Pull request description:

  Enable some simple ShellCheck rules.

  Note for reviewers: `bash` and `shellcheck` on macOS are different from ones on Ubuntu.
  For local tests the latest `shellcheck` version 0.6.0 should be used (see bitcoin#15166).

ACKs for top commit:
  practicalswift:
    utACK 1ac454a
  dongcarl:
    utACK 1ac454a
  fanquake:
    ACK 1ac454a

Tree-SHA512: 8d0a3a5c09fe1a0c22120178f5e6b80f81f746f8c3356b7701ff301c117acb2edea8fe08f08fb54ed73f94b1617515fb239fa28e7ab4121f74872e6494b6f20e
UdjinM6 added a commit to dashpay/dash that referenced this pull request Dec 5, 2021
@UdjinM6
Merge pull request #4605 from UdjinM6/bp21036_min
632c759 
Merges: bitcoin#16184, bitcoin#17029, bitcoin#17308, bitcoin#17409, bitcoin#17483, bitcoin#17756, bitcoin#18331, bitcoin#18556, bitcoin#19622; partial merges: bitcoin#16327, bitcoin#18104, bitcoin#18741, bitcoin#20318, bitcoin#21036
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Dec 16, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@fanquake fanquake fanquake approved these changes

+1 more reviewer

@dongcarl dongcarl dongcarl left review comments

Reviewers whose approvals may not affect merge requirements
Assignees
No one assigned
Labels
Scripts and tools
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@hebasto @DrahtBot @Empact @practicalswift @dongcarl @fanquake