Skip to content
This repository was archived by the owner on Apr 29, 2021. It is now read-only.

Conversation

themattrix
Copy link

I've often found it useful to test the value of global variables after a "run" command. However, the unit under test is run inside a subshell, making the values disappear before they can be tested.

This change introduces the "set_run_vars" command which will grab and set the values of any specified global variables from the run.

@sstephenson
Copy link
Owner

Very interesting. I assume you need this in order to test the side effects of shell functions? Could you share a real test case that uses this technique?

@themattrix
Copy link
Author

Sure thing! Here's a not-entirely-contrived example:

function parse_opts() {
    while getopts ":dv" option; do
        case $option in
            d) DEBUG=1;;
            v) VERBOSE=1;;
            *) usage;;
        esac
    done
}

function usage() {
    exit 1
}

@test "ensure -v sets VERBOSE mode" {
    run parse_opts -v
    set_run_vars VERBOSE
    [ "$VERBOSE" -eq 1 ]
}

@test "ensure -d sets DEBUG mode" {
    run parse_opts -d
    set_run_vars DEBUG
    [ "$DEBUG" -eq 1 ]
}

@mislav
Copy link
Collaborator

mislav commented Nov 11, 2013

I can understand the need for this but I don't like the proposed API nor the implementation in this PR. It seems contrived. I don't have a good alternative, though, but I'll come back if I get inspiration for how it might be handled better

@themattrix
Copy link
Author

I appreciate the feedback and will gladly welcome any improvements you may suggest!

@sstephenson
Copy link
Owner

For the example you've supplied, I'd suggest not using run at all, since you're not testing output or exit status:

@test "ensure -v sets VERBOSE mode" {
    parse_opts -v
    [ "$VERBOSE" -eq 1 ]
}

@test "ensure -d sets DEBUG mode" {
    parse_opts -d
    [ "$DEBUG" -eq 1 ]
}

And I would suggest using run to test the print-usage-and-exit behavior of invalid flags:

@test "invalid option prints usage" {
    run parse_opts -x
    [ "$status" -eq 1 ]
}

So, closing this for now—but please do reopen if you have a compelling case where the technique I've described won't work. (And thanks for the lovely patch.)

@themattrix
Copy link
Author

Fair enough. Here are my reasons for wanting to run such functions with run:

  1. Temporary unsetting of the e flag. Without run, any failing line within the function under test will cause the whole test to fail. (This could be unset/set in the test, of course).
  2. Better isolation from the rest of the test by use of a subshell. Only the requested variables are copied into the scope of the test.

In any case, I appreciate you taking the time to look at my pull request—and thanks for a great unit test framework!

@themattrix
Copy link
Author

@mislav I've changed the API in my fork so that my above example would now look like this:

@test "ensure -v sets VERBOSE mode" {
    run -v VERBOSE parse_opts -v
    [ "$VERBOSE" -eq 1 ]
}

@test "ensure -d sets DEBUG mode" {
    run -v DEBUG parse_opts -d
    [ "$DEBUG" -eq 1 ]
}

It also accepts multiple variables:

@test "run accepts multiple variables" {
    change_vars() {
        var1=1
        var2=2
        var3=3
    }

    run -v var1 -v var2 -v var3 change_vars

    [ "$var1" -eq 1 ]
    [ "$var2" -eq 2 ]
    [ "$var3" -eq 3 ]
}

I like this usage better because it doesn't introduce a new command. What do you think?

@mislav
Copy link
Collaborator

mislav commented Nov 11, 2013

I use bats exclusively to test the outcome of running shell executables. Your use case is different than that. Sam was right when he said that run is not tailored to your use-case. You probably want to make your own helper function that clears and resets the -e flag.

Yes, the subshell is useful for isolation but it seems that you don't want to test in isolation here. If you wanted to, you wouldn't be interested in changes to global environment variables.

yarikoptic pushed a commit to neurodebian/bats that referenced this pull request Aug 6, 2019
Sidesteps the problems encountered during sstephenson#32, and with running
`bin/bats` on Windows environments that don't support symlinks.
yarikoptic pushed a commit to neurodebian/bats that referenced this pull request Aug 6, 2019
Closes sstephenson#90. Closes sstephenson#83. Closes sstephenson#56. Closes sstephenson#55. Closes sstephenson#53. Resolves
outstanding issues from sstephenson#32.

Per all these earlier issues and PRs:

- The original bin/bats symlink was causing problems on Windows.
- The attempted fix from sstephenson#32 created problems for install.sh.
- Per sstephenson#90, changing bin/bats to a one-line script in sstephenson#88 broke some Bats
  installations that rely on symlink schemes (such as when installed via
  https://github.com/basherpm/basher).

For an idea of why I wanted to keep bin/bats as a script due to how
symlinks complicate things on Windows, see the following results I
discovered during a search for "git symlink windows":

- https://github.com/git-for-windows/git/wiki/Symbolic-Links
- https://blogs.windows.com/buildingapps/2016/12/02/symlinks-windows-10/
- libgit2/libgit2#4107
- https://stackoverflow.com/q/5917249

In the course of applying these changes, I realized libexec/bats
computed some extraneous variables, so I removed them, eliminating a few
external processes and subshells. I also cleaned up other small bits of
logic.

On top of making install.sh, bin/bats, and libexec/bats more resilient
and compact, the existing test suite (before adding the new
test/installer.bats file) sped up significantly, running 0.6s-0.7s
faster on my MacBook Pro (2.9 GHz Intel Core i5, 8GB RAM):

  Bash 3.2.57(1)-release before:
  58 tests, 0 failures

  real    0m4.924s
  user    0m3.045s
  sys     0m1.798s

  Bash 3.2.57(1)-release after:
  58 tests, 0 failures

  real    0m4.341s
  user    0m2.808s
  sys     0m1.540s

  Bash 4.4.23(1)-release before:
  58 tests, 0 failures

  real    0m5.228s
  user    0m3.046s
  sys     0m1.952s

  Bash 4.4.23(1)-release after:
  58 tests, 0 failures

  real    0m4.582s
  user    0m2.791s
  sys     0m1.643s

Also tweaks the Dockerfile to update the symlink to point to bin/bats,
not libexec/bats.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants