Skip to content

Conversation

mbland
Copy link
Contributor

@mbland mbland commented Sep 30, 2017

Spawned from #8. Both CI systems will also report test suite run times, and Travis will run macOS builds.

In order to get the macOS build to pass, this also includes a commit to work around the Bash 3.2.57 ERR trap bug. See the commit message for 0f6dde5 for details.

Also worth noting are the build times for this PR:

Linux:
real	0m4.248s
user	0m3.488s
sys	0m1.018s

macOS:
real	0m8.898s
user	0m3.798s
sys	0m4.161s

Windows:
real	0m28.079s
user	0m6.764s
sys	0m14.828s

vs. those from #8:

Linux:
real	0m1.947s
user	0m1.564s
sys	0m0.419s

macOS:
real	0m4.910s
user	0m2.725s
sys	0m1.507s

Windows:
real	0m11.291s
user	0m3.027s
sys	0m5.625s

This follows the example from https://www.appveyor.com/docs/lang/ruby/
except that it doesn't need `install` or `before_test` steps.
This enables us to get an idea of the performance impact of a change.
As with the update to .appveyor.yml in the previous commit, this enables
us to get an idea of the performance impact of a change.
Also sets the `language` to `bash`.
The following build is demonstrating failures I can't reproduce on my
own macOS system:

  https://travis-ci.org/bats-core/bats-core/jobs/281719290
When running under Bash 3.2.57(1)-release on macOS, the following tests
would fail because `BATS_ERROR_STACK_TRACE` would be empty, and hence no
information about the actual error would get printed:

- one failing test
- failing test with significant status
- failing test file outside of BATS_CWD

This is because each of these cases use `FIXTURE_ROOT/failing.bats`, and
the `ERR` trap would not fire for its `eval "( exit ${STATUS:-1} )"`
line. Changing it to `exit ${STATUS:-1}` produced the same effect, and
changing it to `return ${STATUS:-1}` would cause the output to point to
the previous line, which executes `true`.

However, the correct status would be reported to the `EXIT` trap, so now
we call `bats_error_trap` at the very beginning of `bats_teardown_trap`.

All the existing tests now pass under Bash 3.2.57(1)-release, Bash
4.2.25(1)-release (the version from the default Ubuntu 12.04.5/Precise
image on Travis CI), and Bash 4.4.12(1)-release.
Copy link
Member

@btamayo btamayo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is awesome, thanks!

@btamayo
Copy link
Member

btamayo commented Sep 30, 2017

Thank you for this. :) Feel free to merge when ready.

@mbland mbland merged commit 55bf719 into master Sep 30, 2017
@mbland mbland deleted the ci-configs branch September 30, 2017 19:11
@mbland mbland mentioned this pull request Sep 30, 2017
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants