Skip to content

test/system: Silence warning with Bats >= 1.7.0 #1081

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Aug 1, 2022
Merged

test/system: Silence warning with Bats >= 1.7.0 #1081

merged 2 commits into from
Aug 1, 2022

Conversation

debarshiray
Copy link
Member

@debarshiray debarshiray commented Jul 31, 2022 •
edited
Loading

Bats 1.7.0 emits a warning if a command passed to 'run' returns with an
exit code of 127 [1].

This requires Bats >= 1.5.0, which is present in Fedora >=35, and
supports specifying the exit code as an argument to Bats' 'run'
command [2].

However, bats_require_minimum_version can't be used, because it's
only available from Bats 1.7.0, which is new enough that it's absent in
Fedora 35.

[1] Bats commit c6dc2f88361a4f5b
bats-core/bats-core#586
https://bats-core.readthedocs.io/en/stable/warnings/BW01.html

[2] bats-core/bats-core#367
bats-core/bats-core#507
https://bats-core.readthedocs.io/en/stable/writing-tests.html

[3] Bats commit 71d6b71cebc3d32b
bats-core/bats-core#556
https://bats-core.readthedocs.io/en/stable/warnings/BW02.html

debarshiray added a commit to debarshiray/toolbox that referenced this pull request Jul 31, 2022
@debarshiray
test/system: Silence Bats warning about a command exiting with 127
bf58987 
Bats 1.7.0 emits a warning if a command passed to 'run' returns with an
exit code of 127 [1].

[1] Bats commit c6dc2f88361a4f5b
    bats-core/bats-core#586
    https://bats-core.readthedocs.io/en/stable/warnings/BW01.html

containers#1081
@debarshiray debarshiray force-pushed the wip/rishi/bats-silence-command-not-found branch from e620481 to bf58987 Compare July 31, 2022 17:31
@debarshiray debarshiray changed the title test/system: Silence Bats warning about a command exiting with 127 test/system: Silence warning with Bats >= 1.7.0 Jul 31, 2022
@softwarefactory-project-zuul
Copy link

Build failed.

✔️ system-test-fedora-rawhide SUCCESS in 16m 26s
✔️ system-test-fedora-36 SUCCESS in 9m 42s
❌ system-test-fedora-35 FAILURE in 10m 12s
❌ system-test-fedora-34 FAILURE in 10m 06s

@debarshiray debarshiray marked this pull request as draft July 31, 2022 18:06
@debarshiray debarshiray changed the title test/system: Silence warning with Bats >= 1.7.0 [WIP] test/system: Silence warning with Bats >= 1.7.0 Jul 31, 2022
debarshiray added a commit to debarshiray/toolbox that referenced this pull request Jul 31, 2022
@debarshiray
test/system: Silence warning with Bats >= 1.7.0
5a2368d 
Bats 1.7.0 emits a warning if a command passed to 'run' returns with an
exit code of 127 [1].

This requires Bats >= 1.5.0, which is present in Fedora >=35.
However, bats_require_minimum_version can't be used.

[1] Bats commit c6dc2f88361a4f5b
    bats-core/bats-core#586
    https://bats-core.readthedocs.io/en/stable/warnings/BW01.html

containers#1081
@debarshiray debarshiray force-pushed the wip/rishi/bats-silence-command-not-found branch from bf58987 to 5a2368d Compare July 31, 2022 18:09
@softwarefactory-project-zuul
Copy link

Build failed.

✔️ system-test-fedora-rawhide SUCCESS in 16m 27s
✔️ system-test-fedora-36 SUCCESS in 9m 46s
✔️ system-test-fedora-35 SUCCESS in 9m 58s
❌ system-test-fedora-34 FAILURE in 10m 14s

debarshiray added a commit to debarshiray/toolbox that referenced this pull request Jul 31, 2022
@debarshiray
.zuul: Drop testing on Fedora 34
ea1f290 
Fedora 34 reached End of Life on 7th June 2022:
https://docs.fedoraproject.org/en-US/releases/eol/

The subsequent commit will bump the minimum required Bats version to
1.5.0, which is absent in Fedora 34.

containers#1081
debarshiray added a commit to debarshiray/toolbox that referenced this pull request Jul 31, 2022
@debarshiray
test/system: Silence warning with Bats >= 1.7.0
f58a904 
Bats 1.7.0 emits a warning if a command passed to 'run' returns with an
exit code of 127 [1].

This requires Bats >= 1.5.0, which is present in Fedora >=35, and
supports specifying the exit code as an argument to Bats' 'run'
command [2].

However, bats_require_minimum_version can't be used, because it's
only available from Bats 1.7.0, which is new enough that it's absent in
Fedora 35.

[1] Bats commit c6dc2f88361a4f5b
    bats-core/bats-core#547
    https://bats-core.readthedocs.io/en/stable/warnings/BW01.html

[2] bats-core/bats-core#367
    bats-core/bats-core#507
    https://bats-core.readthedocs.io/en/stable/writing-tests.html

[3] Bats commit 71d6b71cebc3d32b
    bats-core/bats-core#556
    https://bats-core.readthedocs.io/en/stable/warnings/BW02.html

containers#1081
@debarshiray debarshiray force-pushed the wip/rishi/bats-silence-command-not-found branch from 5a2368d to f58a904 Compare July 31, 2022 20:00
@debarshiray debarshiray changed the title [WIP] test/system: Silence warning with Bats >= 1.7.0 test/system: Silence warning with Bats >= 1.7.0 Jul 31, 2022
@debarshiray debarshiray marked this pull request as ready for review July 31, 2022 20:03
@softwarefactory-project-zuul
Copy link

Build failed.

✔️ unit-test SUCCESS in 6m 48s
✔️ system-test-fedora-rawhide SUCCESS in 16m 36s
✔️ system-test-fedora-36 SUCCESS in 9m 48s
❌ system-test-fedora-35 FAILURE in 10m 07s

@debarshiray
Copy link
Member Author

Interesting:

not ok 1 test suite: Setup in 32935ms
# (from function `assert_success' in file test/system/libs/bats-assert/src/assert.bash, line 114,
#  from function `_setup_docker_registry' in file test/system/libs/helpers.bash, line 199,
#  in test file test/system/000-setup.bats, line 23)
#   `_setup_docker_registry' failed
# 
# -- command failed --
# status : 126
# output : Error: rootlessport listen tcp 0.0.0.0:50000: bind: address already in use

@debarshiray
Copy link
Member Author

recheck

@softwarefactory-project-zuul
Copy link

Build succeeded.

✔️ unit-test SUCCESS in 6m 47s
✔️ system-test-fedora-rawhide SUCCESS in 16m 37s
✔️ system-test-fedora-36 SUCCESS in 9m 51s
✔️ system-test-fedora-35 SUCCESS in 10m 12s

@martymichal
Copy link
Member

Interesting:

not ok 1 test suite: Setup in 32935ms
# (from function `assert_success' in file test/system/libs/bats-assert/src/assert.bash, line 114,
#  from function `_setup_docker_registry' in file test/system/libs/helpers.bash, line 199,
#  in test file test/system/000-setup.bats, line 23)
#   `_setup_docker_registry' failed
# 
# -- command failed --
# status : 126
# output : Error: rootlessport listen tcp 0.0.0.0:50000: bind: address already in use

This has happened to me in the past and usually the trigger was interrupting the system tests and not allowing them to properly teardown. But in the context of the CI, very curious.

@martymichal martymichal added 3. Bugfix Fixes a bug 2. CI Automation of testing, analysis and other actions 2. Testing Related to testing of Toolbox - unit, system,.. labels Jul 31, 2022
martymichal
martymichal approved these changes Jul 31, 2022
View reviewed changes
Copy link
Member

@martymichal martymichal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks sensible. I'm not sure now if I ever noticed the error this PR is fixing. But it doesn't break the tests, so let's go with it.

debarshiray and others added 2 commits August 1, 2022 10:47
@debarshiray
.zuul: Drop testing on Fedora 34
03daa86 
Fedora 34 reached End of Life on 7th June 2022:
https://docs.fedoraproject.org/en-US/releases/eol/

The subsequent commit will bump the minimum required Bats version to
1.5.0, which is absent from Fedora 34.

containers#1081
@debarshiray
test/system: Silence warning with Bats >= 1.7.0
978bb52 
Bats 1.7.0 emits a warning if a command passed to 'run' returns with an
exit code of 127 [1]:
  BW01: `run`'s command `/opt/bin/toolbox run non-existent-command`
    exited with code 127, indicating 'Command not found'. Use run's
    return code checks, e.g. `run -127`, to fix this message.
        (from function `run' in file
          /usr/lib/bats-core/test_functions.bash, line 299,
         in test file test/system/104-run.bats, line 148)

This requires Bats >= 1.5.0, which is present in Fedora >=35, and
supports specifying the exit code as an argument to Bats' 'run'
command [2].

However, bats_require_minimum_version can't be used, because it's
only available from Bats 1.7.0, which is new enough that it's absent
from Fedora 35.

[1] Bats commit c6dc2f88361a4f5b
    bats-core/bats-core#547
    https://bats-core.readthedocs.io/en/stable/warnings/BW01.html

[2] bats-core/bats-core#367
    bats-core/bats-core#507
    https://bats-core.readthedocs.io/en/stable/writing-tests.html

[3] Bats commit 71d6b71cebc3d32b
    bats-core/bats-core#556
    https://bats-core.readthedocs.io/en/stable/warnings/BW02.html

containers#1081
@debarshiray debarshiray force-pushed the wip/rishi/bats-silence-command-not-found branch from f58a904 to 978bb52 Compare August 1, 2022 08:51
@debarshiray
Copy link
Member Author

I'm not sure now if I ever noticed the error this PR is fixing

Try running the tests locally on Fedora >= 36.

@debarshiray debarshiray merged commit 978bb52 into containers:main Aug 1, 2022
@debarshiray debarshiray deleted the wip/rishi/bats-silence-command-not-found branch August 1, 2022 08:54
nievesmontero pushed a commit to nievesmontero/toolbox that referenced this pull request Aug 24, 2022
@debarshiray @nievesmontero
.zuul: Drop testing on Fedora 34
1d448ce 
Fedora 34 reached End of Life on 7th June 2022:
https://docs.fedoraproject.org/en-US/releases/eol/

The subsequent commit will bump the minimum required Bats version to
1.5.0, which is absent from Fedora 34.

containers#1081
nievesmontero pushed a commit to nievesmontero/toolbox that referenced this pull request Aug 24, 2022
@debarshiray @nievesmontero
test/system: Silence warning with Bats >= 1.7.0
910fdc5 
Bats 1.7.0 emits a warning if a command passed to 'run' returns with an
exit code of 127 [1]:
  BW01: `run`'s command `/opt/bin/toolbox run non-existent-command`
    exited with code 127, indicating 'Command not found'. Use run's
    return code checks, e.g. `run -127`, to fix this message.
        (from function `run' in file
          /usr/lib/bats-core/test_functions.bash, line 299,
         in test file test/system/104-run.bats, line 148)

This requires Bats >= 1.5.0, which is present in Fedora >=35, and
supports specifying the exit code as an argument to Bats' 'run'
command [2].

However, bats_require_minimum_version can't be used, because it's
only available from Bats 1.7.0, which is new enough that it's absent
from Fedora 35.

[1] Bats commit c6dc2f88361a4f5b
    bats-core/bats-core#547
    https://bats-core.readthedocs.io/en/stable/warnings/BW01.html

[2] bats-core/bats-core#367
    bats-core/bats-core#507
    https://bats-core.readthedocs.io/en/stable/writing-tests.html

[3] Bats commit 71d6b71cebc3d32b
    bats-core/bats-core#556
    https://bats-core.readthedocs.io/en/stable/warnings/BW02.html

containers#1081
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@martymichal martymichal martymichal approved these changes

Assignees
No one assigned
Labels
2. CI Automation of testing, analysis and other actions 2. Testing Related to testing of Toolbox - unit, system,.. 3. Bugfix Fixes a bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@debarshiray @martymichal