Skip to content

Conversation

maribu
Copy link
Member

@maribu maribu commented May 6, 2025

Contribution description

When using OUTPUT=COLORTEST the return value of the unit test's main() on failure was 0, while it should be 1. As a result, running the unit tests from a script using

$ OUTPUT=COLORTEXT make BOARD=native64 RIOT_TERMINAL=native -j32 flash term -C tests/unittests

would not catch a test failure.

This commit changes the behavior so that 0 is returned on success and 1 on failure for both regular and colored text output.

Testing procedure

Run

$ TZ=Europe/Berlin OUTPUT=COLORTEXT make BOARD=native64 RIOT_TERMINAL=native -j32 flash term -C tests/unittests

and

$ TZ=UTC OUTPUT=COLORTEXT make BOARD=native64 RIOT_TERMINAL=native -j32 flash term -C tests/unittests

The former will fail, the latter succeed. But both will have an exit code of 0 (echo $?) in master, while ommitting the OUTPUT=COLORTEXT will cause the failed test to have an exit code of 1.

With this PR, failures will have an exit code of 1 and success an exit code of 0, regardless of output.

Issues/PRs references

None

When using `OUTPUT=COLORTEST` the return value of the unit test's
`main()` on failure was `0`, while it should be `1`. As a result,
running the unit tests from a script using

    $ OUTPUT=COLORTEXT make BOARD=native64 RIOT_TERMINAL=native -j32 flash term -C tests/unittests

would not catch a test failure.

This commit changes the behavior so that `0` is returned on success
and `1` on failure for both regular and colored text output.
@maribu maribu requested a review from benpicco May 6, 2025 17:00
@maribu maribu added Type: bug The issue reports a bug / The PR fixes a bug (including spelling errors) CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR labels May 6, 2025
@github-actions github-actions bot added the Area: sys Area: System label May 6, 2025
@riot-ci
Copy link

riot-ci commented May 6, 2025

Murdock results

✔️ PASSED

b0ec60c sys/embunit: fix return value on failure

Success Failures Total Runtime
10336 0 10337 12m:50s

Artifacts

@maribu maribu added this pull request to the merge queue May 7, 2025
Merged via the queue into RIOT-OS:master with commit a71c3a1 May 7, 2025
28 checks passed
@maribu maribu deleted the sys/embunit/fix-return-value branch May 7, 2025 08:02
@maribu
Copy link
Member Author

maribu commented May 7, 2025

Thx :)

@Teufelchen1 Teufelchen1 added this to the Release 2025.07 milestone Jul 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area: sys Area: System CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR Type: bug The issue reports a bug / The PR fixes a bug (including spelling errors)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants