Skip to content

Conversation

sebhoss
Copy link
Contributor

@sebhoss sebhoss commented Dec 12, 2023

The prometheus metric goss_tests_run_outcomes_total has a label called 'outcome' which suffers from two problems:

  1. It won't be set to 'pass' in case all tests succeed
  2. The order of test results determines the final value of the label, e.g. if there is a failed & skipped test then the outcome will be skipped, but if order of results is reversed the outcome will become failed.

This commit fixes both by introducing a helper function which determines whether the outcome can be changed and adds tests to verify that behavior.

fixes #789

Checklist
  • make test-all (UNIX) passes. CI will also test this
  • unit and/or integration tests are included (if applicable)
  • documentation is changed or added (if applicable)

Description of change

The prometheus metric goss_tests_run_outcomes_total has a label called 'outcome' which suffers from two problems:

1) It won't be set to 'pass' in case all tests succeed
2) The order of test results determines the final value of the label, e.g. if there is a failed & skipped test then the outcome will be skipped, but if order of results is reversed the outcome will become failed.

This commit fixes both by introducing a helper function which determines whether the outcome can be changed and adds tests to verify that behavior.

fixes goss-org#789

Signed-off-by: Sebastian Hoß <seb@xn--ho-hia.de>
@sebhoss sebhoss requested a review from aelsabbahy as a code owner December 12, 2023 12:38
@sebhoss
Copy link
Contributor Author

sebhoss commented Dec 12, 2023

pinging @petemounce @timeu since you guys were involved in the original implementation

@timeu
Copy link

timeu commented Dec 12, 2023

From my point of view it looks good. Thanks for the fix

@aelsabbahy
Copy link
Member

Merging based on @timeu review.

Thank you for the great contribution @sebhoss

@aelsabbahy aelsabbahy merged commit 179143d into goss-org:master Dec 12, 2023
@sebhoss
Copy link
Contributor Author

sebhoss commented Dec 12, 2023

Thanks everyone!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Prometheus goss_tests_run_{outcomes, duration} metric has wrong label if the goss suite has no errors/skipped tests
3 participants