Skip to content

feat: Add some duration information to test output. #464

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Apr 28, 2023
Merged

feat: Add some duration information to test output. #464

merged 1 commit into from
Apr 28, 2023

Conversation

smoser
Copy link
Contributor

@smoser smoser commented Apr 28, 2023 •
edited
Loading

The bats test framework provides '--timing' flag that will ammend its "tap" output with test duration information.

Per-test test output is then changed like this:

- ok 1 invoking foo with a nonexistent file prints an error
+ ok 1 invoking foo with a nonexistent file prints an error in 1sec

Not terribly useful, but better than nothing.

Also here change '-t' to the more readable long form '--tap'.

@smoser smoser requested review from rchincha and hallyn as code owners April 28, 2023 14:51
@smoser
Copy link
Contributor Author

smoser commented Apr 28, 2023 •
edited
Loading

I didn't realize it, but you can "view raw logs" or "show timestamps" in the web view of an 'actions' job and see timestamps of the build output. Unfortunately output must be heavily buffered as you can see below, so the timestamps aren't terribly useful.

Output of the tests now shows:

ok 1 namespace arg works in 9sec
ok 2 default namespace arg works in 8sec
ok 3 annotations work in 9sec
ok 4 workdir args in 15sec
ok 5 wildcards work in run section in 17sec
ok 6 --no-squashfs-verity works # skip test not valid for privilege level unpriv
ok 7 mount + umount works # skip test not valid for privilege level unpriv
ok 8 mount + umount + mount a tree of images works # skip test not valid for privilege level unpriv
ok 9 bad existing verity device is rejected # skip test not valid for privilege level unpriv
ok 10 multiple stacker builds in a row in 6sec
ok 11 basic workings in 31sec
ok 12 stacker.yaml without imports can run in 14sec
ok 13 stacker without arguments prints help in 1sec
ok 14 use colons in roots-dir path name should fail in 0sec
ok 15 use colons in layer name should fail in 0sec
ok 16 basic workings with substitutions from a file in 31sec
ok 17 bind as string slice in 7sec

in "view raw logs" you see something like:

2023-04-28T15:02:02.7968351Z ok 1 namespace arg works in 9sec
2023-04-28T15:02:02.7968610Z ok 2 default namespace arg works in 8sec
2023-04-28T15:02:02.7968875Z ok 3 annotations work in 9sec
2023-04-28T15:02:09.7835092Z ok 4 workdir args in 15sec

See above that 9+8+9+15 (total 43 seconds) have elapsed but the timestamps only show < 7 seconds gone. I can only explain that as buffering.

rchincha
rchincha approved these changes Apr 28, 2023
View reviewed changes
Copy link
Contributor

@rchincha rchincha left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@smoser
feat: Add some duration information to test output.
5cab38a 
The bats test framework provides '--timing' flag that will
ammend its "tap" output with test duration information.

Per-test test output is then changed like this:
- ok 1 invoking foo with a nonexistent file prints an error
+ ok 1 invoking foo with a nonexistent file prints an error in 1sec

Not terribly useful, but better than nothing.

Also here change '-t' to the more readable long form '--tap'.

Signed-off-by: Scott Moser <smoser@brickies.net>
@smoser smoser force-pushed the feature/show-test-duration branch from 84f0c39 to 5cab38a Compare April 28, 2023 17:14
@smoser smoser merged commit 02cb2ac into project-stacker:main Apr 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@hallyn hallyn hallyn approved these changes

@mikemccracken mikemccracken mikemccracken approved these changes

@rchincha rchincha rchincha approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@smoser @hallyn @mikemccracken @rchincha