Skip to content

tests/riotboot: migrate to full python test script #11843

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Dec 2, 2019
Merged

tests/riotboot: migrate to full python test script #11843

merged 2 commits into from
Dec 2, 2019

Conversation

fjmolinas
Copy link
Contributor

Contribution description

This PR adapts tests/riotboot automatic test to a single python script. It also eliminate the need to fix an APP_VER=0.

Testing procedure

  • Run:

make -C tests/riotboot flash test

  • Verify CI output.

Issues/PRs references

Expands on #11707.

@fjmolinas fjmolinas added Area: tests Area: tests and testing framework Type: cleanup The issue proposes a clean-up / The PR cleans-up parts of the codebase / documentation CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR CI: run tests If set, CI server will run tests on hardware for the labeled PR Area: OTA Area: Over-the-air updates labels Jul 15, 2019
@fjmolinas fjmolinas requested a review from kaspar030 July 15, 2019 15:04
@fjmolinas fjmolinas force-pushed the pr_tests_riotboot_py branch from 9ee912b to 6eed370 Compare July 15, 2019 16:26
@fjmolinas fjmolinas added the CI: disable test cache If set, CI will always run all tests regardless of whether they have been run successfully before label Jul 15, 2019
@fjmolinas fjmolinas force-pushed the pr_tests_riotboot_py branch from 6eed370 to 9e0198c Compare July 16, 2019 06:27
@fjmolinas fjmolinas removed the CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR label Jul 16, 2019
@fjmolinas fjmolinas force-pushed the pr_tests_riotboot_py branch from 9e0198c to 02070fe Compare July 16, 2019 06:48
@fjmolinas fjmolinas added CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR and removed CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR labels Jul 16, 2019
@fjmolinas fjmolinas force-pushed the pr_tests_riotboot_py branch from 02070fe to 5b2c75c Compare July 16, 2019 07:01
@fjmolinas fjmolinas added CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR and removed CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR labels Jul 16, 2019
@fjmolinas fjmolinas force-pushed the pr_tests_riotboot_py branch from 5b2c75c to dbc8b07 Compare July 17, 2019 09:30
@fjmolinas fjmolinas added the CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR label Jul 17, 2019
@fjmolinas fjmolinas force-pushed the pr_tests_riotboot_py branch 2 times, most recently from c79cbc3 to 3b942a3 Compare July 24, 2019 13:40
@fjmolinas fjmolinas requested a review from aabadie July 24, 2019 13:40
@fjmolinas fjmolinas force-pushed the pr_tests_riotboot_py branch from 3b942a3 to a8c8bcc Compare July 24, 2019 13:46
@fjmolinas fjmolinas mentioned this pull request Aug 7, 2019
Closed
@fjmolinas fjmolinas added CI: disable test cache If set, CI will always run all tests regardless of whether they have been run successfully before CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR CI: run tests If set, CI server will run tests on hardware for the labeled PR and removed CI: disable test cache If set, CI will always run all tests regardless of whether they have been run successfully before CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR CI: run tests If set, CI server will run tests on hardware for the labeled PR labels Aug 8, 2019
@fjmolinas
Copy link
Contributor Author

@kaspar030 I have an Issue where running tests doesn't seem deterministic. I re-launched the tests although last run was successful, and although no code changes have been made, some tests failed this time, interestingly not on the same platform and in one case it succeeded on gnu and the other on lvm. Could it be that the term output isn't getting captured properly? It seems it detects the wrong image version at times.

Also I have seen tests/nordic_softdevice fail a couple of times, seen also in #11956.

@miri64
Copy link
Member

miri64 commented Aug 8, 2019

On a side note could it be possible to add a test progress bar to murdock? its confusing to see the completion at 100% at have it go on for 20 more minutes of testing.

I think it's more visible if you write these kind of requests to ci@riot-os.org ;-)

@fjmolinas
Copy link
Contributor Author

I think it's more visible if you write these kind of requests to ci@riot-os.org ;-)

hmmm.. for some reason the email is bouncing back.

@fjmolinas fjmolinas force-pushed the pr_tests_riotboot_py branch 2 times, most recently from dfea8b9 to eb7eaa8 Compare October 3, 2019 21:54
@fjmolinas fjmolinas added CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR and removed CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR labels Oct 3, 2019
@fjmolinas fjmolinas force-pushed the pr_tests_riotboot_py branch 2 times, most recently from 5a40106 to 7066afd Compare November 27, 2019 09:39
@fjmolinas fjmolinas mentioned this pull request Nov 27, 2019
Merged
@fjmolinas fjmolinas force-pushed the pr_tests_riotboot_py branch from 7066afd to 39467ba Compare November 27, 2019 14:09
@fjmolinas fjmolinas requested review from aabadie and removed request for aabadie November 27, 2019 18:27
aabadie
aabadie requested changes Nov 27, 2019
View reviewed changes
Copy link
Contributor

@aabadie aabadie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few minor comments, otherwise that looks good!

@fjmolinas
Copy link
Contributor Author

A few minor comments, otherwise that looks good!

Can I squash directly?

@aabadie
Copy link
Contributor

aabadie commented Nov 28, 2019

Can I squash directly?

Sure!

@fjmolinas fjmolinas force-pushed the pr_tests_riotboot_py branch from 39467ba to 64fbe19 Compare November 28, 2019 08:56
@fjmolinas fjmolinas removed the CI: disable test cache If set, CI will always run all tests regardless of whether they have been run successfully before label Nov 28, 2019
@fjmolinas fjmolinas force-pushed the pr_tests_riotboot_py branch 2 times, most recently from 2aea9c8 to eba1a5c Compare November 28, 2019 11:59
aabadie
aabadie reviewed Nov 28, 2019
View reviewed changes
@aabadie aabadie removed the CI: run tests If set, CI server will run tests on hardware for the labeled PR label Nov 28, 2019
@fjmolinas fjmolinas force-pushed the pr_tests_riotboot_py branch from eba1a5c to d632d9a Compare December 1, 2019 21:43
aabadie
aabadie requested changes Dec 2, 2019
View reviewed changes
Copy link
Contributor

@aabadie aabadie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still some typos.

The other changes are good and work (tested locally on samr21-xpro).

@fjmolinas
tests/riotboot: migrate to full python test script
1276193
@fjmolinas
tests/riotboot: add shell_commands
e41cebb 
Add shell_commands to use shell version on `test_utils_interactive_sync`
@fjmolinas fjmolinas force-pushed the pr_tests_riotboot_py branch from d632d9a to e41cebb Compare December 2, 2019 16:27
aabadie
aabadie approved these changes Dec 2, 2019
View reviewed changes
Copy link
Contributor

@aabadie aabadie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All good now.

ACK

@aabadie aabadie merged commit 96d9020 into RIOT-OS:master Dec 2, 2019
@fjmolinas fjmolinas added this to the Release 2020.01 milestone Dec 13, 2019
@fjmolinas fjmolinas deleted the pr_tests_riotboot_py branch July 31, 2020 07:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@aabadie aabadie aabadie approved these changes

@kaspar030 kaspar030 Awaiting requested review from kaspar030

Assignees
No one assigned
Labels
Area: OTA Area: Over-the-air updates Area: tests Area: tests and testing framework CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR Type: cleanup The issue proposes a clean-up / The PR cleans-up parts of the codebase / documentation
Projects
None yet
Milestone
Release 2020.01
Development

Successfully merging this pull request may close these issues.
3 participants
@fjmolinas @miri64 @aabadie