Skip to content

[WIP] tests/test_tools: add a test for the testing tools environment #11094

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 3 commits into from
Closed

[WIP] tests/test_tools: add a test for the testing tools environment #11094

wants to merge 3 commits into from

Conversation

cladmi
Copy link
Contributor

@cladmi cladmi commented Mar 4, 2019 •
edited
Loading

Contribution description

Check the interaction with a board through make term.
It is using a shell without echo or prompt for reference.

It currently checks:

  • that there is no local echo
  • Get the output of a one line command without other garbage messages.

Warning

This is currently not ready for merging as it is waiting for a fix on make term.
But it should show the issue. I removed the test from #10952 so made this pull request for tracking the issue.

Testing procedure

Testing on all the different environments.

There is currently a HACK for picocom and pyterm which should be correctly fixed.

BOARD=native make -C tests/test_tools/ flash test
BOARD=samr21-xpro make -C tests/test_tools/ flash test
BOARD=samr21-xpro RIOT_TERMINAL=picocom make -C tests/test_tools/ flash test
BOARD=samr21-xpro RIOT_TERMINAL=socat make -C tests/test_tools/ test

Issues/PRs references

Test split out of #10952 and depending on it.
Found while writing tests in #10949

Depends on : "adding make rawterm"

@cladmi cladmi added Type: bug The issue reports a bug / The PR fixes a bug (including spelling errors) State: WIP State: The PR is still work-in-progress and its code is not in its final presentable form yet Area: tests Area: tests and testing framework CI: needs squashing Commits in this PR need to be squashed; If set, CI systems will mark this PR as unmergable State: waiting for other PR State: The PR requires another PR to be merged first CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR Area: tools Area: Supplementary tools CI: run tests If set, CI server will run tests on hardware for the labeled PR Type: tracking The issue tracks and organizes the sub-tasks of a larger effort labels Mar 4, 2019
@cladmi cladmi added this to the Release 2019.04 milestone Mar 4, 2019
@cladmi cladmi mentioned this pull request Mar 4, 2019
Merged
2 tasks
@cladmi cladmi requested a review from jcarrano March 4, 2019 12:41
@cladmi
Copy link
Contributor Author

cladmi commented Mar 4, 2019

There is no need to currently review this, but it should be used as a base for testing a fix for make term.

@jcarrano jcarrano mentioned this pull request Mar 4, 2019
Closed
1 task
@cladmi cladmi force-pushed the pr/test_tools/assert_term_does_not_local_echo branch from ca9376e to 828499e Compare March 4, 2019 15:39
@cladmi cladmi force-pushed the pr/test_tools/assert_term_does_not_local_echo branch from 828499e to 1826c53 Compare August 22, 2019 10:27
cladmi added 3 commits August 23, 2019 12:14
@cladmi
tests/test_tools: add a test for the testing tools environment
b3cb076 
Check the interaction with a board through make term.
It is using a shell without echo or prompt for reference.

It currently checks that there is no local echo.
@cladmi
tests/test_tools: test to get a command return value
8049ab9 
Get the output of a one line command without other garbage messages.
@cladmi
FIX ME PROPERLY HACK to remove garbage on pyterm for boards
a998291 
I know this does not work for boards that overwrite 'TERMFLAGS', it is
not a fix, just a hack to show what is expected.
@cladmi cladmi force-pushed the pr/test_tools/assert_term_does_not_local_echo branch from 1826c53 to a998291 Compare August 23, 2019 10:15
This was referenced Aug 23, 2019
Merged
Closed
@kb2ma kb2ma modified the milestones: Release 2019.10, Release 2020.01 Oct 9, 2019
@leandrolanzieri leandrolanzieri removed this from the Release 2020.04 milestone Mar 17, 2020
@miri64
Copy link
Member

miri64 commented Jun 25, 2020

Could still be valuable. Anyone willing to pick up @cladmi's work? @leandrolanzieri @fjmolinas @aabadie?

@cladmi
Copy link
Contributor Author

cladmi commented Jun 28, 2020

I think everything is already in https://github.com/RIOT-OS/RIOT/blob/master/tests/test_tools/tests/01-run.py. The tests were merged along the bug fixes.

The "increment" from here was replaced by an "uppercase" as there were issues with matching numbers in pyterm output, but the check is the same.

@cladmi cladmi closed this Jun 28, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jcarrano jcarrano Awaiting requested review from jcarrano

Assignees
No one assigned
Labels
Area: tests Area: tests and testing framework Area: tools Area: Supplementary tools CI: needs squashing Commits in this PR need to be squashed; If set, CI systems will mark this PR as unmergable CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR CI: run tests If set, CI server will run tests on hardware for the labeled PR State: waiting for other PR State: The PR requires another PR to be merged first State: WIP State: The PR is still work-in-progress and its code is not in its final presentable form yet Type: bug The issue reports a bug / The PR fixes a bug (including spelling errors) Type: tracking The issue tracks and organizes the sub-tasks of a larger effort
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@cladmi @miri64 @kb2ma @leandrolanzieri @fjmolinas