Skip to content

tests/*: pexpect: always expect before sendline #9220

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 2 commits into from
Closed

tests/*: pexpect: always expect before sendline #9220

wants to merge 2 commits into from

Conversation

ZetaR60
Copy link
Contributor

@ZetaR60 ZetaR60 commented May 29, 2018

This is my attempt at fixing problems with the Murdock board testing. It intermittently fails on a few of the tests:

--- run_test job results (3 failed, 71 passed, 74 total):
    failed:
    tests/ps_schedstatistics/samr21-xpro
    tests/struct_tm_utility/samr21-xpro
    tests/shell/samr21-xpro

    passed:
    tests/gnrc_ipv6_nib_6ln/samr21-xpro
    tests/xtimer_now64_continuity/samr21-xpro
    tests/buttons/samr21-xpro
    tests/thread_msg_block_wo_queue/samr21-xpro
    tests/rmutex/samr21-xpro
    tests/posix_time/samr21-xpro
    tests/sched_testing/samr21-xpro
    tests/evtimer_msg/samr21-xpro
    tests/sizeof_tcb/samr21-xpro
    tests/pkg_micro-ecc/samr21-xpro
    tests/pthread/samr21-xpro
    tests/xtimer_periodic_wakeup/samr21-xpro
    tests/trickle/samr21-xpro
    tests/xtimer_msg/samr21-xpro
    tests/pthread_tls/samr21-xpro
    tests/cb_mux/samr21-xpro
    tests/gnrc_ndp/samr21-xpro
    tests/thread_msg/samr21-xpro
    tests/isr_yield_higher/samr21-xpro
    tests/lwip_sock_ip/samr21-xpro
    tests/xtimer_hang/samr21-xpro
    tests/thread_msg_block_w_queue/samr21-xpro
    tests/posix_semaphore/samr21-xpro
    tests/nhdp/samr21-xpro
    tests/thread_flags_xtimer/samr21-xpro
    tests/xtimer_reset/samr21-xpro
    tests/thread_flags/samr21-xpro
    tests/gnrc_ipv6_nib/samr21-xpro
    tests/cpp11_condition_variable/samr21-xpro
    tests/gnrc_sock_udp/samr21-xpro
    tests/msg_avail/samr21-xpro
    tests/evtimer_underflow/samr21-xpro
    tests/netdev_test/samr21-xpro
    tests/thread_flood/samr21-xpro
    tests/xtimer_usleep_short/samr21-xpro
    tests/periph_timer/samr21-xpro
    tests/fmt_print/samr21-xpro
    tests/xtimer_remove/samr21-xpro
    tests/pthread_cleanup/samr21-xpro
    tests/pthread_barrier/samr21-xpro
    tests/mutex_unlock_and_sleep/samr21-xpro
    tests/pkg_libcoap/samr21-xpro
    tests/events/samr21-xpro
    tests/irq/samr21-xpro
    tests/bloom_bytes/samr21-xpro
    tests/xtimer_msg_receive_timeout/samr21-xpro
    tests/xtimer_usleep/samr21-xpro
    tests/thread_race/samr21-xpro
    tests/pkg_cayenne-lpp/samr21-xpro
    tests/mutex_order/samr21-xpro
    tests/pkg_tinycbor/samr21-xpro
    tests/cpp11_thread/samr21-xpro
    tests/float/samr21-xpro
    tests/pkg_jsmn/samr21-xpro
    tests/thread_cooperation/samr21-xpro
    tests/pthread_cooperation/samr21-xpro
    tests/ssp/samr21-xpro
    tests/pkg_tiny-asn1/samr21-xpro
    tests/thread_exit/samr21-xpro
    tests/pkg_minmea/samr21-xpro
    tests/msg_try_receive/samr21-xpro
    tests/pthread_condition_variable/samr21-xpro
    tests/pipe/samr21-xpro
    tests/thread_basic/samr21-xpro
    tests/msg_send_receive/samr21-xpro
    tests/bitarithm_timings/samr21-xpro
    tests/cb_mux_bench/samr21-xpro
    tests/od/samr21-xpro
    tests/pkg_umorse/samr21-xpro
    tests/cpp11_mutex/samr21-xpro
    tests/gnrc_sock_ip/samr21-xpro

Those tests that it fails on are exactly those in which the python test script is sending using pexpect as well as receiving. It is intermittent as well, which suggests timing issues with the commands being issued. I have changed the scripts to ensure that before every send, it waits for the > shell prompt.

@aabadie aabadie added Type: bug The issue reports a bug / The PR fixes a bug (including spelling errors) Area: tests Area: tests and testing framework CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR CI: run tests If set, CI server will run tests on hardware for the labeled PR labels May 29, 2018
@ZetaR60 ZetaR60 mentioned this pull request May 29, 2018
Merged
@aabadie aabadie added CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR and removed CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR labels May 29, 2018
@aabadie
Copy link
Contributor

aabadie commented May 29, 2018

@ZetaR60, I triggered Murdock of couple of times, it passed the first time but now it fails again.

@bergzand
Copy link
Member

@ZetaR60 This is an issue with the current CI hardware tests. Mostly caused by flaky serial connections to the boards.

@kaspar030
Copy link
Contributor

@ZetaR60 This is an issue with the current CI hardware tests. Mostly caused by flaky serial connections to the boards.

I think it is caused by the Linux drivers. Usually, a reboot of the raspi makes it work again for a while. ;(

@kYc0o
Copy link
Contributor

kYc0o commented May 29, 2018

I tried to address this issues in the same way as you but we decided that we would tackle this in a different way. I'd suggest to close it and look for a different solution.

@ZetaR60
Copy link
Contributor Author

ZetaR60 commented May 29, 2018

Yeah. It is pretty clear that it thinks it is sending one thing, but the board is getting something else.

@ZetaR60 ZetaR60 closed this May 29, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
Area: tests Area: tests and testing framework CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR CI: run tests If set, CI server will run tests on hardware for the labeled PR Type: bug The issue reports a bug / The PR fixes a bug (including spelling errors)
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@ZetaR60 @aabadie @bergzand @kaspar030 @kYc0o