Skip to content

Conversation

JeremyRubin
Copy link
Contributor

This hack edits the test-driver to output the test suite's stdout (tee'ing it to the file rather than redirecting) and passes -l test_suite so that stuck tests can be seen.

Some hack like this is needed in order for test suites that take a long time, travis will assume they are dead and fail after 10 minutes. (This also helps debug which unittests may be slow). There may be a cleaner way, but as far as I can tell, this is the simplest.

…uts. Someone with more knowledge here should fix this
@fanquake fanquake added the Tests label Aug 31, 2016
@laanwj
Copy link
Member

laanwj commented Sep 1, 2016

@theuni @MarcoFalke agree with this approach?

@theuni
Copy link
Member

theuni commented Sep 1, 2016

I'm hitting a slightly different problem here where the boost test itself appears to be taking >10min. I addressed that here: 195f143, which simply skips the automake test harness altogether for travis. Assuming that works, I think I'd prefer that approach.

@JeremyRubin
Copy link
Contributor Author

JeremyRubin commented Sep 1, 2016

@theuni try with #8632; the problem (for me) was super slow prevector_tests.

I think that I prefer my hack given the output is "-l test_suite" which shows you the names of the tests as they go by, not just an arbitrary progress meter.

(I prefer how you get the progress bar to my hack. My make-fu was not strong enough to figure that out. Can you get the loglevel set to -l test_suite and tee it?)

@JeremyRubin
Copy link
Contributor Author

I think I found a nice compromise:

master...JeremyRubin:test-driver-hack2

This version only runs the sed command in the travis build script.

If this is preferred I can PR it.

@maflcko
Copy link
Member

maflcko commented Sep 9, 2016

Travis will time out after 10 min and if the unit tests take longer than that, we have other issues.

Apart from that, I think it is useful to see how long each test section takes.

@JeremyRubin
Copy link
Contributor Author

@MarcoFalke Well, sometimes it can be hard to reproduce why a test is timing out locally & eat a lot of dev time to build windows or whatever.

This also means that if tests ever grow to take more than 10 minutes (just because there are a lot of them) it will keep working so long as no individual test is that long. (If only we had so many tests).

@maflcko
Copy link
Member

maflcko commented Sep 9, 2016

I'd prefer that test_bitcoin does this by default. (I know there is test_bitcoin -p, but it does not show the current test and the time it took for each test)

This is probably what you meant with

There may be a cleaner way, but as far as I can tell, this is the simplest.

I am fine with this being merged "temporarily".

@laanwj
Copy link
Member

laanwj commented Sep 22, 2016

I'd prefer that test_bitcoin does this by default. (I know there is test_bitcoin -p, but it does not show the current test and the time it took for each test)

I'm not so sure. Tests that pass ideally should generate no output, to make it more efficient to find what went wrong. It's frustrating to have to browse through thousands of debug messages to find out that error that caused travis to fail.

For temporary debugging it's ok of course.

@maflcko
Copy link
Member

maflcko commented Sep 22, 2016

Ok, then this seems too controversial...

If the only goal is to prevent travis from timing out, there is travis_wait as an alternative https://docs.travis-ci.com/user/common-build-problems/#Build-times-out-because-no-output-was-received

@maflcko
Copy link
Member

maflcko commented Sep 30, 2016

@JeremyRubin Closing as this seems too controversial. Though, I think something like master...JeremyRubin:test-driver-hack2 or 195f143 could go in easier if you still need it.

@maflcko maflcko closed this Sep 30, 2016
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Sep 8, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants