-
Notifications
You must be signed in to change notification settings - Fork 37.7k
Ugly hack to print out tests as they are run to mitigate travis timeouts #8633
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
…uts. Someone with more knowledge here should fix this
@theuni @MarcoFalke agree with this approach? |
@theuni try with #8632; the problem (for me) was super slow prevector_tests. I think that I prefer my hack given the output is "-l test_suite" which shows you the names of the tests as they go by, not just an arbitrary progress meter. (I prefer how you get the progress bar to my hack. My make-fu was not strong enough to figure that out. Can you get the loglevel set to -l test_suite and tee it?) |
I think I found a nice compromise: master...JeremyRubin:test-driver-hack2 This version only runs the If this is preferred I can PR it. |
Travis will time out after 10 min and if the unit tests take longer than that, we have other issues. Apart from that, I think it is useful to see how long each test section takes. |
@MarcoFalke Well, sometimes it can be hard to reproduce why a test is timing out locally & eat a lot of dev time to build windows or whatever. This also means that if tests ever grow to take more than 10 minutes (just because there are a lot of them) it will keep working so long as no individual test is that long. (If only we had so many tests). |
I'd prefer that This is probably what you meant with
I am fine with this being merged "temporarily". |
I'm not so sure. Tests that pass ideally should generate no output, to make it more efficient to find what went wrong. It's frustrating to have to browse through thousands of debug messages to find out that error that caused travis to fail. For temporary debugging it's ok of course. |
Ok, then this seems too controversial... If the only goal is to prevent travis from timing out, there is |
@JeremyRubin Closing as this seems too controversial. Though, I think something like master...JeremyRubin:test-driver-hack2 or 195f143 could go in easier if you still need it. |
This hack edits the test-driver to output the test suite's stdout (tee'ing it to the file rather than redirecting) and passes -l test_suite so that stuck tests can be seen.
Some hack like this is needed in order for test suites that take a long time, travis will assume they are dead and fail after 10 minutes. (This also helps debug which unittests may be slow). There may be a cleaner way, but as far as I can tell, this is the simplest.