Skip to content

Conversation

ZyX-I
Copy link
Contributor

@ZyX-I ZyX-I commented Oct 5, 2015

Current busted output type does not allow determining failing test.

Discussion from ZyX-I@99aef30 (this commit was gone after rebasing and I am not sure when GH will clear it):

@justinmk, ZyX-I@99aef30#commitcomment-13584401

This should be reverted before merging. Travis has a log size limit, which we have exceeded at times.

@ZyX-I, ZyX-I@99aef30#commitcomment-13588275

I see only “soft” line limit being exceeded (number of lines which may be displayed in web interface, does not affect raw log). Hard limit (maximum log size, MiB) is not, and also soft limit ends after .ci/script.sh finished, so necessary information is still seen. I think that either this or handler from #2156 should be used always in order to prevent such situations like #2156 (comment):

I changed the output handler temporarily in #2076 just to discover which test was hanging in clang/osx

which should be read as “I had to do a bunch of actions and wait a lot of time just to discover which test was hanging”, same thing will apply for tests crashing lua.

@ZyX-I, ZyX-I@99aef30#commitcomment-13588385

(By the way, utfTerminalDetailed looks nicer and is less noisy; I always use exactly it locally, though named “utf”. But it was removed in 484fd73 by @tarruda.)

@ZyX-I, ZyX-I@99aef30#commitcomment-13588844

And, by the way:

 zyx  …/Proj/c/neovim  LANG=C wc -l build/log.*.txt
    522 build/log.1.txt
   9416 build/log.2.txt
   9423 build/log.3.txt
   8354 build/log.4.txt
   2532 build/log.5.txt
   4046 build/log.6.txt
   9766 build/log.7.txt
  10157 build/log.8.txt
    633 build/log.9.txt
  54849 total
 zyx  …/Proj/c/neovim  LANG=C la build/log.*.txt
-rw-r--r-- 1 zyx   47K Oct  5 13:25 build/log.1.txt
-rw-r--r-- 1 zyx  899K Oct  5 13:25 build/log.2.txt
-rw-r--r-- 1 zyx  900K Oct  5 13:25 build/log.3.txt
-rw-r--r-- 1 zyx  871K Oct  5 13:25 build/log.4.txt
-rw-r--r-- 1 zyx  166K Oct  5 13:25 build/log.5.txt
-rw-r--r-- 1 zyx  447K Oct  5 13:26 build/log.6.txt
-rw-r--r-- 1 zyx 1020K Oct  5 13:26 build/log.7.txt
-rw-r--r-- 1 zyx 1007K Oct  5 13:26 build/log.8.txt
-rw-r--r-- 1 zyx   55K Oct  5 13:26 build/log.9.txt

: this is for build 11543, logs downloaded using for ((I=1; I<10; I++)){travis logs --no-stream 11543.$I > log.$I.txt}. When there is gcov enabled (and it is enabled) it takes 2457 (11543.2) lines which are much more useless then verbose test report since half of the lines contains no useful information (blank line and “Creating '*.gcov'”), other better be joined together, and some are even more useless because of being repeated or talk about something strange:

/home/travis/build/neovim/neovim/build/src/nvim/CMakeFiles/libnvim.dir/fileio.c.gcda:cannot open > data file, assuming not executed
File '/home/travis/build/neovim/neovim/src/nvim/fileio.c'
No executable lines
Removing 'fileio.c.gcov'
…
File '/home/travis/build/neovim/neovim/src/nvim/fileio.c'
Lines executed:64.24% of 3249
Creating 'fileio.c.gcov'
…
File '/home/travis/build/neovim/neovim/src/nvim/fileio.c'
Lines executed:2.92% of 3249
Creating 'fileio.c.gcov'

: three instances of almost each file, header files may be repeated more times (e.g. count ascii.h).

@ZyX-I ZyX-I changed the title travis: Use gtest for busted output type [RFC] travis: Use gtest for busted output type Oct 5, 2015
@ZyX-I ZyX-I added test ci automation for build, test, and release and removed test labels Oct 5, 2015
Current busted output type does not allow determining failing test.
@justinmk
Copy link
Member

I see only “soft” line limit being exceeded (number of lines which may be displayed in web interface, does not affect raw log). Hard limit (maximum log size, MiB) is not, and also soft limit ends after .ci/script.sh finished, so necessary information is still seen.

👍

When there is gcov enabled (and it is enabled) it takes 2457 (11543.2) lines which are much more useless then verbose test report

gcov is an ongoing pain.

justinmk added a commit that referenced this pull request Oct 17, 2015
travis: Use gtest for busted output type
@justinmk justinmk merged commit 75aae5f into neovim:master Oct 17, 2015
@justinmk justinmk removed the RFC label Oct 17, 2015
@ZyX-I ZyX-I deleted the verbose-test-report branch December 20, 2015 00:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci automation for build, test, and release
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants