-
-
Notifications
You must be signed in to change notification settings - Fork 654
Description
As discussed in #32973, the default value of --warn-long
is roughly a minute, independent of the --long
flag, and is only enabled when timing information from previous test runs is available. Our developers' guide suggests that even a "long time" test should complete in under five seconds:
https://doc.sagemath.org/html/en/developer/doctesting.html#optional-arguments
This ticket addresses two problems:
- The timing information is often unavailable. We should still warn about long tests in that case.
- The default of 60 CPU-equivalent seconds is much too high when
--long
is not in use.
Both of these issues stem from using the "wall time" to measure how long a test has run. Wall time is inherently unstable; if you're watching a movie, the tests will take longer. On a slower computer, the tests will take longer. Et cetera. The use of timing information from previous runs and the obscenely high --warn-long
limit partially address that.
So in targeting our two issues, we first aim to switch the timing reports and --warn-long
flag to use CPU time instead of wall time. CPU time is more reliable, since it only reports time actually spent working on the problem. (It's still sensitive to things like physical CPU time and CFLAGS
; those will be addressed in #33022). With CPU time being used, the timings will be much more consistent, allowing us to lower the default value of --warn-long
to 10 CPU-seconds when --long
was not given.
The main difficulty is in accounting for CPU time spent in pexpect subprocesses. We don't wait()
for those and they may not terminate, so the OS's usual methods of retrieving child process statistics don't work for them. At present we have a PID-based solution that works quickly on Linux/BSD, but something similar for macOS would be nice.
Component: doctest framework
Work Issues: cputime by PID on macOS
Author: Michael Orlitzky
Branch/Commit: u/mjo/ticket/32981 @ d7d2cff
Reviewer: https://github.com/sagemath/sage/actions/runs/1992568379
Issue created by migration from https://trac.sagemath.org/ticket/32981