The symptom

On Ubuntu 18.04, with SageMath version 9.6.beta6, Release Date: 2022-03-27, with the GLPK available on the system $ glpsol -v returning GLPSOL: GLPK LP/MIP Solver, v4.65, we observe doctest failures of the following kind:

Failed example:
    lp.solve() # tol 1e-8   # optional - sage.rings.number_field
Expected:
    1.3090169943749475
Got:
    Long-step dual simplex will be used
    1.3090169943749472

Failures always consists of doctests tagged with some numerical tolerence # tol. It seems the fix proposed in #29317 to ignore sentence Long-step dual simplex will be used printed by GLPK does not work for doctests tagged with # tol.

The complete list of failures consist of the following 4 files:

sage -tp --long src/sage/geometry/polyhedron/base.py src/sage/numerical/backends/glpk_backend.pyx src/sage/numerical/mip.pyx src/sage/tests/books/computational-mathematics-with-sagemath/lp_doctest.py

gives

----------------------------------------------------------------------
sage -t --long src/sage/geometry/polyhedron/base.py  # 1 doctest failed
sage -t --long src/sage/numerical/backends/glpk_backend.pyx  # 4 doctests failed
sage -t --long src/sage/numerical/mip.pyx  # 8 doctests failed
sage -t --long src/sage/tests/books/computational-mathematics-with-sagemath/lp_doctest.py  # 2 doctests failed
----------------------------------------------------------------------

Isolation of the problem

The problem was understood in comment:7. The warnings are not ignored by check_output method when tolerance tags are given. The following example illustrate the issue, the last line should be True:

sage: from sage.doctest.parsing import SageOutputChecker, SageDocTestParser
sage: import doctest
sage: optflag = doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS
sage: DTP = SageDocTestParser(('sage','magma','guava'))
sage: OC = SageOutputChecker()
sage: example = "sage: 1.3090169943749475 # tol 1e-8\n1.3090169943749475"
sage: ex = DTP.parse(example)[1]
sage: ex.sage_source
'1.3090169943749475 # tol 1e-8\n'
sage: ex.want
'1.3090169943749475\n'
sage: OC.check_output(ex.want, '1.3090169943749475', optflag)
True
sage: OC.check_output(ex.want, 'ANYTHING1.3090169943749475', optflag)
False
sage: OC.check_output(ex.want, 'Long-step dual simplex will be used\n1.3090169943749475', optflag)
False

The problem with the current code

It can be checked inside of the check_output method at line 925 and below that when a tolerance is given a result True or False is returned before any fixup is performed to remove eventual Long-step dual simplex will be used to the output.

More precisely, at line 934, we return False because everything other than float numbers are not equal. This return False is done too early. Because we want to do eventual fix ups before returning False.

The solution

The solution is not to return False at line 934 when the non-float part is not equal. In fact it is better to first check that all floats intervals intersect (currently done at line 942) and return False if not all intervals intersect.

Then, we can continue with checking the remaining non-floats parts as it is done by default when no tolerance is given instead of returning False.

Further improvements

We create a independent method do_fixup which does the removal of eventual warnings as a last resort solution before returning False.

To ease the review, we keep the list _repr_fixups for now in the header of the module and leave its integration within do_fixup method for the follow-up ticket #33680.

CC: @kiwifb

Component: numerical

Author: Sébastien Labbé

Branch/Commit: 2afb472

Reviewer: John Palmieri

Issue created by migration from https://trac.sagemath.org/ticket/33588