Fix doctests in sage_setup and sage_docbuild for sage-on-distribution #33141
Description
Some doctests in sage_setup and sage_docbuild fail in sage-on-distributions. We can either
- disabled them on distribution with
# optional - sage_spkgwhen source need to be available - fix them when it make sense
- make test of lists more robust by ordering the results when we encounter them
After including #33064, #33085 and 32856, the failing doctests are
sage -t --long --random-seed=204418260052577960635697341654995479733 /usr/lib/python3.10/site-packages/sage_setup/find.py
**********************************************************************
File "/usr/lib/python3.10/site-packages/sage_setup/find.py", line 243, in sage_setup.find._cythonized_dir
Failed example:
_cythonized_dir(SAGE_SRC, editable_install=False)
Exception raised:
Traceback (most recent call last):
File "/usr/lib/python3.10/site-packages/sage/doctest/forker.py", line 694, in _run
self.compile_and_execute(example, compiler, test.globs)
File "/usr/lib/python3.10/site-packages/sage/doctest/forker.py", line 1088, in compile_and_execute
exec(compiled, globs)
File "<doctest sage_setup.find._cythonized_dir[2]>", line 1, in <module>
_cythonized_dir(SAGE_SRC, editable_install=False)
File "/usr/lib/python3.10/site-packages/sage_setup/find.py", line 261, in _cythonized_dir
return Path(SAGE_ROOT) / "build" / "pkgs" / "sagelib" / "src" / "build" / "cythonized"
File "/usr/lib/python3.10/pathlib.py", line 958, in __new__
self = cls._from_parts(args)
File "/usr/lib/python3.10/pathlib.py", line 592, in _from_parts
drv, root, parts = self._parse_args(args)
File "/usr/lib/python3.10/pathlib.py", line 576, in _parse_args
a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not NoneType
**********************************************************************
File "/usr/lib/python3.10/site-packages/sage_setup/find.py", line 303, in sage_setup.find.find_extra_files
Failed example:
extras["sage/ext/interpreters"]
Expected:
['.../src/sage/ext/interpreters/wrapper_cdf.pxd', ...wrapper_cdf.h...]
Got:
['/usr/lib/python3.10/site-packages/sage/ext/interpreters/wrapper_cc.pyx',
'/usr/lib/python3.10/site-packages/sage/ext/interpreters/wrapper_cc.pxd',
'/usr/lib/python3.10/site-packages/sage/ext/interpreters/wrapper_py.pyx',
'/usr/lib/python3.10/site-packages/sage/ext/interpreters/wrapper_rdf.pxd',
'/usr/lib/python3.10/site-packages/sage/ext/interpreters/wrapper_rr.pyx',
'/usr/lib/python3.10/site-packages/sage/ext/interpreters/wrapper_rr.pxd',
'/usr/lib/python3.10/site-packages/sage/ext/interpreters/wrapper_el.pxd',
'/usr/lib/python3.10/site-packages/sage/ext/interpreters/wrapper_rdf.pyx',
'/usr/lib/python3.10/site-packages/sage/ext/interpreters/wrapper_py.pxd',
'/usr/lib/python3.10/site-packages/sage/ext/interpreters/wrapper_el.pyx',
'/usr/lib/python3.10/site-packages/sage/ext/interpreters/wrapper_cdf.pyx',
'/usr/lib/python3.10/site-packages/sage/ext/interpreters/wrapper_cdf.pxd',
'/usr/lib/python3.10/site-packages/sage/ext/interpreters/wrapper_rr.h',
'/usr/lib/python3.10/site-packages/sage/ext/interpreters/wrapper_cc.h',
'/usr/lib/python3.10/site-packages/sage/ext/interpreters/wrapper_cdf.h',
'/usr/lib/python3.10/site-packages/sage/ext/interpreters/wrapper_el.h']
**********************************************************************
and
$ sage -t --long --random-seed=204418260052577960635697341654995479733 /usr/lib/python3.10/site-packages/sage_docbuild/__init__.py
too many failed tests, not using stored timings
Running doctests with ID 2022-02-07-11-13-00-56bf74ee.
Using --optional=pip,sage
Features to be detected: 4ti2,benzene,bliss,buckygen,conway_polynomials,csdp,database_cremona_ellcurve,database_cremona_mini_ellcurve,database_jones_numfield,database_knotinfo,dvipng,graphviz,imagemagick,jupymake,kenzo,latte_int,lrslib,mcqd,meataxe,pandoc,pdf2svg,plantri,pynormaliz,python_igraph,rubiks,sage.combinat,sage.geometry.polyhedron,sage.graphs,sage.groups,sage.plot,sage.rings.number_field,sage.rings.padics,sage.rings.real_double,sage.symbolic,sage_numerical_backends_coin,sagemath_doc_html,sphinx,tdlib
Doctesting 1 file.
sage -t --long --random-seed=204418260052577960635697341654995479733 /usr/lib/python3.10/site-packages/sage_docbuild/__init__.py
**********************************************************************
File "/usr/lib/python3.10/site-packages/sage_docbuild/__init__.py", line 108, in sage_docbuild.builder_helper
Failed example:
try: # optional - sagemath_doc_html
build_many(build_ref_doc, [("docname", "en", "html", {})])
except Exception as E:
"Non-exception during docbuild: abort pool operation" in str(E)
Expected:
True
Got:
False
**********************************************************************
File "/usr/lib/python3.10/site-packages/sage_docbuild/__init__.py", line 189, in sage_docbuild.DocBuilder._output_dir
Failed example:
b._output_dir('html') # optional - sagemath_doc_html
Exception raised:
Traceback (most recent call last):
File "/usr/lib/python3.10/site-packages/sage/doctest/forker.py", line 694, in _run
self.compile_and_execute(example, compiler, test.globs)
File "/usr/lib/python3.10/site-packages/sage/doctest/forker.py", line 1088, in compile_and_execute
exec(compiled, globs)
File "<doctest sage_docbuild.DocBuilder._output_dir[2]>", line 1, in <module>
b._output_dir('html') # optional - sagemath_doc_html
File "/usr/lib/python3.10/site-packages/sage_docbuild/__init__.py", line 193, in _output_dir
sage_makedirs(d)
File "/usr/lib/python3.10/site-packages/sage/misc/misc.py", line 90, in sage_makedirs
os.makedirs(dirname)
File "/usr/lib/python3.10/os.py", line 225, in makedirs
mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/usr/share/doc/sage-doc-9999/html/en/tutorial'
**********************************************************************
File "/usr/lib/python3.10/site-packages/sage_docbuild/__init__.py", line 206, in sage_docbuild.DocBuilder._doctrees_dir
Failed example:
b._doctrees_dir() # optional - sagemath_doc_html
Exception raised:
Traceback (most recent call last):
File "/usr/lib/python3.10/site-packages/sage/doctest/forker.py", line 694, in _run
self.compile_and_execute(example, compiler, test.globs)
File "/usr/lib/python3.10/site-packages/sage/doctest/forker.py", line 1088, in compile_and_execute
exec(compiled, globs)
File "<doctest sage_docbuild.DocBuilder._doctrees_dir[2]>", line 1, in <module>
b._doctrees_dir() # optional - sagemath_doc_html
File "/usr/lib/python3.10/site-packages/sage_docbuild/__init__.py", line 210, in _doctrees_dir
sage_makedirs(d)
File "/usr/lib/python3.10/site-packages/sage/misc/misc.py", line 90, in sage_makedirs
os.makedirs(dirname)
File "/usr/lib/python3.10/os.py", line 215, in makedirs
makedirs(head, exist_ok=exist_ok)
File "/usr/lib/python3.10/os.py", line 215, in makedirs
makedirs(head, exist_ok=exist_ok)
File "/usr/lib/python3.10/os.py", line 225, in makedirs
mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/usr/share/doc/sage-doc-9999/doctrees'
**********************************************************************
File "/usr/lib/python3.10/site-packages/sage_docbuild/__init__.py", line 377, in sage_docbuild.AllBuilder.get_all_documents
Failed example:
'en/tutorial' in documents
Expected:
True
Got:
False
**********************************************************************
File "/usr/lib/python3.10/site-packages/sage_docbuild/__init__.py", line 559, in sage_docbuild.ReferenceBuilder.get_all_documents
Failed example:
refdir = os.path.join(os.environ['SAGE_DOC_SRC'], 'en', b.name)
Exception raised:
Traceback (most recent call last):
File "/usr/lib/python3.10/site-packages/sage/doctest/forker.py", line 694, in _run
self.compile_and_execute(example, compiler, test.globs)
File "/usr/lib/python3.10/site-packages/sage/doctest/forker.py", line 1088, in compile_and_execute
exec(compiled, globs)
File "<doctest sage_docbuild.ReferenceBuilder.get_all_documents[2]>", line 1, in <module>
refdir = os.path.join(os.environ['SAGE_DOC_SRC'], 'en', b.name)
File "/usr/lib/python3.10/os.py", line 679, in __getitem__
raise KeyError(key) from None
KeyError: 'SAGE_DOC_SRC'
**********************************************************************
File "/usr/lib/python3.10/site-packages/sage_docbuild/__init__.py", line 560, in sage_docbuild.ReferenceBuilder.get_all_documents
Failed example:
sorted(b.get_all_documents(refdir))
Exception raised:
Traceback (most recent call last):
File "/usr/lib/python3.10/site-packages/sage/doctest/forker.py", line 694, in _run
self.compile_and_execute(example, compiler, test.globs)
File "/usr/lib/python3.10/site-packages/sage/doctest/forker.py", line 1088, in compile_and_execute
exec(compiled, globs)
File "<doctest sage_docbuild.ReferenceBuilder.get_all_documents[3]>", line 1, in <module>
sorted(b.get_all_documents(refdir))
NameError: name 'refdir' is not defined
**********************************************************************
File "/usr/lib/python3.10/site-packages/sage_docbuild/__init__.py", line 1050, in sage_docbuild.ReferenceSubBuilder.auto_rest_filename
Failed example:
ReferenceSubBuilder("reference").auto_rest_filename("sage.combinat.partition")
Expected:
'.../doc/en/reference/sage/combinat/partition.rst'
Got:
'/usr/share/doc/sage-doc-9999/en/reference/sage/combinat/partition.rst'
**********************************************************************
Depends on #33064
Depends on #33085
Depends on #32856
CC: @antonio-rojas
Component: distribution
Author: François Bissey
Branch/Commit: 5382264
Reviewer: Matthias Koeppe
Issue created by migration from https://trac.sagemath.org/ticket/33141