LaTeX isn't particularly good at typesetting Unicode text. While the utf8 input encoding can help recognizing the most common symbols even if the input is Unicode, the output is essentially restricted to 8-bit encodings so every special symbol has to be set up somewhere. src/doc/common/conf.py has a list of such setup rules, although some are pretty hackish, replacing unicode symbols with stuff that only looks remotely similar.

Unicode in doctest output (and perhaps other parts of the reference manual as well) has caused problems in the past, e.g. ticket:15802 and #18270 comment:18. It is likely to cause more problems in the future, when we make more extensive use of Unicode in Sage output once #18357 gets implemented. So Unicode in the doctests is going to be increasingly common, and it would be nice if we didn't have to jump through hoops to get that represented in the PDF output.

As far as I understand it, both XeTeX / xelatex and luaTeX / lualatex provide a more modern approach to font handling, and considerably better Unicode support. I haven't yet gotten things to work that way, but my hope would be that we could gain access to all the symbols of a sufficiently complete Unicode font. Apparently, current versions of Sphinx don't officially support either of these engines yet. And things don't work out of the box. But perhaps things can be made to work with a bit of patching. If so, that might greatly improve the situation.

See also #14153

CC: @nexttime @embray @jdemeyer @hivert

Component: documentation

Keywords: unicode

Issue created by migration from https://trac.sagemath.org/ticket/18370