I've been getting to the bottom of why tracebacks aren't displaying Cython sources on Python 3. The problem turns out to be multi-faceted, and fixing it right would also mean fixing some things in general, since the way it works now is a hack.

Basically, the only reason it works at all on Python 2, is that the linecache module on Python 2 has some code which, given a relative filename (with an extension unrecognized by the import system) like sage/rings/integer.pyx, it will search for this file under all sys.path entries and, if found, read the source lines from that file.

This, in turn, only works for Sage because we actually install the .pyx sources in the sage package.

This does not work in Python 3. In linecache.updatecache, before it tries the sys.path search, it checks if the module object has a __loader__, and calls its get_source() method if it exists. On Python 2 this isn't a problem since modules don't necessarily have a __loader__, and in particular extension modules don't. But on the reworked import system in Python 3, pretty much every module has a __loader__--in the case of extension modules an ExtensionFileLoader. But the built-in ExtensionFileLoader of course knows nothing about Cython so its get_source() method just returns None. linecache.updatecache assumes this is correct (why would the loader lie?) and returns.

The simplest way to fix this is to remove the get_source() method from the ExtensionFileLoader class. This way, Python 3 works the same way as Python 2.

Upstream:

• https://bugs.python.org/issue32797
• bpo-32797: linecache should search for sources if loader doesn't find them python/cpython#6653

Upstream: Reported upstream. No feedback yet.

Component: cython

Author: Jeroen Demeyer, Erik Bray

Branch: 1a9225f

Reviewer: Frédéric Chapoton

Issue created by migration from https://trac.sagemath.org/ticket/24681