Compiler crashes while analyzing fused method with optional arguments only on Python 3.11 #5894
Description
Describe the bug
I've observed that Cython crashes when analyzing a cdef class with a cdef method taking fused type arguments and optional arguments, but not always consistently, and only on Python 3.11. It seemed to affect all versions of Cython 3, including the current master branch.
Error compiling Cython file:
------------------------------------------------------------
...
return self._discover_peaks(mz_array, intensity_array, _start_mz, _stop_mz, _start_index, _stop_index)
@cython.nonecheck(False)
@cython.cdivision(True)
@cython.boundscheck(False)
cpdef size_t _discover_peaks(self, np.ndarray[cython.floating, ndim=1, mode='c'] mz_array, np.ndarray[cython.floating, ndim=1, mode='c'] intensity_array,
^
------------------------------------------------------------
src\ms_peak_picker\_c\peak_picker.pyx:248:10: Compiler crash in AnalyseDeclarationsTransform
ModuleNode.body = StatListNode(peak_picker.pyx:1:0)
StatListNode.stats[22] = StatListNode(peak_picker.pyx:140:5)
StatListNode.stats[0] = CClassDefNode(peak_picker.pyx:140:5,
as_name = 'PeakProcessor',
class_name = 'PeakProcessor',
module_name = '',
punycode_class_name = 'PeakProcessor',
visibility = 'private')
CClassDefNode.body = StatListNode(peak_picker.pyx:142:4)
StatListNode.stats[8] = CompilerDirectivesNode(peak_picker.pyx:248:10)
CompilerDirectivesNode.body = StatListNode(peak_picker.pyx:248:10)
StatListNode.stats[0] = CFuncDefNode(peak_picker.pyx:248:10,
args = [...]/7,
doc = 'Carries out the peak picking process on `mz_array` and `intensity_array`. All\n peaks picked are appended to :attr:`peak_data`.\n\n Parameters\n ----------\n mz_array : np.ndarray\n The m/z values to pick peaks from\n intensity_array : np.ndarray\n The intensity values to pick peaks from\n start_mz : float, optional\n The minimum m/z to pick peaks above\n stop_mz : float, optional\n
The maximum m/z to pick peaks below\n\n Returns\n -------\n int\n The current number of peaks accumulated\n ',
has_fused_arguments = True,
is_c_class_method = 1,
modifiers = [...]/0,
outer_attrs = [...]/2,
overridable = 1,
visibility = 'private')
File 'FusedNode.py', line 60, in __init__: FusedCFuncDefNode(peak_picker.pyx:248:10,
fused_compound_types = [...]/1,
match = "dest_sig[{{dest_sig_idx}}] = '{{specialized_type_name}}'",
no_match = 'dest_sig[{{dest_sig_idx}}] = None',
nodes = [...]/2)
File 'FusedNode.py', line 201, in copy_cdef: FusedCFuncDefNode(peak_picker.pyx:248:10,
fused_compound_types = [...]/1,
match = "dest_sig[{{dest_sig_idx}}] = '{{specialized_type_name}}'",
no_match = 'dest_sig[{{dest_sig_idx}}] = None',
nodes = [...]/2)
Compiler crash traceback from this point on:
File "c:\users\joshua\dev\cython\Cython\Compiler\FusedNode.py", line 200, in copy_cdef
cindex = env.cfunc_entries.index(self.node.entry)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "c:\users\joshua\dev\cython\Cython\Compiler\Symtab.py", line 255, in __repr__
return "%s(<%x>, name=%s, type=%s)" % (type(self).__name__, id(self), self.name, self.type)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
File "c:\users\joshua\dev\cython\Cython\Compiler\PyrexTypes.py", line 299, in __str__
return self.declaration_code("", for_display = 1).strip()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "c:\users\joshua\dev\cython\Cython\Compiler\PyrexTypes.py", line 3297, in declaration_code
arg_decl_list.append(self.op_arg_struct.declaration_code(Naming.optional_args_cname))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'declaration_code'
As the traceback seems to show, the op_arg_struct attribute of the CFuncType node behind the declaration is left as None. I was not able to trace out where it should have been set.
The method signature in question is
@cython.nonecheck(False)
@cython.cdivision(True)
@cython.boundscheck(False)
cpdef size_t _discover_peaks(self, np.ndarray[cython.floating, ndim=1, mode='c'] mz_array, np.ndarray[cython.floating, ndim=1, mode='c'] intensity_array,
double start_mz, double stop_mz, Py_ssize_t start_index=0, Py_ssize_t stop_index=0) noexcept:
"""Carries out the peak picking process on `mz_array` and `intensity_array`. All
peaks picked are appended to :attr:`peak_data`.
If the default arguments are removed, the code compiles without issue.
Code to reproduce the behaviour:
No response
Expected behaviour
No response
OS
Windows, Linux
Python version
3.11.5
Cython version
<= 3.0.6
Additional context
The project compiles without issue on Python 3.8-3.10 without issue. The full project is https://github.com/mobiusklein/ms_peak_picker, with the tag for the last released version with the defect https://github.com/mobiusklein/ms_peak_picker/releases/tag/v0.1.43
I was unable reduce this to a minimal example as it seemed to interact with Cython.Compiler.FusedNode the env context variable passed through much of Cython which holds state beyond just the class.