-
Notifications
You must be signed in to change notification settings - Fork 28
Description
A segmentation fault can be produced in SageMath (valid in Sage 10.7.beta6) via the following operation:
sage: matrix.zero(GF(4),2,0).with_permuted_rows(Permutation([2,1]))
This calls mzed_row_swap
-> mzd_row_swap
-> _mzd_row_swap
where the first line is this:
if ((rowa == rowb) || (startblock >= M->width)) { return; }
startblock
and M->width
are both expected to be 0
, in which case the function should exit early.
However, instead this check looks like it's failing, and a segmentation fault is produced instead. It is likely M->width
is not being correctly initialised to 0
for this matrix.
The problem also exists for _mzed_col_swap
:
sage: matrix.zero(GF(4),0,2).with_permuted_columns(Permutation([2,1]))
Related PR (SageMath fix for the above issue): sagemath/sage#40291