DEP: signal: remove support for object arrays and longdoubles in `{correlate,convolve,lfilter,sosfilter}` #23212

j-bowhay · 2025-06-22T20:36:30Z

As part of array api conversion support for object arrays is scheduled for removal in this release.

…al.{correlate,convolve,lfilter,sosfilter}

ev-br

LGTM.

Can also remove some C code, too: https://github.com/scipy/scipy/blob/main/scipy/signal/_lfilter.cc#L24, https://github.com/scipy/scipy/blob/main/scipy/signal/_correlate_nd.cc#L181 and maybe some more from the _sigtools extension.

Can be done either here or in a follow-up though.

j-bowhay · 2025-06-22T21:25:54Z

Done thanks, that made me spot a few areas I'd missed

ev-br · 2025-06-22T21:35:10Z

In fact, also _sosfilt.pyx can be cleaned up, if you're up to it.

scipy/signal/tests/test_signaltools.py

Co-authored-by: Evgeni Burovski <evgeny.burovskiy@gmail.com>

scipy/signal/tests/test_signaltools.py

ev-br · 2025-06-23T15:32:16Z

I think the CI failure is due to this line: https://github.com/scipy/scipy/blob/main/scipy/signal/tests/test_signaltools.py#L2951: butter returns numpy arrays for python scalars. A low-tech solution (used also elsewhere) is to wrap it first or second argument into an xp.asarray/

ev-br · 2025-06-24T09:38:37Z

This patch seems to fix the dask failure:

$ git diff
diff --git a/scipy/signal/_signaltools.py b/scipy/signal/_signaltools.py
index 2cdb6c45c4..b085f758f4 100644
--- a/scipy/signal/_signaltools.py
+++ b/scipy/signal/_signaltools.py
@@ -4755,6 +4755,7 @@ def filtfilt(b, a, x, axis=-1, padtype='odd', padlen=None, method='pad',
     # method == "pad"
     edge, ext = _validate_pad(padtype, padlen, x, axis,
                               ntaps=max(len(a), len(b)))
+    ext = xp.asarray(ext)
 
     # Get the steady state of the filter's step response.
     zi = lfilter_zi(b, a)

EDIT: Sorry, no. The patch above is wrong, and causes other issues elsewhere. A correct patch:

$ git diff
diff --git a/scipy/signal/_signaltools.py b/scipy/signal/_signaltools.py
index 2cdb6c45c4..61460f999b 100644
--- a/scipy/signal/_signaltools.py
+++ b/scipy/signal/_signaltools.py
@@ -4741,8 +4741,8 @@ def filtfilt(b, a, x, axis=-1, padtype='odd', padlen=None, method='pad',
     """
     xp = array_namespace(b, a, x)
 
-    b = np.atleast_1d(b)
-    a = np.atleast_1d(a)
+    b = np.atleast_1d(np.asarray(b))
+    a = np.atleast_1d(np.asarray(a))
     x = np.asarray(x)
 
     if method not in ["pad", "gust"]:
diff --git a/scipy/signal/tests/test_signaltools.py b/scipy/signal/tests/test_signaltools.py
index 8d6f82bedd..f922e5c606 100644
--- a/scipy/signal/tests/test_signaltools.py
+++ b/scipy/signal/tests/test_signaltools.py
@@ -1649,8 +1649,7 @@ class TestResample:
             x[0] = 0
             x[-1] = 0
 
-            h = signal.firwin(31, 1. / down_factor, window='hamming')
-            h = xp.asarray(h)   # XXX: convert firwin
+            h = signal.firwin(31, xp.asarray(1. / down_factor), window='hamming')
             yf = filtfilt(h, 1.0, x, padtype='constant')[::down_factor]
 
             # Need to pass convolved version of filter to resample_poly,

The underlying reason is that np.atleast_1d calls np.asanyarray not np.asarray

ev-br · 2025-06-24T13:14:10Z

Array-api-strict test failure is fixed by this:

@@ -2803,7 +2802,7 @@ class TestFiltFilt:
         if is_jax(xp) and self.filtfilt_kind == 'sos':
             pytest.skip(reason='sosfilt works in-place')
 
-        zpk = tf2zpk(xp.asarray([1, 2, 3]), xp.asarray([1, 2, 3]))
+        zpk = tf2zpk(xp.asarray([1.0, 2, 3]), xp.asarray([1.0, 2, 3]))
         out = self.filtfilt(zpk, xp.arange(12), xp=xp)
         atol= 3e-9 if is_cupy(xp) else 5.28e-11
         xp_assert_close(out, xp.arange(12, dtype=xp.float64), atol=atol)

The what remains is a small tolerance bump on CuPy. Phew.

scipy/signal/tests/test_signaltools.py

j-bowhay · 2025-07-04T19:30:03Z

CI finally green, hopefully this is ready to go!

lucascolley

thanks Jake, great cleanup!

ev-br · 2025-07-05T19:36:25Z

Minus 500 LOC of obscure C code, yay!

j-bowhay · 2025-07-05T19:38:28Z

Is there anywhere else we're hiding compiled code just for handling object arrays?

ev-br · 2025-07-05T20:50:59Z

is there anywhere else we're hiding compiled code just for handling object arrays?

Let's see:

$ git grep NPY_OBJECT     # None found anymoe, yay!
$ git grep "object\["
scipy/sparse/_csparsetools.pyx.in:cpdef lil_get1(cnp.npy_intp M, cnp.npy_intp N, object[:] rows, object[:] datas,
scipy/sparse/_csparsetools.pyx.in:cpdef int lil_insert(cnp.npy_intp M, cnp.npy_intp N, object[:] rows,
scipy/sparse/_csparsetools.pyx.in:                     object[:] datas, cnp.npy_intp i, cnp.npy_intp j,
scipy/sparse/_csparsetools.pyx.in:def lil_get_lengths(object[:] input,
scipy/sparse/_csparsetools.pyx.in:def _lil_get_lengths_{{NAME}}(object[:] input,
scipy/sparse/_csparsetools.pyx.in:def _lil_flatten_to_array_{{NAME}}(object[:] input not None, cnp.ndarray[{{T}}] output not None):
scipy/sparse/_csparsetools.pyx.in:                  object[:] rows,
scipy/sparse/_csparsetools.pyx.in:                  object[:] datas,
scipy/sparse/_csparsetools.pyx.in:                  object[:] new_rows,
scipy/sparse/_csparsetools.pyx.in:                  object[:] new_datas,
scipy/sparse/_csparsetools.pyx.in:                            object[:] rows,
scipy/sparse/_csparsetools.pyx.in:                            object[:] datas,
scipy/sparse/_csparsetools.pyx.in:                            object[:] new_rows,
scipy/sparse/_csparsetools.pyx.in:                            object[:] new_datas,
scipy/sparse/_csparsetools.pyx.in:                  object[:] rows,
scipy/sparse/_csparsetools.pyx.in:                  object[:] data,
scipy/sparse/_csparsetools.pyx.in:                                         object[:] rows,
scipy/sparse/_csparsetools.pyx.in:                                         object[:] data,
scipy/sparse/_csparsetools.pyx.in:                       object[:] new_rows, object[:] new_datas,
scipy/spatial/_ckdtree.pyx:            object[::1] vout

Don't know about sparsetools usage; there's something in KDTree.

Let's see about object arrays in pure python:

$ git grep "dtype=object"
doc/source/release/0.7.0-notes.rst:* string arrays have ``dtype='U...'`` instead of ``dtype=object``
doc/source/release/1.3.0-notes.rst:* `#9645 <https://github.com/scipy/scipy/issues/9645>`__: \`scipy.stats.mode\` crashes with variable length arrays (\`dtype=object\`)
doc/source/tutorial/io.rst:   >>> obj_arr = np.zeros((2,), dtype=object)
doc/source/tutorial/io.rst:   array([1, 'a string'], dtype=object)
scipy/cluster/tests/test_disjoint_set.py:    tokens = np.array(tokens, dtype=object)
scipy/interpolate/_interpolate.py:            r2 = np.empty(prod(self.c.shape[2:]), dtype=object)
scipy/interpolate/tests/test_interpolate.py:            assert_raises(ValueError, p, np.array([[0.1, 0.2], [0.4]], dtype=object))
scipy/io/matlab/_mio.py:       old-style NumPy arrays with dtype=object. Setting this flag to
scipy/io/matlab/_mio5.py:        narr = np.asanyarray(source, dtype=object)
scipy/io/matlab/_mio5_utils.pyx:        result = np.empty(length, dtype=object)
scipy/io/matlab/_mio5_utils.pyx:                return np.empty(tupdims, dtype=object).T
scipy/io/matlab/_mio5_utils.pyx:        result = np.empty(length, dtype=object)
scipy/io/matlab/_miobase.py:   old-style NumPy arrays with dtype=object. Setting this flag to
scipy/io/matlab/tests/test_mio.py:        mlarr([[1,2,3]])), dtype=object).reshape(1,-1)
scipy/io/matlab/tests/test_mio.py:    mlarr(3)), dtype=object).reshape(1,-1)
scipy/io/matlab/tests/test_mio.py:objarr = np.empty((1,1),dtype=object)
scipy/io/matlab/tests/test_mio.py:CN = np.zeros((1,2), dtype=object)
scipy/io/matlab/tests/test_mio.py:CN[0,1] = np.zeros((1,3), dtype=object)
scipy/io/matlab/tests/test_mio.py:CN[0,1][0,2] = np.zeros((1,2), dtype=object)
scipy/io/matlab/tests/test_mio.py:    cell = np.ndarray((1,2),dtype=object)
scipy/io/matlab/tests/test_mio.py:    cells = np.ndarray((1,2),dtype=object)
scipy/io/matlab/tests/test_mio.py:    cells = np.ndarray((1,1),dtype=object)
scipy/io/matlab/tests/test_mio.py:    savemat(stream, {'A':np.array(1, dtype=object)})
scipy/io/matlab/tests/test_mio.py:                                     ['world']], dtype=object))
scipy/io/tests/test_idl.py:                         np.array([b"cheese", b"bacon", b"spam"], dtype=object))
scipy/io/tests/test_idl.py:        assert_identical(s.scalars.e, np.array([b"spam"], dtype=object))
scipy/io/tests/test_idl.py:                               np.array([b"cheese", b"bacon", b"spam"], dtype=object))
scipy/io/tests/test_idl.py:                                            dtype=object))
scipy/io/tests/test_idl.py:                                                    dtype=object))
scipy/linalg/_special_matrices.py:            L_n = np.empty((n, n), dtype=object)
scipy/signal/tests/test_ltisys.py:        tf = (np.array([1, [2, 3]], dtype=object), [1, 6])
scipy/signal/tests/test_ltisys.py:        tf = (np.array([[1, -3], [1, 2, 3]], dtype=object), [1, 6, 5])
scipy/sparse/_lil.py:                self.rows = np.empty((M,), dtype=object)
scipy/sparse/_lil.py:                self.data = np.empty((M,), dtype=object)
scipy/spatial/_ckdtree.pyx:            result = np.empty(retshape, dtype=object)
scipy/spatial/tests/test_distance.py:        data = np.array([[myclass()]], dtype=object)
scipy/spatial/tests/test_distance.py:        data = np.array([[myclass()], [myclass()]], dtype=object)
scipy/spatial/tests/test_kdtree.py:    y = np.empty(shape=(10, ), dtype=object)
scipy/spatial/transform/tests/test_rotation.py:    array = np.array([single, multiple], dtype=object)
scipy/special/tests/test_basic.py:            exp = np.array(_nest_me(nucleus, k=levels), dtype=object)
scipy/special/tests/test_basic.py:        n = asarray([4, 4, 4, 4], dtype=object)
scipy/special/tests/test_basic.py:        k = asarray([1, 2, 3, 4], dtype=object)
scipy/stats/_levy_stable/__init__.py:    [[], [1]] + [v[2] for v in _builtincoeffs.values()], dtype=object
scipy/stats/_stats_py.py:                return np.array(arrays, dtype=object)
scipy/stats/tests/test_contingency.py:                  np.array([[1, 2], ["dd", 4]], dtype=object), 'cramer')
scipy/stats/tests/test_stats.py:            stats.mode(np.arange(3, dtype=object))
scipy/stats/tests/test_stats.py:            stats.lmoment(np.array(self.data, dtype=object))

ISTM we can discount scipy/io; the rest would be nice to audit and remove (potentially with deprecations if needs be).

DEP: signal: remove support for object arrays and longdoubles in sign…

d49c09c

…al.{correlate,convolve,lfilter,sosfilter}

j-bowhay requested a review from ev-br June 22, 2025 20:36

j-bowhay requested review from larsoner and ilayn as code owners June 22, 2025 20:36

j-bowhay removed request for ilayn and larsoner June 22, 2025 20:36

github-actions bot added scipy.signal deprecated Items related to behavior that has been deprecated labels Jun 22, 2025

ev-br approved these changes Jun 22, 2025

View reviewed changes

lucascolley added this to the 1.17.0 milestone Jun 22, 2025

lucascolley changed the title ~~DEP: signal: remove support for object arrays and longdoubles in signal.{correlate,convolve,lfilter,sosfilter}~~ DEP: signal: remove support for object arrays and longdoubles in {correlate,convolve,lfilter,sosfilter} Jun 22, 2025

more clean up

1ee30fa

remove object related code from _sosfilt.pyx

b17bf7d

j-bowhay force-pushed the signal_object_array branch from 139831b to b17bf7d Compare June 23, 2025 06:55

remove unused import

ca95d0d

ev-br reviewed Jun 23, 2025

View reviewed changes

scipy/signal/tests/test_signaltools.py Outdated Show resolved Hide resolved

j-bowhay commented Jun 23, 2025

View reviewed changes

Apply suggestions from code review

6c20257

Co-authored-by: Evgeni Burovski <evgeny.burovskiy@gmail.com>

j-bowhay commented Jun 23, 2025

View reviewed changes

scipy/signal/tests/test_signaltools.py Outdated Show resolved Hide resolved

Update scipy/signal/tests/test_signaltools.py

6ffa925

j-bowhay mentioned this pull request Jun 23, 2025

META: deprecation tracker #15765

Open

32 tasks

j-bowhay added 2 commits June 23, 2025 22:52

add some xp.asarrays

f5f0580

even more cleanup

5ed36db

j-bowhay force-pushed the signal_object_array branch from 8836a50 to 5ed36db Compare June 23, 2025 22:03

tolerance bump

eb85433

suggestions

a3a14de

j-bowhay commented Jun 24, 2025

View reviewed changes

scipy/signal/tests/test_signaltools.py Outdated Show resolved Hide resolved

j-bowhay and others added 4 commits June 24, 2025 16:56

Update scipy/signal/tests/test_signaltools.py

450d256

minor refactor to help debugging

e36762c

fix test

72f32b0

Merge remote-tracking branch 'upstream/main' into signal_object_array

0c62446

lucascolley added the needs-release-note a maintainer should add a release note written by a reviewer/author to the wiki label Jul 5, 2025

lucascolley approved these changes Jul 5, 2025

View reviewed changes

lucascolley merged commit 3b51780 into scipy:main Jul 5, 2025
38 checks passed

Uh oh!

DEP: signal: remove support for object arrays and longdoubles in {correlate,convolve,lfilter,sosfilter} #23212

DEP: signal: remove support for object arrays and longdoubles in {correlate,convolve,lfilter,sosfilter} #23212

Uh oh!

Conversation

j-bowhay commented Jun 22, 2025

Uh oh!

ev-br left a comment

Choose a reason for hiding this comment

Uh oh!

j-bowhay commented Jun 22, 2025

Uh oh!

ev-br commented Jun 22, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ev-br commented Jun 23, 2025

Uh oh!

ev-br commented Jun 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ev-br commented Jun 24, 2025

Uh oh!

Uh oh!

j-bowhay commented Jul 4, 2025

Uh oh!

lucascolley left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ev-br commented Jul 5, 2025

Uh oh!

j-bowhay commented Jul 5, 2025

Uh oh!

ev-br commented Jul 5, 2025

Uh oh!

Uh oh!

DEP: signal: remove support for object arrays and longdoubles in `{correlate,convolve,lfilter,sosfilter}` #23212

DEP: signal: remove support for object arrays and longdoubles in `{correlate,convolve,lfilter,sosfilter}` #23212

ev-br commented Jun 24, 2025 •

edited

Loading