looks like just one real CI failure from a brief look:

____________________________ TestBounds.test_basic _____________________________
[gw1] linux -- Python 3.10.12 /usr/bin/python3-dbg

self = <scipy.optimize.tests.test_cobyla.TestBounds object at 0x7f006a4c9f90>

    def test_basic(self):
        def f(x):
            return np.sum(x**2)
    
        lb = [-1, None, 1, None, -0.5]
        ub = [-0.5, -0.5, None, None, -0.5]
        bounds = [(a, b) for a, b in zip(lb, ub)]
        # these are converted to Bounds internally
    
        res = minimize(f, x0=[1, 2, 3, 4, 5], method='cobyla', bounds=bounds)
        ref = [-0.5, -0.5, 1, 0, -0.5]
        assert res.success
>       assert_allclose(res.x, ref, atol=1e-3)
E       AssertionError: 
E       Not equal to tolerance rtol=1e-07, atol=0.001
E       
E       Mismatched elements: 2 / 5 (40%)
E       Max absolute difference among violations: 1.85025255
E       Max relative difference among violations: 0.25694824
E        ACTUAL: array([-0.5     , -0.5     ,  1.256948,  1.850253, -0.5     ])
E        DESIRED: array([-0.5, -0.5,  1. ,  0. , -0.5])

bounds     = [(-1, -0.5), (None, -0.5), (1, None), (None, None), (-0.5, -0.5)]
f          = <function TestBounds.test_basic.<locals>.f at 0x7f0057df9440>
lb         = [-1, None, 1, None, -0.5]
ref        = [-0.5, -0.5, 1, 0, -0.5]
res        =  message: Return from COBYLA because the trust region radius reaches its lower bound.
 success: True
  status: 0
     fun: 5.753353369726385
       x: [-5.000e-01 -5.000e-01  1.257e+00  1.850e+00 -5.000e-01]
    nfev: 38
   maxcv: 0.0
self       = <scipy.optimize.tests.test_cobyla.TestBounds object at 0x7f006a4c9f90>
ub         = [-0.5, -0.5, None, None, -0.5]

It appears that the issue with the one you pointed out is that one of the bounds is fixed, and this leads COBYLA to go a little haywire when trying to satisfy it. I'll add some code to detect fixed bounds and eliminate them, which is both a sensible thing to do and also what COBYQA does.

And there's also test_basinhopping. It takes longer now than the 20s allotted for it, taking up to 33.8 seconds on Windows fail slow full. Obviously since the algorithm is now in Python it's not surprising that it's slower, so I've increased the window for that test to 35s.

Besides those two there are some mypy failures which I need to look into.

Fixed bounds and test_basinhopping are fixed, last up is the mypy failures.

please just apply this with your next round of changes:

diff --git a/mypy.ini b/mypy.ini
index 95deab150a..3c5dfd9e86 100644
--- a/mypy.ini
+++ b/mypy.ini
@@ -751,3 +751,6 @@ ignore_errors = True
 
 [mypy-scipy._lib.array_api_compat.*]
 ignore_errors = True
+
+[mypy-scipy._lib.pyprima.*]
+ignore_errors = True

I ran the DFO (Derivative Free Optimization) benchmarks to compare against the existing COBYLA implementation and here are the results

43 problems are improved either in final value or in number of function evaluations. 22 are worse in those metrics. A notable outlier is test #27. I did some digging and found this this is the objective function it is trying to evaluate

def func(x):
    m = 20
    fvec = np.zeros(m)
    for i in range(m):
            temp = (i + 1) / 5
            tmp1 = x[0] + temp * x[1] - np.exp(temp)
            tmp2 = x[2] + np.sin(temp) * x[3] - np.cos(temp)
            fvec[i] = tmp1 * tmp1 + tmp2 * tmp2
    y = np.sum(fvec ** 2)
    if np.isnan(y):
        return np.inf
    return y
x0 = [250.  50. -50. -10.]

If I remove the maxfun limitation on the new implementation it eventually gets to the optimum value, although it takes ~13000 function evaluations on my machine as compared to <1000 for the old implementation. Problem 17 is also somewhat of an outlier. If I tighten the tolerance on it to 1e-6 from the default 1e-4 the result becomes comparable to the old implementation. Different problems will fare better or worse depending on the initial/final trust region radius (which in scipy are called rhobeg and tol respectively) and these parameters are exposed to the user so I don't think this outlier is something to be overly concerned with.

For me the benchmarks look like a net win. Let's include one sentence about the parameters that can be tweaked in the release notes to give users a hint.

Not to block, or in fact take away, anything from this great addition, I just want to understand, is this going to live in your fork and not in prima repo? I don't have any background info hence my question.

Not to block, or in fact take away, anything from this great addition, I just want to understand, is this going to live in your fork and not in prima repo? I don't have any background info hence my question.

At the moment yes. It's my understanding that @zaikunzhang is occupied and that PRIMA is not particularly high on his list of priorities. As such I haven't had a chance to discuss this with him and see if he has any comments, but given my previous work with him, I think he would take it as a good sign that the Python implementation is bit-for-bit identical* to the Fortran version on all problems in the test set that we used for the Python bindings. Hopefully he is able to get bandwidth sometime in the future and we'll be able to merge the Python implementation back into the prima repo. I've been rebasing my work on the latest stuff from him so at the moment the fork is up to date with the latest. All that said, I don't think we should wait for him to move forward on this.

* This is true when the USE_NAIVE_MATH flag in Python is on, and Fortran is compiled in debug mode. I've chosen to default that flag to off so that numpy routines are used for martix multiplication and so forth instead of the naive implementations. My reasoning is that most practical problems do not need to preserve all floating point calculations, but when the flag is on it sometimes results in more function evaluations and therefore more matrix math, and so the speedup/slowdown is not as significant as you might think. And also Fortran behaves differently when compiled with any optimization so there's a parallel there. I'm open to changing this.

Test failures are unrelated, this should be mergeable now.

We will want to transfer lucascolley/pyprima to the SciPy org before merge.

We will want to transfer lucascolley/pyprima to the SciPy org before merge.

Argh, missed that, sry. Maintainer should have rights to create repos under scipy org, at least the button works for me. Is there an approval process to follow?

Maintainer should have rights to create repos under scipy org

thanks Daniel, done!

Thanks @lucascolley . If there are no more comments, I will merge this next week.

@lucascolley @nbelakovski : the new dependency checker tach is complaining now. Would the suggested fix by tach suffice?

I would suggest just adding

[[modules]]
path = "scipy._lib.pyprima"
unchecked = true

Like we have for cobyqa

Agreed, and done.

Alright, I pulled the trigger here finally.

Thanks for your tremendous work here @nbelakovski ! Hat tip to all reviewers as well.

We should see if we can close #15527 and #13952

And of course thanks for your great work @nbelakovski

Thanks all, and thanks for the help in getting it across the finish line!

(sorry for reposting)

Hello! Author of PRIMA here.

Thanks for working on this. I am sorry for not being able to respond.

For anything derived from PRIMA, may I request that the following is stated clearly?

1. PRIMA is developed by Zaikun Zhang, with "Zaikun Zhang" linked to my homepage zhangzk.net
2. PRIMA means "Reference Implementation for Powell's Methods with Modernization and Amelioration", which provides a modernized and improved implementation of M.J.D. Powell's derivative-free optimization methods, with "M.J.D. Powell" linked to zhangzk.net/powell.html

As an example, see lfortran.org/blog/2025/03/lfortran-compiles-prima

Note that PRIMA contains bug fixes and improvements that do not exist in Powell's original implementation of the solvers. Results produced by PRIMA are surely different from those produced by the original solvers by Powell. Therefore, it is simply wrong to pretend that PRIMA is just Powell's solvers.

It took me years to develop PRIMA, which almost cost my career and my life. (This partially explains why I have been unable to respond recently. I have to strive to survive as a young professor).

As a professor, all I need is a citation / acknowledgment when my work is useful, which is not well recognized in academia when it comes to software development. In addition, for obvious reasons, people tend to cite Powell without mentioning me when they use PRIMA instead of Powell's original solvers. (I have no doubt that my contribution is insignificant compared with Powell's, and I acknowledge this whenever I introduce PRIMA to others. See libprima.net for example). This is crucial for my career.

Thank you very much.

@dschmitz89 and @lucascolley thank you for finishing the review/merge here! Sorry for going missing in action for a while.

@nbelakovski awesome work, thank you 🙏🏼

As a professor, all I need is a citation / acknowledgment when my work is useful, which is not well recognized in academia when it comes to software development.

We definitely recognize that problem and the need for citations. I know you saw it, but just for completeness: gh-22738 added references to your paper in two places.