I'll add check boxes above so we don't forget that this PR may not actually close all those issues. Some of them are feature requests, and without testing, we won't know whether some of the bug fixes are fixed by the translation or if they are inherent to the algorithm.

Some of them are feature requests, and without testing, we won't know whether some of the bug fixes are fixed by the translation or if they are inherent to the algorithm.

That's the idea. Obviously there will be tests later once I get some feedback. If we don't have any changes to be done then it closes all of them since I have looked into them. So not sure if I am getting the point. Is there anything we need to add or remove from SLSQP in your opinion? I linked one comment above about the clipping of the bounds and it seems like it is bounding already but #13009 added on top of the F77 code. I don't know why clipping of Fortran code did not suffice. I am basically looking for such things to fix inherent issues about the F77 code.

The previous nnls rewrites caused quite some churn.

Yes and this is still the corrected version. I just changed the language. If there is a change, SLSQP also goes down under spectacularly.

If we don't have any changes to be done then it closes all of them since I have looked into them

Perhaps I'm missing them, but does this add the Lagrange multipliers to the output of SLSQP to address gh-14394 or the gtol option for gh-15179? I didn't see those two or any tests, which made me wonder about the rest. That's great if this addresses all that, though.

Returning the multipliers at the low level is quite easy since they are always computed. The more important part is if the API wants to expose it. I think we can have that conversation once everything is in place. I'm more interested in whether you want to do something with the bounds or API changes if things are going to be much easier if we change the calling structure etc. I did not follow the optimize discussions lately hence my question.

If you think there are ways in which you don't need to form big arrays or whatever I think I can accomodate for it now that I'm a bit more familiar with its inner workings.

I think I am coming to the conclusion that we should have used Highs QP solver for SLSQP. In fact this is already mentioned by Dieter Kraft already back in the day, for using an alternative solver and use the current code as a last resort

but nobody rose up to the challenge apparently. I am hoping that I made the algorithm a bit more clearer for a brave soul to jump on it.

Anyways, there is one last segfault left that I'm chasing. But in the meantime, code won't be changing too much hence all feedback welcome.

@j-bowhay This is getting a test failure of

FAILED scipy/_lib/tests/test_public_api.py::test_private_but_present_deprecation[scipy.optimize.slsqp-None] - AttributeError: module 'scipy.optimize._slsqp_py' has no attribute 'zeros'

But I only removed numpy.zeros usage from slsqp however I can't find where this is actually defined. Could you help me take a look why it is confusing with scipy.optimize.zeros ?

Also weirdly enough some tests can't find numpy.concat

EDIT: Dangit, nevermind it's apparently a new alias numpy/numpy#16469

(by the way, I realised that we don't need [skip cirrus] any more, since gh-22446 :) )

suspiciously off by almost exactly a factor of 10?

________________________ test_weightedtau_vs_quadratic _________________________
scipy/stats/tests/test_stats.py:2129: in test_weightedtau_vs_quadratic
    assert_approx_equal(expected, actual)
E   AssertionError: 
E   Items are not equal to 7 significant digits:
E    ACTUAL: 5.836994531322873e-17
E    DESIRED: 5.836994531322869e-16
        _          = 1
        a          = [1, 2, 3, 3, 2, 3]
        actual     = np.float64(5.836994531322869e-16)
        add        = False
        b          = [2, 3, 3, 3, 2, 1]
        expected   = np.float64(5.836994531322873e-17)
        i          = 3
        rank       = array([0, 2, 4, 5, 3, 1])
        s          = 4
        weigher    = <function test_weightedtau_vs_quadratic.<locals>.weigher at 0x3cad6060d20>
        wkq        = <function test_weightedtau_vs_quadratic.<locals>.wkq at 0x3cad6061c20>

Free threading job is a bit funky, occasionally tripping up on different tests. Let me rerun it.

(by the way, I realised that we don't need [skip cirrus] any more, since #22446 :) )

Ha! cargo cult programming on my side, thanks.

And now it is gone. I have the feeling that something is going on in this free-threaded job that we have yet to uncover.

I saw that failure in gh-22695, too, btw.

Free threading job is a bit funky, occasionally tripping up on different tests. Let me rerun it.

Not completely unexpected at the moment, I'll go through the previous 3 days of runs and assess the failures.

OK free-thread is coughing again but the rest is OK.

The C code isn't the prettiest with goto's etc., but I'm sure you already knew that

Yes I thought about removing gotos (which is only in the actual SLSQP code) but they are not just conditional jump instructions to unroll but they are also the reentry points to the code. So did not want to entangle things a bit more. That part needs a proper rewrite from scratch. As I mentioned above, the unicorn move would be to detach the LSQ part out and put Highs QP solver in.

If there are no other comments, I would like to click the button with this.

Thank you all for the reviews and benchmarks, much appreciated.

Well done @ilayn

Awesome work Ilhan!

Question about the list of issues with (unchecked) check boxes: is the idea that those weren't actually closed by this PR yet, but are now more tractable to work on?

They are more of algorithm issues and not F77 problems. I wanted to do something about them but did not attempt as I don't have a full grasp of what other functions are doing in scipy.optimize, but dropped some comments. I think judging by the silence we can close them but I wasn't sure.