Skip to content

Conversation

ilayn
Copy link
Member

@ilayn ilayn commented May 18, 2023

Contuniation of #18391


As discussed in #18367 for linalg.interpolative, iterative solvers are also in the same state that it is almost impossible to implement any enhancements such as returning the iteration information and so on. This PR, reimplements all code in pure Python and with a bit of care to inplace operations thanks to NumPy no performance is lost.

@mdhaber @tupui Touched optimize tests for handling DperecationWarnings in these solvers and also did some PEP8 cleanup in separate commits. So should be easier to review those parts. When this goes in, I can go into nonlinear optimize code to find where these are triggered.

@h-vetinari The fixtures were generating ~1000 tests and skipping 500ish of them causing lots of s, so modified the fixtures such that tests are skipped and collect-time. Please have a look.

When ready, I'll remove the Fortran files and the compilation machinery as the last step.

Reference issue

Closes #5022
Closes #6198
Closes #12748
Closes #15533
Closes #15693
Closes #17744

These ones are auxiliary signature issues, that will need addressing once this PR goes in

Related #12624
Related #15657
Related #15738
Related #16123

Memory Lane for past discussions

#1466
#7623
#7624
#8400
#10341
#16050

What does this implement/fix?

  • Rewrote all code in Python instead of using very old Fortran templates
  • The performance is comparable and slightly faster with Python
  • Removes all dependencies to _isolve/iterative Fortran code.
  • The tol/atol handling is just a mess and is waiting to be overhauled for at least 5 years. This PR will unite both issues for iterative solvers at least
  • Most tests are drop-in replacement passed. Several segfaults are avoided in the meantime that was annoying the CI for a very long time sporadically.
  • Not my favorite coding style but I went the Fortran files line-by-line and tried to be as faithful as possible. GMRES might have some tolerance changes according to my tests.

image

@ilayn ilayn added enhancement A new feature or improvement scipy.sparse.linalg Documentation Issues related to the SciPy documentation. Also check https://github.com/scipy/scipy.org maintenance Items related to regular maintenance tasks labels May 18, 2023
@ilayn ilayn added this to the 1.11.0 milestone May 18, 2023
Copy link
Member

@h-vetinari h-vetinari left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I only really looked at the test changes; I like them a lot! Also the commits are nicely separated1, good work! :)

PS. Thanks for moving the atol/rtol/tol-machinery to _get_atol 👍

Footnotes

  1. the diff of the first one is hard to read, but I guess that's in the nature of such a change.

@ilayn
Copy link
Member Author

ilayn commented May 20, 2023

I bumped the milestone on this one because there are other solvers that needs attention about deprecations. We can wait another 6 months on top of 15 years I think :)

@ilayn
Copy link
Member Author

ilayn commented May 29, 2023

I spoke to Pauli offline and went over about his preconditioning tolerance machinery. There is no problem as far as we can see. Also a tolerance initialization bug was causing some relaxation in the solution which is now fixed.

@ilayn
Copy link
Member Author

ilayn commented May 30, 2023

If everyone is happy, I'd like to add the Fortran code deletion commit on top and merge soon. Please let me know if you spot any issues in the meantime.

Copy link
Member

@j-bowhay j-bowhay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor tweak needed due to bumping the milestone

Copy link
Member

@j-bowhay j-bowhay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor tweak needed due to bumping the milestone

@ilayn ilayn requested review from rgommers and larsoner as code owners May 30, 2023 20:08
Copy link
Member

@h-vetinari h-vetinari left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cannot re-review in depth, but the test changes look benign, which means that the implementation should be doing what it's supposed to. I trust you on this. 👍

Copy link
Member

@rgommers rgommers left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great @ilayn! I had a look now, and can review in more detail and get this merged - just can't promise it happens this week, because I have a trip starting tomorrow.

One issue I just noticed with the tol deprecation (which came up in an unrelated PR earlier today): it won't trigger if a user writes something like: bicg(A, b, x0, None, 1000, M). To address that, the signature can be changed to use tol=_NoValue with _NoValue being a constant defined in the file. And then also emit the deprecation warning if the user passes None.

The performance is comparable and slightly faster with Python

I didn't see any benchmark results here or on the initial PR - may be worth updating the PR description with whatever results you have?

@ilayn
Copy link
Member Author

ilayn commented May 31, 2023

bicg(A, b, x0, None, 1000, M)

It's a bit better than that. if somebody was not using tol before, they don't get any warning because now tol=None and rtol is the new tol. Hence no disruption. When they want to fiddle with its value, they will get the warning. For the actual users that use a value for tol also get a warning and we move them to rtol. So I think we got covered this part.

may be worth updating the PR description with whatever results you have?

Yes, I should. I'll generate something .

@rgommers
Copy link
Member

It's a bit better than that. if somebody was not using tol before, they don't get any warning because now tol=None and rtol is the new tol. Hence no disruption.

That is a good idea in principle, however my point was that you can then not remove the keyword. Use of a positional None should be noisy first before it can be ripped out.

@ilayn
Copy link
Member Author

ilayn commented May 31, 2023

Here are some very funky results :)

image

I have no idea what is going on with cgs or gmres (possibly I'm measuring failure speed, I'll run those again) but in general we are in good shape The difference looks big but the scale is zoomed hence it's not that big.

The expensive parts of these solvers are the matvecs and preconditioner ops but they were already being done on the python side anyways hence the performance is pretty much the same if we don't stray away from inplace NumPy ops.

I'll check on GMRES one more time to see if I forgot some inplace linalg function but I'd take this PR against that sphagetti even if we lost 20% to be honest.

@ilayn
Copy link
Member Author

ilayn commented May 31, 2023

That is a good idea in principle, however my point was that you can then not remove the keyword. Use of a positional None should be noisy first before it can be ripped out.

Ah you mean you don't want to yank out a keyword from the middle to disturb the positional arguments if I understand correctly. But I think that kind of breakage is justified in this situation. Anyways let's continue when you are back 😃 I'll also need to reset myself from all this FORTRAN radiation I've been exposed lately. .

@ilayn
Copy link
Member Author

ilayn commented Jun 8, 2023

Just as a reminder, how should we proceed with the tol keyword here? Don't know a separate PR might be a better strategy to deal with it but I guess @rgommers saying is a better way to go once and for all.

@h-vetinari
Copy link
Member

Just as a reminder, how should we proceed with the tol keyword here?

The deprecation warning from #8400 (in gcrotmk & lgmres) has been warning for 5 years not to use tol= and recommends to use atol=.

In fact, it's warning as soon as atol=None, and since that argument's at the very end of the signature, someone would have to fill all arguments positionally and set atol to something non-None to both: not see that warning and be broken by removal of tol.

I think this is very unlikely usage, and considering the length of time this has been warning, it'd be defensible IMO to just remove tol.

Haven't looked how many other functions are affected by this though, and I can also see the argument that it'd be cleaner to deprecate positional usage completely (and then switch to keyword-only arguments).

@ilayn
Copy link
Member Author

ilayn commented Jun 17, 2023

@ @h-vetinari @rgommers @j-bowhay Should we merge this and deal with the _NoValue() right after this? I am a bit worried that it's too crowded in this PR to find all the occurences of the tol=None usage. I can submit something very quickly for that separately.

@ilayn ilayn force-pushed the iterative_no_fortran branch from 1fda004 to 99a57bc Compare June 17, 2023 10:58
@h-vetinari
Copy link
Member

Fine by me1

Footnotes

  1. not that I wouldn't prefer it to be available in a separate commit or PR before merging, but I don't want to impede your progress on this, and I trust you'll follow up.

@ilayn
Copy link
Member Author

ilayn commented Jun 18, 2023

OK, I'll open the PR in a bit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Documentation Issues related to the SciPy documentation. Also check https://github.com/scipy/scipy.org enhancement A new feature or improvement maintenance Items related to regular maintenance tasks scipy.sparse.linalg
Projects
None yet
4 participants