Skip to content

Conversation

mdhaber
Copy link
Contributor

@mdhaber mdhaber commented May 26, 2024

Reference issue

Closes gh-7242
Closes gh-18445
Toward gh-6379
Toward gh-18867

What does this implement/fix?

There have been requests for scalar rootfinders and minimizers to have a callback interface and be vectorized like newton. It would also be nice for such rootfinders and minimizers to have a consistent interface, accept both absolute and relative tolerances on the abscissa and function value, provide detailed output information, and be array-API compatible. This PR attempts to address these needs by providing a public interface to the following private functions:

Rationale for adding new functions rather than extending root_scalar and root_minimize can be found in #7242 (comment) and #20624 (comment), respectively. These functions are put in a new elementwise namespace rather than the base optimize namespace to keep the function names manageable without conflicting with existing function names. The APIs for rootfind and minimize have been designed with the option to add a method argument in mind; they will be equally appropiate for other common bracketing and derivative-based rootfinding and minimization methods.

Additional information

Currently, only elementwise.rootfind has array API support (see gh-20689); the rest will be converted over the summer.

Questions:

  1. The private functions are extensively tested in test_chandrupatla.py and test_bracket.py. In this PR, I don't want to convert the tests completely to use the new interfaces; this can be done when new methods are added and the tests are parameterized over all methods. For this PR, how about we leave the tests untouched but a) rather than importing the private functions in the test file, we import only the public functions and b) include in the test file a wrapper that exposes the public function with the interface of the private function. (Yes, effectively, the arguments and returns would get converted back and forth.) This would keep the diff tiny so that this PR can focus on the API rather than the nitty gritty. Update: Done.
  2. I've been careful to use "root" to refer to the abscissa at which the function value is "zero", and sometimes I am careful to make a similar distinction between "minimizer" and "minimum". From this perspective, the function name bracket_minimum might not be consistent with bracket_root; perhaps bracket_minimum should be bracket_minimizer or bracket_root should be bracket_zero. This is worth discussing. Update: "minimum" is acceptable. "minimizer" could be confused with an algorithm or function for performing minimization.
  3. Should we mark arrays as read-only before passing them into user callables? I'd been thinking about it already, and it came up recently in BUG: integrate: Make arrays passed to odeint user functions read-only. #20803. They might already be copies, in which case it wouldn't matter, but it might be good anyway. Update: I don't think there is an array API standard way to do this, so I've documented that callables must not mutate the arrays.

To do:

  • More extensive examples
  • Note conditions under which functions will converge successfully
  • Add tests according to the resolution of question 1

@mdhaber mdhaber added enhancement A new feature or improvement scipy.optimize array types Items related to array API support and input array validation (see gh-18286) labels May 26, 2024
@mdhaber mdhaber requested review from steppi and tupui May 26, 2024 20:46
@mdhaber mdhaber requested a review from andyfaff as a code owner May 26, 2024 20:46
@github-actions github-actions bot added the Meson Items related to the introduction of Meson as the new build system for SciPy label May 26, 2024
@tupui
Copy link
Member

tupui commented May 27, 2024

For the naming I think I would do

elementwise.root
elementwise.bracket_root
elementwise.minimum
elementwise.bracket_minimum

Or use verbs for the above to have elementwise.minimize, elementwise.bracket_minimizer

@mdhaber
Copy link
Contributor Author

mdhaber commented May 27, 2024

It sounds like you are suggesting to change from verbs to nouns, but then at the end you write that verbs are OK? They are all intended to be verbs right now. "bracket" is a verb already, and whether it's "bracket_minimum" or "bracket_minimizer" is a separate question noted in the top post. The one that is questionable as a verb is "rootfind". "root-finding" is a present participle verb that is often used as an adjective, suggesting that "root-find" could be used as a verb. It's not common to do so in English, but there are a fair number of functions named rootfind that have a similar purpose. To better address the comment, what is the crux of the suggestions?

@tupui
Copy link
Member

tupui commented May 27, 2024

I meant that I would either use nouns or verbs. Not mix.

Also this could work

elementwise.root_finder
elementwise.bracket_root_finder
elementwise.minimizer
elementwise.bracket_minimizer

@mdhaber
Copy link
Contributor Author

mdhaber commented May 27, 2024

I meant that I would either use nouns or verbs. Not mix.

Right. I interpret the existing names as verbs. (To use verb or noun for a function is an age-old debate, and we are inconsistent in optimize (verb); e.g., we have minimize and bracket (verbs), but also root and linprog (noun/nounish).)

"minimize" is a verb. "bracket" is a verb or a noun - it seems you (@tupui) can interpret it as a verb, too, given the suggestion of bracket_minimumizer as a verb above. So rootfind seems to be the one in question. It was intended to be a verb in the sense that "root" is just a noun that modifies the verb "find", but if group prefers for the name to start with find, OK.

@tupui
Copy link
Member

tupui commented May 27, 2024

bracket_minimizer yeah this one I just confused myself and did not think about bracket as a verb even.

For root and find, I think that it would be better to harmonise it with minimize as I proposed above (modulo the grammar). So if we have a bracket_minimum and minimum then seems normal to me to have bracket_root with root

@mdhaber
Copy link
Contributor Author

mdhaber commented May 27, 2024

I interpret that as mixing verbs and nouns, though. There, minimum and root are nouns, but bracket_root and bracket_minimum seem to be verbs because of the word order, e.g. "to bracket a root" -> bracket_root. minimum_bracket and root_bracket would be much easier to interpret as nouns.

@tupui
Copy link
Member

tupui commented May 27, 2024

Yeah I mean, braketer_root maybe

@mdhaber
Copy link
Contributor Author

mdhaber commented May 27, 2024

This is getting quite long, so this will be my last post about the names until we hear from others.

Assuming by braketer_root you mean bracketer_root, this would be a noun. But I think it goes against the principles of why a function might be named with a noun instead of a verb - to identify what the function returns. The function itself could be said to be a "root bracketer", but it returns a "bracket", not a "bracketer".

As a compromise, I can live with find as the verb for both root and minimum/minimizer.

  • find_root
  • find_minimum (or find_minimizer)
  • bracket_root
  • bracket_minimum (or bracket_minimizer)

Making these all look consistent as nouns may be tricky. root, minimum/minimizer, and root_bracket are fine, but I am tempted to write "minimization bracket" before "minimum/minimizer bracket".

@steppi
Copy link
Contributor

steppi commented May 27, 2024

As a compromise, I can live with find as the verb for both root and minimum/minimizer.

I prefer minimize, my preferred order is

$$\mathrm{minimize} > \mathrm{find\_minimum} > \mathrm{find\_minimizer}$$

It just so happens that there isn't a good verb to describe finding roots of a function, but I don't think that means we should abandon a good verb for finding the minimum/minimizer. I like using minimum over minimizer due to an ambiguity between minimizer as the x value where a minimum is attained, and an algorithm which finds this x value. Similarly, for the related bracketing function, I like bracket_minimum. I think bracket_minimizer might even be interpreted as a bracketing minimization algorithm by some people.

I like bracket_root for the function which finds an initial bracket for the root. I think I prefer find_root over rootfind, but don't have a strong opinion on this one.

@mdhaber
Copy link
Contributor Author

mdhaber commented May 27, 2024

OK, so with minimizer out of the way, we're looking at:

  • find_root or rootfind
  • find_minimum or minimize
  • bracket_root
  • bracket_minimum

I think whichever way we go, I would prefer to pair rootfind/minimize (exactly what we have now) or find_root/find_minimum.

@tupui
Copy link
Member

tupui commented May 27, 2024

My vote is on

find_root
find_minimum
bracket_root
bracket_minimum

Symmetric for users

@steppi
Copy link
Contributor

steppi commented May 27, 2024

I'd accept either find_root/find_minimum or rootfind/minimize.

@mdhaber
Copy link
Contributor Author

mdhaber commented May 27, 2024

Great. How about questions 1 and 3?

@tupui
Copy link
Member

tupui commented May 27, 2024

Yes to both for me.

I prefer to keep this focused and for the arrays I personally preferred when my data is not modified by the function using it.

@steppi
Copy link
Contributor

steppi commented May 27, 2024

Great. How about questions 1 and 3?

For question 1) +1 for keeping the diff small by writing a wrapper like you described.
For question 2) I think marking arrays read-only is the principled way to do it, so I'm +1 for that too. I assume requiring user supplied functions written in Cython to have arguments marked const shouldn't be an issue here right, since there's no backwards compatibility to worry about?

@mdhaber
Copy link
Contributor Author

mdhaber commented May 27, 2024

I assume requiring user supplied functions written in Cython to have arguments marked const shouldn't be an issue here right, since there's no backwards compatibility to worry about?

Right.

I prefer to keep this focused

I assume you mean that this should be a separate PR. Probably - I think it can/should be changed in the EIM infrastructure anyway, not this code.

I'll add the tests.

@mdhaber
Copy link
Contributor Author

mdhaber commented May 28, 2024

I'm not sure why the CircleCI doc build job is not showing up above, but the link is here. As you can see in the rendered docs, I'm not getting the expected TOC from elementwise.py and Sphinx is reporting "WARNING: toctree contains reference to nonexisting document 'reference/optimize.elementwise'". I'm trying to find the examples of scipy.stats.sampling, qmc, contingency, and I think what I have is analogous - do you know how to fix this? Fixed, we think - thanks @steppi!

@mdhaber
Copy link
Contributor Author

mdhaber commented May 28, 2024

6565ce2 demonstrates what I mean for the tests, item 1. LMK if that looks good and I'll repeat that for the other functions.

@steppi
Copy link
Contributor

steppi commented May 28, 2024

6565ce2 demonstrates what I mean for the tests, item 1. LMK if that looks good and I'll repeat that for the other functions.

That looks good to me.

@mdhaber
Copy link
Contributor Author

mdhaber commented May 28, 2024

I'll try to fix the failure after the doc build finishes.

________________________ test_all_modules_are_expected _________________________
scipy/_lib/tests/test_public_api.py:255: in test_all_modules_are_expected
    raise AssertionError(f'Found unexpected modules: {modnames}')
E   AssertionError: Found unexpected modules: ['scipy.optimize.elementwise']
        _          = False
        ignore_errors = <function test_all_modules_are_expected.<locals>.ignore_errors at 0x144ca0b80>
        modname    = 'scipy.version'
        modnames   = ['scipy.optimize.elementwise']

@lucascolley lucascolley removed the Meson Items related to the introduction of Meson as the new build system for SciPy label May 29, 2024
@mdhaber mdhaber changed the title ENH: optimize.elementwise: vectorized scalar optimization and rootfinding tools ENH: optimize.elementwise: vectorized scalar optimization and root finding tools May 29, 2024
[skip ci]

Co-authored-by: Matt Haberland <mhaberla@calpoly.edu>
Copy link
Member

@tupui tupui left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Matt. This has gone through the forum and LGTM 🚀

@tupui tupui merged commit 68bdfd3 into scipy:main Jul 3, 2024
@tupui tupui added this to the 1.15.0 milestone Jul 3, 2024
@lucascolley lucascolley added the needs-release-note a maintainer should add a release note written by a reviewer/author to the wiki label Jul 3, 2024
@lucascolley lucascolley removed the needs-release-note a maintainer should add a release note written by a reviewer/author to the wiki label Jun 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
array types Items related to array API support and input array validation (see gh-18286) enhancement A new feature or improvement scipy.optimize
Projects
None yet
4 participants