Skip to content

Conversation

mdhaber
Copy link
Contributor

@mdhaber mdhaber commented May 26, 2024

Reference issue

Toward gh-20544
Follow-up to gh-20777

What does this implement/fix?

Similar to gh-20777 but for hypothesis tests with a beta null distribution.

Additional information

I also added cdf to _SimpleChi2 and used it in combine_pvalues.

@mdhaber mdhaber added scipy.stats enhancement A new feature or improvement array types Items related to array API support and input array validation (see gh-18286) labels May 26, 2024
@mdhaber mdhaber requested review from person142 and steppi as code owners May 26, 2024 01:48

def __betaincc(a, b, x):
# not perfect; might want to just rely on SciPy
return betainc(b, a, 1-x)
Copy link
Contributor Author

@mdhaber mdhaber May 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@steppi This is supposed to implement betaincc in terms of betainc for array backends that have betainc but not betaincc. I imagine that when x is close to 1, 1 - betainc(a, b, x) might be better for unusual values of a and b. I thought I'd check with you what to do here: is this good enough, is there something better we can do (possibly involving evaluating both versions and choosing the better one), or should we not attempt to provide this sort of implementation?

Copy link
Contributor

@steppi steppi Jun 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not going to work well or at all for small x, but I see no issues for now since the dispatch to alternative backends is only enabled when the experimental SCIPY_ARRAY_API flag is set. By the time what's now hidden behind SCIPY_ARRAY_API=1 becomes the default behavior, we should have an implementation of betaincc that other backends can use. I don't think the values of a and b for which a beta distribution's pdf has a sharp peak at x=1 are that relevant for these NHSTs, so the time being this should work well enough for your use case on backends that don't support betaincc.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this implementation and the naive 1-betainc be complementary? Like if the result is close to 1, then 1-betainc should be fine, right?

Copy link
Contributor Author

@mdhaber mdhaber Jun 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Never mind I suppose the result is close to zero precisely when x is close to 1, so neither approach would have good precision.
But yeah, I think that by the time this is no longer experimental, all the array libraries will support betaincc.

@@ -81,11 +81,30 @@ def __xlogy(x, y, *, xp=xp):
return __xlogy


def _chdtr(xp, spx):
Copy link
Contributor Author

@mdhaber mdhaber May 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is basically just a copy-paste of _chdtrc, so I thought it would be simple enough to include here. It isn't the main point of this PR, though, so I can take it out if preferred.

Since it's copy-paste, this begs the question of whether we can share code. Rather than trying that now, I think we should do a few more of these to see how it should be factored.

Copy link
Member

@j-bowhay j-bowhay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
array types Items related to array API support and input array validation (see gh-18286) enhancement A new feature or improvement scipy.special scipy.stats
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants