Skip to content

Conversation

mdhaber
Copy link
Contributor

@mdhaber mdhaber commented Jul 21, 2019

See #10124
First commit will probably break at least one CI test.
A subsequent commit will patch.

@mdhaber mdhaber added the backport-candidate This fix should be ported by a maintainer to previous SciPy versions. label Jul 21, 2019
@mdhaber mdhaber added this to the 1.3.1 milestone Jul 21, 2019
@mdhaber mdhaber changed the title TST:optimize: fixed mistake in linprog test_bug_10124 TST: optimize: fixed linprog disp=True bug Jul 21, 2019
@mdhaber mdhaber changed the title TST: optimize: fixed linprog disp=True bug TST: optimize: fixed linprog "disp": True bug Jul 21, 2019
@tylerjereddy
Copy link
Contributor

For backporting purposes, it would also be helpful to clearly identify the commits responsible for fixing doc builds more generally on master as noted here: #10286 (comment)

I assume those fixes are orthogonal to this one, since master is already passing doc builds.

@mdhaber
Copy link
Contributor Author

mdhaber commented Jul 22, 2019

@tylerjereddy I'm not sure that I understand. This PR has no commits for fixing doc builds.

@mdhaber
Copy link
Contributor Author

mdhaber commented Jul 22, 2019

Unfortunately 9eed350 did not break test_bug_10124, and I'm unable to find another test in this branch for which setting option "disp":True causes it fail.

However, I first noticed the mistake in test_bug_10124 while working on gh-10495, and when I fix the test in that branch, it started failing on CI. I'll use that branch to confirm that the patch is good, then submit the patch here.

@mdhaber
Copy link
Contributor Author

mdhaber commented Jul 22, 2019

Patch worked there. After CI is green, let's merge this for 1.3.1.

@tylerjereddy
Copy link
Contributor

This PR has no commits for fixing doc builds.

The cognate issue is labeled "documentation." I'll want doc builds passing on the maintenance branch before this gets backported

@mdhaber
Copy link
Contributor Author

mdhaber commented Jul 23, 2019

Ah. I see. It is not really a documentation issue; the problem just happened to be found in the course of running the example problem in the documentation.

@mdhaber
Copy link
Contributor Author

mdhaber commented Jul 23, 2019

Just wanted to summarize what's been going on here so the interested reader (future self) doesn't have to piece it together from several issues and PRs.

The issue reported in gh-10124 was that linprog would fail on some platforms with the disp option set True. Specifically:

fmt = '{0:<20.13}{1:<20.13}{2:<20.13}{3:<17.13}{4:<20.13}{5:<20.13}'
print(fmt.format(rho_p, rho_d, rho_g, alpha, rho_mu, obj))

fails if one of the arguments for fmt.format (e.g. alpha) is not a float. While most of the arguments are almost always a float, it's possible for at least some of them to be an int. Apparently an int isn't compatible with these format specifiers, so the original fix in gh-10135 was to make all the arguments floats before passing them into fmt.format.

    fmt = '{0:<20.13}{1:<20.13}{2:<20.13}{3:<17.13}{4:<20.13}{5:<20.13}'
    print(fmt.format(
        float(rho_p),
        float(rho_d),
        float(rho_g),
        float(alpha) if isinstance(alpha, numbers.Number) else alpha,
        float(rho_mu),
        float(obj)))

The reason for the ternary on alpha was that it is always a str ('-') on the zeroth iteration of the algorithm, in which case it should be left alone. This fixed the problem on my machine, and the new test designed to check this issue passed on CI.

In the course of working on gh-10495 I discovered a mistake in the test. I fixed the mistake in the test in commit 9eed350 of this PR, yet tests still passed. However, there was definitely still a bug lurking, because applying the same fix to the test in the branch I was using for gh-10495 (1047bbb) caused the tests to fail: sometimes alpha was an array with a single element. I didn't figure out exactly why (I should dig into it some point - added to list of maintenance items in gh-9671), but I want to get a fix in 1.3.1, so my solution here is:

alpha if isinstance(alpha, str) else float(alpha)

(as I believe @Kai-Striega recommended originally). When alpha is '-' in the first iteration, it's left alone. Regardless of what other type alpha has been observed to be (float, int, or single-element array), it's turned into a float before being formatted.

@tylerjereddy
Copy link
Contributor

@Kai-Striega Does this look ok to you? It would be clearer if this was fail before/ succeed after as far as the fix goes, rather than having to use another branch for verification.

Is the inconvenience of having to specify a float instead of an int enough to warrant backporting? Is that a regression over previous behavior or something that can wait until 1.4?

@mdhaber
Copy link
Contributor Author

mdhaber commented Jul 31, 2019

I suppose it's not a huge deal. Maybe we don't have to backport unless we get a bug report.

@Kai-Striega
Copy link
Member

@tylerjereddy changes look good to me, except for a small question is: numbers used anywhere else? If not can we avoid importing it.

The test's error comes from changing the options to be different from those used in the tests. I've done a quick search and couldn't find any other instances where this occurs. @mdhaber would you be able to give it a search and see if it occurs anywhere else?

@mdhaber
Copy link
Contributor Author

mdhaber commented Aug 2, 2019

@Kai-Striega oops numbers isn't needed anymore. I'll get rid of it.

I've only seen the error happen in the autoscale branch. I don't think it's existing options that cause it; it's because autoscale changes the problem formulation. That's not merged yet, though.

Since this issue isn't as urgent as I initially thought, we can hold this one off until autoscale is merged. That way we can see the bug fixed in unit tests here in this PR.

@pv pv merged commit a104095 into scipy:master Aug 4, 2019
@pv
Copy link
Member

pv commented Aug 4, 2019

Thanks --- let's just merge this quick, as it appears to be a straightforward fix.
If it's not urgent fix, then the backport-candidate label (and milestone) status can be updated.

@tylerjereddy tylerjereddy removed the backport-candidate This fix should be ported by a maintainer to previous SciPy versions. label Aug 4, 2019
@tylerjereddy tylerjereddy modified the milestones: 1.3.1, 1.4.0 Aug 4, 2019
@mdhaber mdhaber added the backport-candidate This fix should be ported by a maintainer to previous SciPy versions. label Oct 11, 2019
@mdhaber
Copy link
Contributor Author

mdhaber commented Oct 11, 2019

@tylerjereddy I re-added the backport candidate label because this fix was requested for 1.2.X in #10915.

@tylerjereddy
Copy link
Contributor

Thanks Matt, hopefully I can start clawing back on regular SciPy duties with the SciPy paper resubmitted :)

@tylerjereddy
Copy link
Contributor

I'll aim to include this in 1.3.2 as well unless there's a reason not to.

@mdhaber
Copy link
Contributor Author

mdhaber commented Oct 22, 2019

I don't see a reason not to. Sounds good.

@tylerjereddy tylerjereddy modified the milestones: 1.4.0, 1.3.2 Oct 31, 2019
@tylerjereddy tylerjereddy removed the backport-candidate This fix should be ported by a maintainer to previous SciPy versions. label Nov 29, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants