Skip to content

TST: fix an optimize.linprog test that fails intermittently. #7874

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Sep 17, 2017
Merged

TST: fix an optimize.linprog test that fails intermittently. #7874

merged 1 commit into from
Sep 17, 2017

Conversation

rgommers
Copy link
Member

@rgommers rgommers commented Sep 17, 2017 •
edited
Loading

The tolerance for this test was borderline. It fails about 1 in 3
runs on 32-bit Linux. Note that this test is run in 3 subclasses
with different options dicts. There's no pattern in the failures
it seems, they all fail intermittently.

[ci skip]

@rgommers rgommers added maintenance Items related to regular maintenance tasks scipy.optimize labels Sep 17, 2017
@rgommers rgommers added this to the 1.0.0 milestone Sep 17, 2017
@rgommers
Copy link
Member Author

Will merge this before branching.

@mdhaber can you please double check?

@mdhaber
Copy link
Contributor

mdhaber commented Sep 17, 2017

Is there any information about which particular test this is?

@rgommers
Copy link
Member Author

Is there any information about which particular test this is?

Yes. The whole output (with a couple of extra lines due to my debug-by-print-statement):

self = <scipy.optimize.tests.test_linprog.TestLinprogIPSparse object at 0x9b83d20c>

    def test_bug_6690(self):
        # https://github.com/scipy/scipy/issues/6690
        A_eq = np.array([[0., 0., 0., 0.93, 0., 0.65, 0., 0., 0.83, 0.]])
        b_eq = np.array([0.9626])
        A_ub = np.array([[0., 0., 0., 1.18, 0., 0., 0., -0.2, 0.,
                          -0.22],
                         [0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
                         [0., 0., 0., 0.43, 0., 0., 0., 0., 0., 0.],
                         [0., -1.22, -0.25, 0., 0., 0., -2.06, 0., 0.,
                          1.37],
                         [0., 0., 0., 0., 0., 0., 0., -0.25, 0., 0.]])
        b_ub = np.array([0.615, 0., 0.172, -0.869, -0.022])
        bounds = np.array(
            [[-0.84, -0.97, 0.34, 0.4, -0.33, -0.74, 0.47, 0.09, -1.45, -0.73],
             [0.37, 0.02, 2.86, 0.86, 1.18, 0.5, 1.76, 0.17, 0.32, -0.15]]).T
        c = np.array([-1.64, 0.7, 1.8, -1.06, -1.16,
                      0.26, 2.13, 1.53, 0.66, 0.28])
    
        with suppress_warnings() as sup:
            sup.filter(RuntimeWarning, "scipy.linalg.solve\nIll...")
            sup.filter(OptimizeWarning, "Solving system with option...")
            print(self.method, self.options)
            sol = linprog(c, A_ub=A_ub, b_ub=b_ub, A_eq=A_eq, b_eq=b_eq,
                          bounds=bounds, method=self.method,
                          options=self.options)
        print(sol)
>       _assert_success(sol, desired_fun=-1.191)

A_eq       = array([[ 0.  ,  0.  ,  0.  ,  0.93,  0.  ,  0.65,  0.  ,  0.  ,  0.83,  0.  ]])
A_ub       = array([[ 0.  ,  0.  ,  0.  ,  1.18,  0.  ,  0.  ,  0.  , -0.2 ,  0.  ,
       ...       [ 0.  ,  0.  ,  0.  ,  0.  ,  0.  ,  0.  ,  0.  , -0.25,  0.  ,  0.  ]])
b_eq       = array([ 0.9626])
b_ub       = array([ 0.615,  0.   ,  0.172, -0.869, -0.022])
bounds     = array([[-0.84,  0.37],
       [-0.97,  0.02],
       [ 0.34,  2.86],
       [ ...,  1.76],
       [ 0.09,  0.17],
       [-1.45,  0.32],
       [-0.73, -0.15]])
c          = array([-1.64,  0.7 ,  1.8 , -1.06, -1.16,  0.26,  2.13,  1.53,  0.66,  0.28])
self       = <scipy.optimize.tests.test_linprog.TestLinprogIPSparse object at 0x9b83d20c>
sol        =      con: array([  4.20234039e-07])
     fun: -1.1910003060365739
 message: 'O...     ,
        0.49999989,  0.47      ,  0.09      ,  0.31999958, -0.73      ])
sup        = <numpy.testing.utils.suppress_warnings object at 0x9b83936c>

../scipy/scipy/optimize/tests/test_linprog.py:693: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

res =      con: array([  4.20234039e-07])
     fun: -1.1910003060365739
 message: 'O...     ,
        0.49999989,  0.47      ,  0.09      ,  0.31999958, -0.73      ])
desired_fun = -1.191, desired_x = None, rtol = 1e-08, atol = 1e-08

    def _assert_success(res, desired_fun=None, desired_x=None,
                        rtol=1e-8, atol=1e-8):
        # res: linprog result object
        # desired_fun: desired objective function value or None
        # desired_x: desired solution or None
        assert_(res.success)
        assert_equal(res.status, 0)
        if desired_fun is not None:
            assert_allclose(res.fun, desired_fun,
                            err_msg="converged to an unexpected objective value",
>                           rtol=rtol, atol=atol)
E           AssertionError: 
E           Not equal to tolerance rtol=1e-08, atol=1e-08
E           converged to an unexpected objective value
E           (mismatch 100.0%)
E            x: array(-1.191)
E            y: array(-1.191)

atol       = 1e-08
desired_fun = -1.191
desired_x  = None
res        =      con: array([  4.20234039e-07])
     fun: -1.1910003060365739
 message: 'O...     ,
        0.49999989,  0.47      ,  0.09      ,  0.31999958, -0.73      ])
rtol       = 1e-08

../scipy/scipy/optimize/tests/test_linprog.py:123: AssertionError

@mdhaber
Copy link
Contributor

mdhaber commented Sep 17, 2017

There are a few tests where I've relaxed tolerances, and I would do the same here if it's having trouble.
But I don't recall this ever failing before. Is this happening locally or on TravisCI?

@rgommers
Copy link
Member Author

Locally. Probably due to 32-bit Linux, we don't have that on TravisCI.

@rgommers
Copy link
Member Author

I see now that this is not 100% correct, my comment about options not being defined is not true. This test was simply failing in multiple subclasses.

@rgommers
Copy link
Member Author

So I'll remove the line options = {"sparse": False} again, it's not doing anything. The change in rtol is still correct.

@rgommers
TST: fix an optimize.linprog test that fails intermittently.
ceafcd3 
The tolerance for this test was borderline.  It fails about 1 in 3
runs on 32-bit Linux.  Note that this test is run in 3 subclasses
with different ``options`` dicts.  There's no pattern in the failures
it seems, they all fail intermittently.

[ci skip]
@rgommers
Copy link
Member Author

Skipped CI because merging it straight away. Review still welcome of course.

@rgommers rgommers merged commit 2702e67 into scipy:master Sep 17, 2017
@rgommers rgommers deleted the fix-linprog-failure branch September 17, 2017 08:23
@rgommers rgommers mentioned this pull request Sep 17, 2017
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
maintenance Items related to regular maintenance tasks scipy.optimize
Projects
None yet
Milestone
1.0.0
Development

Successfully merging this pull request may close these issues.
2 participants
@rgommers @mdhaber