Skip to content

Stochastic CI failure in TestSineIntegral #8577

@jakelishman

Description

@jakelishman

Environment

  • Qiskit Terra version: main @ a2ae2ed
  • Python version: observed on 3.7 and 3.10
  • Operating system: Windows

What is happening?

CI fails stochastically - as rough estimate, maybe around 1 in 5 CI runs that make it to the second stage - on Windows in an amplitude estimator test (TestSineIntegral.test_confidence_intervals). The traceback from CI is:

test.python.algorithms.test_amplitude_estimators.TestSineIntegral.test_confidence_intervals_2
---------------------------------------------------------------------------------------------

Captured traceback:
~~~~~~~~~~~~~~~~~~~
    Traceback (most recent call last):

      File "D:\a\1\s\test-job\lib\site-packages\ddt.py", line 191, in wrapper
    return func(self, *args, **kwargs)

      File "D:\a\1\s\test\python\algorithms\test_amplitude_estimators.py", line 412, in test_confidence_intervals
    self.assertGreater(self._statevector.time_taken, 0.0)

      File "C:\hostedtoolcache\windows\Python\3.7.9\x64\lib\unittest\case.py", line 1251, in assertGreater
    self.fail(self._formatMessage(msg, standardMsg))

      File "C:\hostedtoolcache\windows\Python\3.7.9\x64\lib\unittest\case.py", line 693, in fail
    raise self.failureException(msg)

    AssertionError: 0.0 not greater than 0.0

Note that the actual failure is that the time_taken attribute of the statevector simulator is coming back as 0.0, which fails a test asserting that it's larger. On a quick scan through the code, I see that utils.run_circuits adds a time_taken = 0.0 attribute to its output right before returning it, if the attribute doesn't already exist. I don't know if that's relevant.

How can we reproduce the issue?

Currently I don't have a reproducer - it's stochastic in CI, and seemingly only on Windows.

What should happen?

Perhaps a time_taken field isn't getting put in correctly somewhere in which case that needs correcting, or perhaps the simulation runs sufficiently fast that the timing method occasionally rounds down to 0? This latter option has some evidence in that the failure on appears on Windows, since timing is different (and I believe less granular) there. If it is the rounding behaviour, the solution may be to remove the failing line in the test, but it would be good to know the reason first.

Any suggestions?

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingmod: algorithmsRelated to the Algorithms moduletype: qaIssues and PRs that relate to testing and code quality

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions