Skip to content

Test involving multiprocessing crashes when pytest 5.1.2 and xdist are used #5971

New issue
New issue
Closed
#6412
Closed
Test involving multiprocessing crashes when pytest 5.1.2 and xdist are used#5971
#6412
Assignees
nicoddemus
Labels
plugin: xdistrelated to the xdist external plugintype: bugproblem that needs to be addressedtype: regressionindicates a problem that was introduced in a release which was working previously
@yuqian90

Description

@yuqian90
yuqian90
opened on Oct 16, 2019

This simple test testCrashXdist1 crashs pytest when xdist is used. It seems to start when pytest==5.1.2 is used. When using pytest==5.1.1, pytest does not crash and ValueError is raised correctly. My OS is linux.

The most likely cause is this change although I'm not sure: #5786

def badFunc():
    raise ValueError()


def testCrashXdist1():
    import multiprocessing

    with multiprocessing.Pool(1) as pool:
        assert pool.apply(badFunc)


def testCrashXdist2():
    assert badFunc()
$ py.test testcrash.py -n1 -k testCrashXdist1
============================================================================================================ test session starts ============================================================================================================
platform linux -- Python 3.6.5, pytest-5.1.2, py-1.8.0, pluggy-0.13.0
rootdir: /tmp/crash
plugins: forked-1.1.1, xdist-1.29.0
gw0 [1]
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/_pytest/main.py", line 191, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/_pytest/main.py", line 235, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/pluggy/hooks.py", line 286, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/pluggy/manager.py", line 92, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/pluggy/manager.py", line 86, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/xdist/remote.py", line 71, in pytest_runtestloop
INTERNALERROR>     self.run_one_test(torun)
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/xdist/remote.py", line 85, in run_one_test
INTERNALERROR>     self.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/pluggy/hooks.py", line 286, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/pluggy/manager.py", line 92, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/pluggy/manager.py", line 86, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/_pytest/runner.py", line 72, in pytest_runtest_protocol
INTERNALERROR>     runtestprotocol(item, nextitem=nextitem)
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/_pytest/runner.py", line 87, in runtestprotocol
INTERNALERROR>     reports.append(call_and_report(item, "call", log))
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/_pytest/runner.py", line 171, in call_and_report
INTERNALERROR>     hook.pytest_runtest_logreport(report=report)
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/pluggy/hooks.py", line 286, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/pluggy/manager.py", line 92, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/pluggy/manager.py", line 86, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/xdist/remote.py", line 109, in pytest_runtest_logreport
INTERNALERROR>     config=self.config, report=report
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/pluggy/hooks.py", line 286, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/pluggy/manager.py", line 92, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/pluggy/manager.py", line 86, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/_pytest/reports.py", line 328, in pytest_report_to_serializable
INTERNALERROR>     data = report._to_json()
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/_pytest/reports.py", line 164, in _to_json
INTERNALERROR>     return _report_to_json(self)
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/_pytest/reports.py", line 394, in _report_to_json
INTERNALERROR>     d["longrepr"] = serialize_longrepr(report)
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/_pytest/reports.py", line 381, in serialize_longrepr
INTERNALERROR>     serialize_repr_crash(repr_crash),
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/_pytest/reports.py", line 367, in serialize_repr_crash
INTERNALERROR>     return reprcrash.__dict__.copy()
INTERNALERROR> AttributeError: 'NoneType' object has no attribute '__dict__'
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/_pytest/main.py", line 191, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/_pytest/main.py", line 235, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/pluggy/hooks.py", line 286, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/pluggy/manager.py", line 92, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/pluggy/manager.py", line 86, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/xdist/dsession.py", line 115, in pytest_runtestloop
INTERNALERROR>     self.loop_once()
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/xdist/dsession.py", line 138, in loop_once
INTERNALERROR>     call(**kwargs)
INTERNALERROR>   File "/tmp/testpytestvenv/lib/python3.6/site-packages/xdist/dsession.py", line 180, in worker_workerfinished
INTERNALERROR>     assert not crashitem, (crashitem, node)
INTERNALERROR> AssertionError: ('testcrash.py::testCrashXdist1', <WorkerController gw0>)
INTERNALERROR> assert not 'testcrash.py::testCrashXdist1'


$ pip list
Package            Version
------------------ -------
apipkg             1.5
atomicwrites       1.3.0
attrs              19.3.0
execnet            1.7.1
importlib-metadata 0.23
more-itertools     7.2.0
packaging          19.2
pip                19.3
pluggy             0.13.0
py                 1.8.0
pyparsing          2.4.2
pytest             5.1.2
pytest-forked      1.1.1
pytest-xdist       1.29.0
setuptools         41.4.0
six                1.12.0
wcwidth            0.1.7
wheel              0.33.6
zipp               0.6.0

Metadata

Metadata

Assignees

Labels

plugin: xdistrelated to the xdist external plugintype: bugproblem that needs to be addressedtype: regressionindicates a problem that was introduced in a release which was working previously

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions