Skip to content

test_tokenize_object_with_recursion_error() triggers stack overflow on Windows due to IPython imports #9804

New issue
New issue
Open
Open
test_tokenize_object_with_recursion_error() triggers stack overflow on Windows due to IPython imports#9804
Labels
needs attentionIt's been a while since this was pushed on. Needs attention from the owner or a maintainer.testsUnit tests and/or continuous integration
@ghost

Description

@ghost
ghost
opened on Jan 4, 2023

Running the following test:
py.test dask -k test_tokenize_object_with_recursion_error

Triggers the following error:

ask/tests/test_base.py::test_tokenize_object_with_recursion_error Windows fatal exception: stack overflow

Thread 0x00005a34 (most recent call first):
  File "C:\Program Files\Python310\lib\concurrent\futures\thread.py", line 81 in _worker
  File "C:\Program Files\Python310\lib\threading.py", line 946 in run
  File "C:\Program Files\Python310\lib\threading.py", line 1009 in _bootstrap_inner
  File "C:\Program Files\Python310\lib\threading.py", line 966 in _bootstrap

Thread 0x00005de4 (most recent call first):
  File "C:\Program Files\Python310\lib\socket.py", line 293 in accept
  File "C:\Users\pafox\Code\dask\.venv\lib\site-packages\pytest_rerunfailures.py", line 457 in run_server
  File "C:\Program Files\Python310\lib\threading.py", line 946 in run
  File "C:\Program Files\Python310\lib\threading.py", line 1009 in _bootstrap_inner
  File "C:\Program Files\Python310\lib\threading.py", line 966 in _bootstrap

Current thread 0x00000b14 (most recent call first):
  File "C:\Users\pafox\Code\dask\dask\dask\utils.py", line 639 in __call__
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 978 in _normalize_seq_func
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 992 in normalize_seq
  File "C:\Users\pafox\Code\dask\dask\dask\utils.py", line 640 in __call__
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 978 in _normalize_seq_func
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 992 in normalize_seq
  File "C:\Users\pafox\Code\dask\dask\dask\utils.py", line 640 in __call__
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 961 in normalize_dict
  File "C:\Users\pafox\Code\dask\dask\dask\utils.py", line 640 in __call__
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 978 in _normalize_seq_func
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 992 in normalize_seq
  File "C:\Users\pafox\Code\dask\dask\dask\utils.py", line 640 in __call__
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 978 in _normalize_seq_func
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 992 in normalize_seq
  File "C:\Users\pafox\Code\dask\dask\dask\utils.py", line 640 in __call__
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 961 in normalize_dict
  File "C:\Users\pafox\Code\dask\dask\dask\utils.py", line 640 in __call__
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 978 in _normalize_seq_func
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 992 in normalize_seq
  File "C:\Users\pafox\Code\dask\dask\dask\utils.py", line 640 in __call__
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 978 in _normalize_seq_func
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 992 in normalize_seq
  File "C:\Users\pafox\Code\dask\dask\dask\utils.py", line 640 in __call__
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 961 in normalize_dict
  File "C:\Users\pafox\Code\dask\dask\dask\utils.py", line 640 in __call__
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 978 in _normalize_seq_func
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 992 in normalize_seq
  File "C:\Users\pafox\Code\dask\dask\dask\utils.py", line 640 in __call__
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 978 in _normalize_seq_func
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 992 in normalize_seq
  File "C:\Users\pafox\Code\dask\dask\dask\utils.py", line 640 in __call__
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 961 in normalize_dict
  File "C:\Users\pafox\Code\dask\dask\dask\utils.py", line 640 in __call__
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 978 in _normalize_seq_func
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 992 in normalize_seq
  File "C:\Users\pafox\Code\dask\dask\dask\utils.py", line 640 in __call__
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 978 in _normalize_seq_func
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 992 in normalize_seq
  File "C:\Users\pafox\Code\dask\dask\dask\utils.py", line 640 in __call__
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 961 in normalize_dict
  File "C:\Users\pafox\Code\dask\dask\dask\utils.py", line 640 in __call__
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 978 in _normalize_seq_func
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 992 in normalize_seq
  File "C:\Users\pafox\Code\dask\dask\dask\utils.py", line 640 in __call__
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 978 in _normalize_seq_func
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 992 in normalize_seq
  File "C:\Users\pafox\Code\dask\dask\dask\utils.py", line 640 in __call__
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 961 in normalize_dict
  File "C:\Users\pafox\Code\dask\dask\dask\utils.py", line 640 in __call__
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 978 in _normalize_seq_func
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 992 in normalize_seq
  File "C:\Users\pafox\Code\dask\dask\dask\utils.py", line 640 in __call__
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 978 in _normalize_seq_func
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 992 in normalize_seq
  File "C:\Users\pafox\Code\dask\dask\dask\utils.py", line 640 in __call__
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 961 in normalize_dict
  File "C:\Users\pafox\Code\dask\dask\dask\utils.py", line 640 in __call__
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 978 in _normalize_seq_func
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 992 in normalize_seq
  File "C:\Users\pafox\Code\dask\dask\dask\utils.py", line 640 in __call__
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 978 in _normalize_seq_func
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 992 in normalize_seq
  File "C:\Users\pafox\Code\dask\dask\dask\utils.py", line 640 in __call__
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 961 in normalize_dict
  File "C:\Users\pafox\Code\dask\dask\dask\utils.py", line 640 in __call__
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 978 in _normalize_seq_func
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 992 in normalize_seq
  File "C:\Users\pafox\Code\dask\dask\dask\utils.py", line 640 in __call__
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 978 in _normalize_seq_func
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 992 in normalize_seq
  File "C:\Users\pafox\Code\dask\dask\dask\utils.py", line 640 in __call__
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 961 in normalize_dict
  File "C:\Users\pafox\Code\dask\dask\dask\utils.py", line 640 in __call__
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 978 in _normalize_seq_func
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 992 in normalize_seq
  File "C:\Users\pafox\Code\dask\dask\dask\utils.py", line 640 in __call__
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 978 in _normalize_seq_func
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 992 in normalize_seq
  File "C:\Users\pafox\Code\dask\dask\dask\utils.py", line 640 in __call__
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 961 in normalize_dict
  File "C:\Users\pafox\Code\dask\dask\dask\utils.py", line 640 in __call__
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 978 in _normalize_seq_func
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 992 in normalize_seq
  File "C:\Users\pafox\Code\dask\dask\dask\utils.py", line 640 in __call__
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 978 in _normalize_seq_func
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 992 in normalize_seq
  File "C:\Users\pafox\Code\dask\dask\dask\utils.py", line 640 in __call__
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 961 in normalize_dict
  File "C:\Users\pafox\Code\dask\dask\dask\utils.py", line 640 in __call__
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 978 in _normalize_seq_func
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 992 in normalize_seq
  File "C:\Users\pafox\Code\dask\dask\dask\utils.py", line 640 in __call__
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 978 in _normalize_seq_func
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 992 in normalize_seq
  File "C:\Users\pafox\Code\dask\dask\dask\utils.py", line 640 in __call__
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 961 in normalize_dict
  File "C:\Users\pafox\Code\dask\dask\dask\utils.py", line 640 in __call__
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 978 in _normalize_seq_func
  File "C:\Users\pafox\Code\dask\dask\dask\base.py", line 992 in normalize_seq
  File "C:\Users\pafox\Code\dask\dask\dask\utils.py", line 640 in __call__
  ...

This issue appears to be related to an increased recursion limit, set by an indirect import of jedi from IPython, which is imported in turn by test_typing.py:
from IPython.display import DisplayObject

This can be fixed by resetting the recursion limit to the default of 1000 after this import.

In theory this could also be fixed by increasing the stack size of the pytest main thread, but there doesn't seem to be any easy way to do this.

  • Dask version: dask-2022.12.1+16.gf721de65f
  • Python version: 3.10.4
  • Operating System: Windows 11
  • Install method (conda, pip, source): source

Metadata

Metadata

Assignees

No one assigned

    Labels

    needs attentionIt's been a while since this was pushed on. Needs attention from the owner or a maintainer.testsUnit tests and/or continuous integration

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions