multiprocessing on py>=3.7 causes threading.RLock pickling exception #920
Description
- I have marked all applicable categories:
- exception-raising bug
- visual output bug
- documentation request (i.e. "X is missing from the documentation." If instead I want to ask "how to use X?" I understand StackOverflow#tqdm is more appropriate)
- new feature request
- I have visited the source website, and in particular
read the known issues - I have searched through the issue tracker for duplicates
- I have mentioned version numbers, operating system and
environment, where applicable:import tqdm, sys print(tqdm.__version__, sys.version, sys.platform)
The current parallel-bars example doesn't work as it is raising the following exception on the anaconda prompt terminal:
Multi-processing
Traceback (most recent call last):
File "D:\Anaconda3\envs\sample-project\lib\runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "D:\Anaconda3\envs\sample-project\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "D:\Programming\personal_projects\sample-project\sample_tdqm.py", line 38, in <module>
p = Pool(initializer=tqdm.set_lock, initargs=(tqdm.get_lock(),))
File "D:\Anaconda3\envs\sample-project\lib\multiprocessing\context.py", line 119, in Pool
context=self.get_context())
File "D:\Anaconda3\envs\sample-project\lib\multiprocessing\pool.py", line 176, in __init__
self._repopulate_pool()
File "D:\Anaconda3\envs\sample-project\lib\multiprocessing\pool.py", line 241, in _repopulate_pool
w.start()
File "D:\Anaconda3\envs\sample-project\lib\multiprocessing\process.py", line 112, in start
self._popen = self._Popen(self)
File "D:\Anaconda3\envs\sample-project\lib\multiprocessing\context.py", line 322, in _Popen
return Popen(process_obj)
File "D:\Anaconda3\envs\sample-project\lib\multiprocessing\popen_spawn_win32.py", line 89, in __init__
reduction.dump(process_obj, to_child)
File "D:\Anaconda3\envs\sample-project\lib\multiprocessing\reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
TypeError: can't pickle _thread.RLock objectsHowever, this one from an old comment in an old PR does work.
Environment info
4.43.0 3.7.3 | packaged by conda-forge | (default, Jul 1 2019, 22:01:29) [MSC v.1900 64 bit (AMD64)] win32