RuntimeError: can't start new thread #522
Description
Apologies in advanced if I haven't filed this correctly, I'm relatively new to this.
I get a RuntimeError: can't start new thread whilst installing packages with conda, and the issue appears to be with when tqdm is called, I filed the issue here conda/conda#7040 and the owner suggested that I should check if it had been reported with upstream tqdm. As far as I can tell this issue hasn't been logged before, nor is it a known issue.
I have provided an example of how I can get the error, whilst using conda on a login node on a HPC whilst trying to install python to an empty environment in miniconda, I mentioned in conda/conda#7040 that this is possibly caused by limited system-wide resources on the login nodes, and pasted the output of ulimit -a if relevant.
If I instead did conda install python --quiet then I don't get the issue, this is reproducable for other packages and also using both system python and python 3.6.4 downloaded through conda in quiet mode.
In this example:
Python version: Python 2.7.5
OS: Scientific Linux 7.2 (Nitrogen)
tqdm: 4.14.0 according to https://github.com/conda/conda/blob/master/conda/_vendor/tqdm/__init__.py#L3
[s1758208@login04(eddie) ~]$ conda create -n test
Solving environment: done
## Package Plan ##
environment location: /exports/eddie/scratch/s1758208/minitest/envs/test
Proceed ([y]/n)? y
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
# $ conda activate test
#
# To deactivate an active environment, use
#
# $ conda deactivate
[s1758208@login04(eddie) ~]$ conda activate test
(test) [s1758208@login04(eddie) ~]$ conda install python
Solving environment: done
## Package Plan ##
environment location: /exports/eddie/scratch/s1758208/minitest/envs/test
added / updated specs:
- python
The following packages will be downloaded:
package | build
---------------------------|-----------------
setuptools-38.5.1 | py36_0 525 KB
libstdcxx-ng-7.2.0 | hdf63c60_3 2.5 MB
libgcc-ng-7.2.0 | hdf63c60_3 6.1 MB
pip-9.0.1 | py36_5 2.2 MB
python-3.6.4 | hc3d631a_3 29.1 MB
------------------------------------------------------------
Total: 40.4 MB
The following NEW packages will be INSTALLED:
ca-certificates: 2017.08.26-h1d4fec5_0
certifi: 2018.1.18-py36_0
libedit: 3.1-heed3624_0
libffi: 3.2.1-hd88cf55_4
libgcc-ng: 7.2.0-hdf63c60_3
libstdcxx-ng: 7.2.0-hdf63c60_3
ncurses: 6.0-h9df7e31_2
openssl: 1.0.2n-hb7f436b_0
pip: 9.0.1-py36_5
python: 3.6.4-hc3d631a_3
readline: 7.0-ha6073c6_4
setuptools: 38.5.1-py36_0
sqlite: 3.22.0-h1bed415_0
tk: 8.6.7-hc745277_3
wheel: 0.30.0-py36hfd4bba0_1
xz: 5.2.3-h55aa19d_2
zlib: 1.2.11-ha838bed_2
Proceed ([y]/n)? y
Downloading and Extracting Packages
setuptools 38.5.1: ##################################################### | 100%
libstdcxx-ng 7.2.0: #################################################### | 100%
# >>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<
Traceback (most recent call last):
File "/exports/eddie/scratch/s1758208/minitest/lib/python3.6/site-packages/conda/exceptions.py", line 789, in __call__
return func(*args, **kwargs)
File "/exports/eddie/scratch/s1758208/minitest/lib/python3.6/site-packages/conda/cli/main.py", line 78, in _main
exit_code = do_call(args, p)
File "/exports/eddie/scratch/s1758208/minitest/lib/python3.6/site-packages/conda/cli/conda_argparse.py", line 77, in do_call
exit_code = getattr(module, func_name)(args, parser)
File "/exports/eddie/scratch/s1758208/minitest/lib/python3.6/site-packages/conda/cli/main_install.py", line 11, in execute
install(args, parser, 'install')
File "/exports/eddie/scratch/s1758208/minitest/lib/python3.6/site-packages/conda/cli/install.py", line 255, in install
handle_txn(progressive_fetch_extract, unlink_link_transaction, prefix, args, newenv)
File "/exports/eddie/scratch/s1758208/minitest/lib/python3.6/site-packages/conda/cli/install.py", line 281, in handle_txn
progressive_fetch_extract.execute()
File "/exports/eddie/scratch/s1758208/minitest/lib/python3.6/site-packages/conda/core/package_cache.py", line 584, in execute
exc = self._execute_actions(prec_or_spec, prec_actions)
File "/exports/eddie/scratch/s1758208/minitest/lib/python3.6/site-packages/conda/core/package_cache.py", line 599, in _execute_actions
progress_bar = ProgressBar(desc, not context.verbosity and not context.quiet, context.json)
File "/exports/eddie/scratch/s1758208/minitest/lib/python3.6/site-packages/conda/common/io.py", line 390, in __init__
self.pbar = tqdm(desc=description, bar_format=bar_format, ascii=True, total=1)
File "/exports/eddie/scratch/s1758208/minitest/lib/python3.6/site-packages/conda/_vendor/tqdm/_tqdm.py", line 388, in __new__
cls.monitor = TMonitor(cls, cls.monitor_interval)
File "/exports/eddie/scratch/s1758208/minitest/lib/python3.6/site-packages/conda/_vendor/tqdm/_tqdm.py", line 83, in __init__
self.start()
File "/exports/eddie/scratch/s1758208/minitest/lib/python3.6/threading.py", line 846, in start
_start_new_thread(self._bootstrap, ())
RuntimeError: can't start new thread