conda install RuntimeError: can't start new thread #7040
Description
I'm submitting a...
- bug report
- feature request
Current Behavior
Similar behaviour to issue #6787 and #6624 installing a new package, specifically one that needs downloading, causes a RuntimeError: can't start new thread. Using conda 4.4.11 on a HPC login node, with limited stack size, which I noted in the comments on #6787 could potentially be stopping new threads from being created.
I also was reading pull request #6653 and I believe I get the same behaviour that the RuntimeError only appears when a new thread is created, because if I install in quiet mode I don't get the same issue, however this is broken if I add a new channel to conda config
ulimit -a if relevant
[s1758208@login04(eddie) ~]$ ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 256015
max locked memory (kbytes, -l) unlimited
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 524288
cpu time (seconds, -t) 600
max user processes (-u) 200
virtual memory (kbytes, -v) 2097152
file locks (-x) unlimited
Steps to Reproduce
[s1758208@login04(eddie) ~]$ conda create -n test
[s1758208@login04(eddie) ~]$ conda activate test
(test) [s1758208@login04(eddie) ~]$ conda install python
Solving environment: done
## Package Plan ##
environment location: /exports/csce/eddie/ph/groups/rbm_ml/michael/myconda/envs/test
added / updated specs:
- python
The following packages will be downloaded:
package | build
---------------------------|-----------------
python-3.6.4 | hc3d631a_3 29.1 MB
libstdcxx-ng-7.2.0 | hdf63c60_3 2.5 MB
certifi-2018.1.18 | py36_0 144 KB
zlib-1.2.11 | ha838bed_2 101 KB
sqlite-3.22.0 | h1bed415_0 1.5 MB
wheel-0.30.0 | py36hfd4bba0_1 67 KB
openssl-1.0.2n | hb7f436b_0 3.4 MB
libffi-3.2.1 | hd88cf55_4 43 KB
libgcc-ng-7.2.0 | hdf63c60_3 6.1 MB
ncurses-6.0 | h9df7e31_2 920 KB
tk-8.6.7 | hc745277_3 3.2 MB
setuptools-38.5.1 | py36_0 525 KB
pip-9.0.1 | py36_5 2.2 MB
libedit-3.1 | heed3624_0 171 KB
ca-certificates-2017.08.26 | h1d4fec5_0 263 KB
------------------------------------------------------------
Total: 50.2 MB
The following NEW packages will be INSTALLED:
ca-certificates: 2017.08.26-h1d4fec5_0
certifi: 2018.1.18-py36_0
libedit: 3.1-heed3624_0
libffi: 3.2.1-hd88cf55_4
libgcc-ng: 7.2.0-hdf63c60_3
libstdcxx-ng: 7.2.0-hdf63c60_3
ncurses: 6.0-h9df7e31_2
openssl: 1.0.2n-hb7f436b_0
pip: 9.0.1-py36_5
python: 3.6.4-hc3d631a_3
readline: 7.0-ha6073c6_4
setuptools: 38.5.1-py36_0
sqlite: 3.22.0-h1bed415_0
tk: 8.6.7-hc745277_3
wheel: 0.30.0-py36hfd4bba0_1
xz: 5.2.3-h55aa19d_2
zlib: 1.2.11-ha838bed_2
Proceed ([y]/n)? y
Downloading and Extracting Packages
python 3.6.4: ########################################################## | 100%
libstdcxx-ng 7.2.0: #################################################### | 100%
certifi 2018.1.18: ##################################################### | 100%
# >>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<
Traceback (most recent call last):
File "/home/s1758208/miniconda3/lib/python3.6/site-packages/conda/exceptions.py", line 789, in __call__
return func(*args, **kwargs)
File "/home/s1758208/miniconda3/lib/python3.6/site-packages/conda/cli/main.py", line 78, in _main
exit_code = do_call(args, p)
File "/home/s1758208/miniconda3/lib/python3.6/site-packages/conda/cli/conda_argparse.py", line 77, in do_call
exit_code = getattr(module, func_name)(args, parser)
File "/home/s1758208/miniconda3/lib/python3.6/site-packages/conda/cli/main_install.py", line 11, in execute
install(args, parser, 'install')
File "/home/s1758208/miniconda3/lib/python3.6/site-packages/conda/cli/install.py", line 255, in install
handle_txn(progressive_fetch_extract, unlink_link_transaction, prefix, args, newenv)
File "/home/s1758208/miniconda3/lib/python3.6/site-packages/conda/cli/install.py", line 281, in handle_txn
progressive_fetch_extract.execute()
File "/home/s1758208/miniconda3/lib/python3.6/site-packages/conda/core/package_cache.py", line 584, in execute
exc = self._execute_actions(prec_or_spec, prec_actions)
File "/home/s1758208/miniconda3/lib/python3.6/site-packages/conda/core/package_cache.py", line 599, in _execute_actions
progress_bar = ProgressBar(desc, not context.verbosity and not context.quiet, context.json)
File "/home/s1758208/miniconda3/lib/python3.6/site-packages/conda/common/io.py", line 390, in __init__
self.pbar = tqdm(desc=description, bar_format=bar_format, ascii=True, total=1)
File "/home/s1758208/miniconda3/lib/python3.6/site-packages/conda/_vendor/tqdm/_tqdm.py", line 388, in __new__
cls.monitor = TMonitor(cls, cls.monitor_interval)
File "/home/s1758208/miniconda3/lib/python3.6/site-packages/conda/_vendor/tqdm/_tqdm.py", line 83, in __init__
self.start()
File "/home/s1758208/miniconda3/lib/python3.6/threading.py", line 846, in start
_start_new_thread(self._bootstrap, ())
RuntimeError: can't start new thread
Expected Behavior
new package is installed
Environment Information
`conda info`
active environment : test
active env location : /exports/csce/eddie/ph/groups/rbm_ml/michael/myconda/envs/test
shell level : 1
user config file : /home/s1758208/.condarc
populated config files : /home/s1758208/.condarc
conda version : 4.4.11
conda-build version : not installed
python version : 3.6.4.final.0
base environment : /home/s1758208/miniconda3 (writable)
channel URLs : https://repo.continuum.io/pkgs/main/linux-64
https://repo.continuum.io/pkgs/main/noarch
https://repo.continuum.io/pkgs/free/linux-64
https://repo.continuum.io/pkgs/free/noarch
https://repo.continuum.io/pkgs/r/linux-64
https://repo.continuum.io/pkgs/r/noarch
https://repo.continuum.io/pkgs/pro/linux-64
https://repo.continuum.io/pkgs/pro/noarch
package cache : /exports/csce/eddie/ph/groups/rbm_ml/michael/myconda/pkgs
envs directories : /exports/csce/eddie/ph/groups/rbm_ml/michael/myconda/envs
/home/s1758208/miniconda3/envs
/home/s1758208/.conda/envs
platform : linux-64
user-agent : conda/4.4.11 requests/2.18.4 CPython/3.6.4 Linux/3.10.0-327.36.3.el7.x86_64 rhel/7.2 glibc/2.17
UID:GID : 1824569:1608220
netrc file : /home/s1758208/.netrc
offline mode : False
`conda config --show-sources`
==> /home/s1758208/.condarc <==
envs_dirs:
- /exports/csce/eddie/ph/groups/rbm_ml/michael/myconda/envs
pkgs_dirs:
- /exports/csce/eddie/ph/groups/rbm_ml/michael/myconda/pkgs
channels:
- defaults
`conda list --show-channel-urls`
# packages in environment at /exports/csce/eddie/ph/groups/rbm_ml/michael/myconda/envs/test:
#
# Name Version Build Channel