Race conditions because of DESTDIR #26018
Description
Using a staging directory for package install can introduce race conditions. For example, when installing a Python package, the package is installed first and then the egg-info directory is installed. The latter is what pkg_resources uses to consider a package "installed".
However, when a DESTDIR install is used, the order that files appear in the final installation directory might be different.
If the egg-info directory is copied to site-packages before the actual Python package, then this can happen for example:
Installing alabaster-0.7.10
Installing package alabaster using pip2
Ignoring indexes: https://pypi.python.org/simple
Processing /home/jdemeyer/sage-test/local/var/tmp/sage/build/alabaster-0.7.10/src
Running setup.py (path:/tmp/pip-0ac_4C-build/setup.py) egg_info for package from file:///home/jdemeyer/sage-test/local/var/tmp/sage/build/alabaster-0.7.10/src
Running command python setup.py egg_info
running egg_info
creating pip-egg-info/alabaster.egg-info
writing pip-egg-info/alabaster.egg-info/PKG-INFO
writing top-level names to pip-egg-info/alabaster.egg-info/top_level.txt
writing dependency_links to pip-egg-info/alabaster.egg-info/dependency_links.txt
writing manifest file 'pip-egg-info/alabaster.egg-info/SOURCES.txt'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-0ac_4C-build/setup.py", line 37, in <module>
'Programming Language :: Python :: 3.4',
File "/home/jdemeyer/sage-test/local/lib/python2.7/site-packages/setuptools/__init__.py", line 129, in setup
return distutils.core.setup(**attrs)
File "/home/jdemeyer/sage-test/local/lib/python2.7/distutils/core.py", line 151, in setup
dist.run_commands()
File "/home/jdemeyer/sage-test/local/lib/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/home/jdemeyer/sage-test/local/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/home/jdemeyer/sage-test/local/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 278, in run
self.find_sources()
File "/home/jdemeyer/sage-test/local/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 293, in find_sources
mm.run()
File "/home/jdemeyer/sage-test/local/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 524, in run
self.add_defaults()
File "/home/jdemeyer/sage-test/local/lib/python2.7/site-packages/setuptools/command/egg_info.py", line 563, in add_defaults
rcfiles = list(walk_revctrl())
File "/home/jdemeyer/sage-test/local/lib/python2.7/site-packages/setuptools/command/sdist.py", line 20, in walk_revctrl
for item in ep.load()(dirname):
File "/home/jdemeyer/sage-test/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2405, in load
return self.resolve()
File "/home/jdemeyer/sage-test/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2411, in resolve
module = __import__(self.module_name, fromlist=['__name__'], level=0)
File "/home/jdemeyer/sage-test/local/lib/python2.7/site-packages/setuptools_scm/__init__.py", line 9, in <module>
from .utils import trace
ImportError: No module named utils
What you see here is that setuptools_scm is used by pkg_resources even though the package setuptools_scm has not fully been installed.
See also: #26160 (py3: race condition with .pth files)
CC: @embray @dimpase @jhpalmieri
Component: build
Author: Erik Bray, John Palmieri
Branch: 6b20b84
Reviewer: John Palmieri, Matthias Koeppe
Issue created by migration from https://trac.sagemath.org/ticket/26018