Issue installing via pip inside docker container for linux/arm64 #84
Description
Outline
I am getting cython related errors when attempting to pip install sparse_dot_topn inside my dockerfile. I am unsure how the error is arising as I can see the files in the sparse_dot_topn repo and they have not changed in 2 years! Thanks in advance.
I am on an Apple M1 with macOS 13.2, unsure if that is relevant here though as we are inside the container.
Problem
FROM python:3.9.7-slim
RUN apt-get update && apt-get -y upgrade
RUN apt-get install -y build-essential cmake git wget
RUN pip install pip --upgrade
RUN pip install sparse_dot_topn==0.3.0 --no-cache-dirRunning the command:
docker buildx build --platform linux/amd64,linux/arm64 .
I get the following error but only in linux/arm64:
[linux/arm64 8/9] RUN pip install sparse_dot_topn==0.3.0 --no-cache-dir:
0.301 Collecting sparse_dot_topn==0.3.0
0.332 Downloading sparse_dot_topn-0.3.0.tar.gz (15 kB)
0.338 Installing build dependencies: started
1.371 Installing build dependencies: finished with status 'done'
1.372 Getting requirements to build wheel: started
1.463 Getting requirements to build wheel: finished with status 'done'
1.466 Installing backend dependencies: started
6.541 Installing backend dependencies: finished with status 'done'
6.542 Preparing metadata (pyproject.toml): started
6.794 Preparing metadata (pyproject.toml): finished with status 'done'
6.798 Requirement already satisfied: setuptools>=42 in /usr/local/lib/python3.9/site-packages (from sparse_dot_topn==0.3.0) (57.5.0)
6.799 Requirement already satisfied: cython>=0.29.15 in /usr/local/lib/python3.9/site-packages (from sparse_dot_topn==0.3.0) (3.0.0)
6.799 Requirement already satisfied: numpy>=1.16.6 in /usr/local/lib/python3.9/site-packages (from sparse_dot_topn==0.3.0) (1.25.1)
6.800 Requirement already satisfied: scipy>=1.2.3 in /usr/local/lib/python3.9/site-packages (from sparse_dot_topn==0.3.0) (1.11.1)
6.828 Building wheels for collected packages: sparse_dot_topn
6.829 Building wheel for sparse_dot_topn (pyproject.toml): started
8.972 Building wheel for sparse_dot_topn (pyproject.toml): finished with status 'error'
8.982 error: subprocess-exited-with-error
8.982
8.982 × Building wheel for sparse_dot_topn (pyproject.toml) did not run successfully.
8.982 │ exit code: 1
8.982 ╰─> [166 lines of output]
8.982 running bdist_wheel
8.982 running build
8.982 running build_py
8.982 creating build
8.982 creating build/lib.linux-aarch64-cpython-39
8.982 creating build/lib.linux-aarch64-cpython-39/sparse_dot_topn
8.982 copying sparse_dot_topn/__init__.py -> build/lib.linux-aarch64-cpython-39/sparse_dot_topn
8.982 copying sparse_dot_topn/awesome_cossim_topn.py -> build/lib.linux-aarch64-cpython-39/sparse_dot_topn
8.982 running egg_info
8.982 writing sparse_dot_topn.egg-info/PKG-INFO
8.982 writing dependency_links to sparse_dot_topn.egg-info/dependency_links.txt
8.982 writing requirements to sparse_dot_topn.egg-info/requires.txt
8.982 writing top-level names to sparse_dot_topn.egg-info/top_level.txt
8.982 reading manifest file 'sparse_dot_topn.egg-info/SOURCES.txt'
8.982 reading manifest template 'MANIFEST.in'
8.982 adding license file 'LICENSE'
8.982 writing manifest file 'sparse_dot_topn.egg-info/SOURCES.txt'
8.982 copying sparse_dot_topn/array_wrappers.pxd -> build/lib.linux-aarch64-cpython-39/sparse_dot_topn
8.982 copying sparse_dot_topn/array_wrappers.pyx -> build/lib.linux-aarch64-cpython-39/sparse_dot_topn
8.982 copying sparse_dot_topn/sparse_dot_topn.pyx -> build/lib.linux-aarch64-cpython-39/sparse_dot_topn
8.982 copying sparse_dot_topn/sparse_dot_topn_parallel.h -> build/lib.linux-aarch64-cpython-39/sparse_dot_topn
8.982 copying sparse_dot_topn/sparse_dot_topn_source.h -> build/lib.linux-aarch64-cpython-39/sparse_dot_topn
8.982 copying sparse_dot_topn/sparse_dot_topn_threaded.pyx -> build/lib.linux-aarch64-cpython-39/sparse_dot_topn
8.982 running build_ext
8.982 /tmp/pip-build-env-qpr73hsx/normal/lib/python3.9/site-packages/Cython/Compiler/Main.py:381: FutureWarning: Cython directive 'language_level' not set, using '3str' for now (Py3). This has changed from earlier releases! File: /tmp/pip-install-ac4zogpb/sparse-dot-topn_e59d95f11f06430da4966db7104429ac/sparse_dot_topn/array_wrappers.pxd
8.982 tree = Parsing.p_module(s, pxd, full_module_name)
8.982 Compiling ./sparse_dot_topn/array_wrappers.pyx because it changed.
8.982 [1/1] Cythonizing ./sparse_dot_topn/array_wrappers.pyx
8.982 building 'sparse_dot_topn.array_wrappers' extension
8.982 creating build/temp.linux-aarch64-cpython-39
8.982 creating build/temp.linux-aarch64-cpython-39/sparse_dot_topn
8.982 gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/usr/local/include/python3.9 -I/tmp/pip-build-env-qpr73hsx/normal/lib/python3.9/site-packages/numpy/core/include -c ./sparse_dot_topn/array_wrappers.cpp -o build/temp.linux-aarch64-cpython-39/./sparse_dot_topn/array_wrappers.o -std=c++0x -pthread -O3
8.982 g++ -shared -Wl,--strip-all build/temp.linux-aarch64-cpython-39/./sparse_dot_topn/array_wrappers.o -L/usr/local/lib -o build/lib.linux-aarch64-cpython-39/sparse_dot_topn/array_wrappers.cpython-39-aarch64-linux-gnu.so
8.982 /tmp/pip-build-env-qpr73hsx/normal/lib/python3.9/site-packages/Cython/Compiler/Main.py:381: FutureWarning: Cython directive 'language_level' not set, using '3str' for now (Py3). This has changed from earlier releases! File: /tmp/pip-install-ac4zogpb/sparse-dot-topn_e59d95f11f06430da4966db7104429ac/sparse_dot_topn/sparse_dot_topn.pyx
8.982 tree = Parsing.p_module(s, pxd, full_module_name)
8.982
8.982 Error compiling Cython file:
8.982 ------------------------------------------------------------
8.982 ...
8.982 # April 14, 2021
8.982
8.982 # distutils: language = c++
8.982
8.982 from libcpp.vector cimport vector
8.982 from array_wrappers cimport ArrayWrapper_int, ArrayWrapper_double
8.982 ^
8.982 ------------------------------------------------------------
8.982
8.982 sparse_dot_topn/sparse_dot_topn.pyx:23:0: 'array_wrappers.pxd' not found
8.982
8.982 Error compiling Cython file:
8.982 ------------------------------------------------------------
8.982 ...
8.982 # April 14, 2021
8.982
8.982 # distutils: language = c++
8.982
8.982 from libcpp.vector cimport vector
8.982 from array_wrappers cimport ArrayWrapper_int, ArrayWrapper_double
8.982 ^
8.982 ------------------------------------------------------------
8.982
8.982 sparse_dot_topn/sparse_dot_topn.pyx:23:0: 'array_wrappers/ArrayWrapper_int.pxd' not found
8.982
8.982 Error compiling Cython file:
8.982 ------------------------------------------------------------
8.982 ...
8.982 # April 14, 2021
8.982
8.982 # distutils: language = c++
8.982
8.982 from libcpp.vector cimport vector
8.982 from array_wrappers cimport ArrayWrapper_int, ArrayWrapper_double
8.982 ^
8.982 ------------------------------------------------------------
8.982
8.982 sparse_dot_topn/sparse_dot_topn.pyx:23:0: 'array_wrappers/ArrayWrapper_double.pxd' not found
8.982
8.982 Error compiling Cython file:
8.982 ------------------------------------------------------------
8.982 ...
8.982
8.982 if nnz_max_is_too_small:
8.982
8.982 # raise Exception("In sparse_dot_topn.pyx")
8.982
8.982 c_indices = np.asarray(ArrayWrapper_int(vCj)).squeeze(axis=0)
8.982 ^
8.982 ------------------------------------------------------------
8.982
8.982 sparse_dot_topn/sparse_dot_topn.pyx:211:25: 'ArrayWrapper_int' is not a constant, variable or function identifier
8.982
8.982 Error compiling Cython file:
8.982 ------------------------------------------------------------
8.982 ...
8.982 if nnz_max_is_too_small:
8.982
8.982 # raise Exception("In sparse_dot_topn.pyx")
8.982
8.982 c_indices = np.asarray(ArrayWrapper_int(vCj)).squeeze(axis=0)
8.982 c_data = np.asarray(ArrayWrapper_double(vCx)).squeeze(axis=0)
8.982 ^
8.982 ------------------------------------------------------------
8.982
8.982 sparse_dot_topn/sparse_dot_topn.pyx:212:22: 'ArrayWrapper_double' is not a constant, variable or function identifier
8.982 Compiling ./sparse_dot_topn/sparse_dot_topn.pyx because it changed.
8.982 [1/1] Cythonizing ./sparse_dot_topn/sparse_dot_topn.pyx
8.982 Traceback (most recent call last):
8.982 File "/usr/local/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
8.982 main()
8.982 File "/usr/local/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
8.982 json_out['return_val'] = hook(**hook_input['kwargs'])
8.982 File "/usr/local/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
8.982 return _build_backend().build_wheel(wheel_directory, config_settings,
8.982 File "/tmp/pip-build-env-qpr73hsx/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 416, in build_wheel
8.982 return self._build_with_temp_dir(['bdist_wheel'], '.whl',
8.982 File "/tmp/pip-build-env-qpr73hsx/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 401, in _build_with_temp_dir
8.982 self.run_setup()
8.982 File "/tmp/pip-build-env-qpr73hsx/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 338, in run_setup
8.982 exec(code, locals())
8.982 File "<string>", line 65, in <module>
8.982 File "/tmp/pip-build-env-qpr73hsx/overlay/lib/python3.9/site-packages/setuptools/__init__.py", line 107, in setup
8.982 return distutils.core.setup(**attrs)
8.982 File "/tmp/pip-build-env-qpr73hsx/overlay/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 185, in setup
8.982 return run_commands(dist)
8.982 File "/tmp/pip-build-env-qpr73hsx/overlay/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
8.982 dist.run_commands()
8.982 File "/tmp/pip-build-env-qpr73hsx/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
8.982 self.run_command(cmd)
8.982 File "/tmp/pip-build-env-qpr73hsx/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 1234, in run_command
8.982 super().run_command(command)
8.982 File "/tmp/pip-build-env-qpr73hsx/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
8.982 cmd_obj.run()
8.982 File "/tmp/pip-build-env-qpr73hsx/overlay/lib/python3.9/site-packages/wheel/bdist_wheel.py", line 343, in run
8.982 self.run_command("build")
8.982 File "/tmp/pip-build-env-qpr73hsx/overlay/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
8.982 self.distribution.run_command(command)
8.982 File "/tmp/pip-build-env-qpr73hsx/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 1234, in run_command
8.982 super().run_command(command)
8.982 File "/tmp/pip-build-env-qpr73hsx/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
8.982 cmd_obj.run()
8.982 File "/tmp/pip-build-env-qpr73hsx/overlay/lib/python3.9/site-packages/setuptools/_distutils/command/build.py", line 131, in run
8.982 self.run_command(cmd_name)
8.982 File "/tmp/pip-build-env-qpr73hsx/overlay/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
8.982 self.distribution.run_command(command)
8.982 File "/tmp/pip-build-env-qpr73hsx/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 1234, in run_command
8.982 super().run_command(command)
8.982 File "/tmp/pip-build-env-qpr73hsx/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
8.982 cmd_obj.run()
8.982 File "/tmp/pip-build-env-qpr73hsx/overlay/lib/python3.9/site-packages/setuptools/command/build_ext.py", line 84, in run
8.982 _build_ext.run(self)
8.982 File "/tmp/pip-build-env-qpr73hsx/overlay/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
8.982 self.build_extensions()
8.982 File "/tmp/pip-build-env-qpr73hsx/overlay/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 467, in build_extensions
8.982 self._build_extensions_serial()
8.982 File "/tmp/pip-build-env-qpr73hsx/overlay/lib/python3.9/site-packages/setuptools/_distutils/command/build_ext.py", line 493, in _build_extensions_serial
8.982 self.build_extension(ext)
8.982 File "/tmp/pip-build-env-qpr73hsx/overlay/lib/python3.9/site-packages/setuptools/command/build_ext.py", line 246, in build_extension
8.982 _build_ext.build_extension(self, ext)
8.982 File "/tmp/pip-build-env-qpr73hsx/normal/lib/python3.9/site-packages/Cython/Distutils/build_ext.py", line 122, in build_extension
8.982 new_ext = cythonize(
8.982 File "/tmp/pip-build-env-qpr73hsx/normal/lib/python3.9/site-packages/Cython/Build/Dependencies.py", line 1134, in cythonize
8.982 cythonize_one(*args)
8.982 File "/tmp/pip-build-env-qpr73hsx/normal/lib/python3.9/site-packages/Cython/Build/Dependencies.py", line 1301, in cythonize_one
8.982 raise CompileError(None, pyx_file)
8.982 Cython.Compiler.Errors.CompileError: ./sparse_dot_topn/sparse_dot_topn.pyx
8.982 [end of output]
8.982
8.982 note: This error originates from a subprocess, and is likely not a problem with pip.
8.982 ERROR: Failed building wheel for sparse_dot_topn
8.982 Failed to build sparse_dot_topn
8.983 ERROR: Could not build wheels for sparse_dot_topn, which is required to install pyproject.toml-based projects
------