Skip to content

BUG: problem with building scipy against system boost #15260

New issue
New issue
Closed
Closed
BUG: problem with building scipy against system boost#15260
Labels
Build issuesIssues with building from source, including different choices of architecture, compilers and OSMesonItems related to the introduction of Meson as the new build system for SciPyinvalidCan't be reproduced, or is not actionable
@kloczek

Description

@kloczek
kloczek
opened on Dec 22, 2021

Describe your issue.

I'm using github autogenerated source tar ball from v1.7.3 tag
I'm trying to package scipy module as an rpm package. So I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account. First step is to build module using python3 -sBm build -w`.
Looks like

Prepared site.cfg files has:

[amd]
library_dirs = /usr/lib64
include_dirs = /usr/include/suitesparse
amd_libs = amd

[umfpack]
library_dirs = /usr/lib64
include_dirs = /usr/include/suitesparse
umfpack_libs = umfpack

[openblas]
libraries = flexiblas
library_dirs = /usr/lib64

Looks like wit those settings build by default is trying to use boost cloned as git submodule.

Is any procedure to be able build scipy using ststem installed boost?
Do I need to add something more to my site.cfg?

Reproducing Code Example

git clone -t v1.7.3 https://github.com/scipy/scipy/
cd scipy
python3 -sBm build -w

Error message

Here is build output:

+ /usr/bin/python3 -sBm build -w
/usr/lib/python3.8/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
* Creating virtualenv isolated environment...
* Installing packages in isolated environment... (Cython>=0.29.18,<3.0, numpy; python_version=='3.10' and platform_python_implementation=='PyPy', numpy; python_version=='3.8' and platform_python_implementation=='PyPy', numpy; python_version=='3.9' and platform_python_implementation=='PyPy', numpy==1.16.5; python_version=='3.7' and platform_machine!='aarch64' and platform_python_implementation != 'PyPy', numpy==1.17.3; python_version=='3.8' and (platform_machine!='arm64' or platform_system!='Darwin') and platform_python_implementation != 'PyPy', numpy==1.19.2; python_version=='3.7' and platform_machine=='aarch64', numpy==1.19.2; python_version=='3.8' and platform_machine=='aarch64', numpy==1.19.3; python_version=='3.9' and (platform_machine!='arm64' or platform_system!='Darwin') and platform_python_implementation != 'PyPy', numpy==1.20.0; python_version=='3.7' and platform_python_implementation=='PyPy', numpy==1.21.4; python_version=='3.10' and platform_python_implementation != 'PyPy', pybind11>=2.4.3,<2.8.0, pythran>=0.9.12,<0.10.0, setuptools<58.0.0, wheel<0.38.0)
* Getting dependencies for wheel...
running egg_info
creating scipy.egg-info
writing scipy.egg-info/PKG-INFO
writing dependency_links to scipy.egg-info/dependency_links.txt
writing requirements to scipy.egg-info/requires.txt
writing top-level names to scipy.egg-info/top_level.txt
writing manifest file 'scipy.egg-info/SOURCES.txt'
reading manifest file 'scipy.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no previously-included files matching '*_subr_*.f' found under directory 'scipy/linalg/src/id_dist/src'
no previously-included directories found matching 'benchmarks/env'
no previously-included directories found matching 'benchmarks/results'
no previously-included directories found matching 'benchmarks/html'
no previously-included directories found matching 'benchmarks/scipy'
no previously-included directories found matching 'doc/build'
no previously-included directories found matching 'doc/source/generated'
no previously-included directories found matching '*/__pycache__'
warning: no previously-included files matching '*.pyc' found anywhere in distribution
warning: no previously-included files matching '*~' found anywhere in distribution
warning: no previously-included files matching '*.bak' found anywhere in distribution
warning: no previously-included files matching '*.swp' found anywhere in distribution
warning: no previously-included files matching '*.pyo' found anywhere in distribution
adding license file 'LICENSE.txt'
adding license file 'LICENSES_bundled.txt'
writing manifest file 'scipy.egg-info/SOURCES.txt'
* Installing packages in isolated environment... (wheel)
* Building wheel...
Running from SciPy source directory.
Cythonizing sources
Running scipy/linalg/_generate_pyx.py
Running scipy/special/_generate_pyx.py
Running scipy/stats/_generate_pyx.py
Processing scipy/_lib/_ccallback_c.pyx
Processing scipy/_lib/_test_deprecation_call.pyx
Processing scipy/_lib/_test_deprecation_def.pyx
Processing scipy/_lib/messagestream.pyx
Processing scipy/cluster/_hierarchy.pyx
Processing scipy/cluster/_optimal_leaf_ordering.pyx
Processing scipy/cluster/_vq.pyx
Processing scipy/fftpack/convolve.pyx
Processing scipy/interpolate/_bspl.pyx
Processing scipy/interpolate/_ppoly.pyx
Processing scipy/interpolate/interpnd.pyx
Processing scipy/io/matlab/mio5_utils.pyx
Processing scipy/io/matlab/mio_utils.pyx
Processing scipy/io/matlab/streams.pyx
Processing scipy/linalg/_decomp_update.pyx.in
Processing scipy/linalg/_matfuncs_sqrtm_triu.pyx
Processing scipy/linalg/_solve_toeplitz.pyx
Processing scipy/linalg/cython_blas.pyx
Processing scipy/linalg/cython_lapack.pyx
Processing scipy/ndimage/src/_cytest.pyx
Processing scipy/ndimage/src/_ni_label.pyx
Processing scipy/optimize/_bglu_dense.pyx
Processing scipy/optimize/_group_columns.pyx
Processing scipy/optimize/_highs/cython/src/_highs_constants.pyx
Processing scipy/optimize/_highs/cython/src/_highs_wrapper.pyx
Processing scipy/optimize/_lsq/givens_elimination.pyx
Processing scipy/optimize/_trlib/_trlib.pyx
Processing scipy/optimize/cython_optimize/_zeros.pyx.in
Processing scipy/signal/_max_len_seq_inner.pyx
Processing scipy/signal/_peak_finding_utils.pyx
Processing scipy/signal/_sosfilt.pyx
Processing scipy/signal/_spectral.pyx
Processing scipy/signal/_upfirdn_apply.pyx
Processing scipy/sparse/_csparsetools.pyx.in
Processing scipy/sparse/csgraph/_flow.pyx
Processing scipy/sparse/csgraph/_matching.pyx
Processing scipy/sparse/csgraph/_min_spanning_tree.pyx
Processing scipy/sparse/csgraph/_reordering.pyx
Processing scipy/sparse/csgraph/_shortest_path.pyx
Processing scipy/sparse/csgraph/_tools.pyx
Processing scipy/sparse/csgraph/_traversal.pyx
Processing scipy/spatial/_hausdorff.pyx
Processing scipy/spatial/_voronoi.pyx
Processing scipy/spatial/ckdtree.pyx
Processing scipy/spatial/transform/rotation.pyx
Processing scipy/spatial/qhull.pyx
Processing scipy/special/_comb.pyx
Processing scipy/special/_ellip_harm_2.pyx
Processing scipy/special/_test_round.pyx
Processing scipy/special/_ufuncs.pyx
Processing scipy/special/_ufuncs_cxx.pyx
Processing scipy/special/cython_special.pyx
Processing scipy/stats/_qmc_cy.pyx
Processing scipy/stats/_sobol.pyx
Processing scipy/stats/_stats.pyx
Processing scipy/stats/biasedurn.pyx
Processing scipy/stats/_boost/src/beta_ufunc.pyx
Processing scipy/stats/_boost/src/binom_ufunc.pyx
Processing scipy/stats/_boost/src/nbinom_ufunc.pyx
warning: _cython_special_custom.pxi:9:8: Unreachable code
warning: _cython_special_custom.pxi:13:4: Unreachable code
warning: _cython_special_custom.pxi:21:8: Unreachable code
warning: _cython_special_custom.pxi:25:4: Unreachable code
warning: _cython_special_custom.pxi:33:8: Unreachable code
warning: _cython_special_custom.pxi:37:4: Unreachable code
warning: _cython_special_custom.pxi:45:8: Unreachable code
warning: _cython_special_custom.pxi:49:4: Unreachable code
lapack_opt_info:
lapack_mkl_info:
customize UnixCCompiler
  libraries mkl_rt not found in ['/tmp/build-env-m9a9wxsh/lib', '/usr/local/lib64', '/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/']
  NOT AVAILABLE

openblas_lapack_info:
customize UnixCCompiler
customize UnixCCompiler
customize UnixCCompiler
C compiler: /usr/bin/gcc -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fdata-sections -ffunction-sections -flto=auto -flto-partition=none -D_GNU_SOURCE -fPIC -fwrapv -ffat-lto-objects -D_GNU_SOURCE -fPIC -fwrapv -O2 -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fdata-sections -ffunction-sections -flto=auto -flto-partition=none -D_GNU_SOURCE -fPIC -fwrapv -ffat-lto-objects -O2 -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fdata-sections -ffunction-sections -flto=auto -flto-partition=none -D_GNU_SOURCE -fPIC -fwrapv -ffat-lto-objects -O2 -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fdata-sections -ffunction-sections -flto=auto -flto-partition=none -fPIC

creating /tmp/tmpi3kqtfni/tmp
creating /tmp/tmpi3kqtfni/tmp/tmpi3kqtfni
compile options: '-c'
gcc: /tmp/tmpi3kqtfni/source.c
/usr/bin/gcc /tmp/tmpi3kqtfni/tmp/tmpi3kqtfni/source.o -L/usr/lib64 -lflexiblas -o /tmp/tmpi3kqtfni/a.out
customize UnixCCompiler
  FOUND:
    libraries = ['flexiblas', 'flexiblas']
    library_dirs = ['/usr/lib64']
    language = c
    define_macros = [('HAVE_CBLAS', None)]

  FOUND:
    libraries = ['flexiblas', 'flexiblas']
    library_dirs = ['/usr/lib64']
    language = c
    define_macros = [('HAVE_CBLAS', None)]

/tmp/build-env-m9a9wxsh/lib64/python3.8/site-packages/numpy/distutils/system_info.py:782: UserWarning: Specified path /tmp/build-env-m9a9wxsh/lib64/python3.8/site-packages/numpy/__init__.py/include/python3.8 is invalid.
  return self.get_paths(self.section, key)
/tmp/build-env-m9a9wxsh/lib64/python3.8/site-packages/numpy/distutils/system_info.py:782: UserWarning: Specified path /usr/local/include/python3.8 is invalid.
  return self.get_paths(self.section, key)
/tmp/build-env-m9a9wxsh/lib64/python3.8/site-packages/numpy/distutils/system_info.py:782: UserWarning: Specified path /usr/include/suitesparse/python3.8 is invalid.
  return self.get_paths(self.section, key)
blas_opt_info:
blas_mkl_info:
customize UnixCCompiler
  libraries mkl_rt not found in ['/tmp/build-env-m9a9wxsh/lib', '/usr/local/lib64', '/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/']
  NOT AVAILABLE

blis_info:
customize UnixCCompiler
  libraries blis not found in ['/tmp/build-env-m9a9wxsh/lib', '/usr/local/lib64', '/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/']
  NOT AVAILABLE

openblas_info:
customize UnixCCompiler
customize UnixCCompiler
customize UnixCCompiler
  FOUND:
    libraries = ['flexiblas', 'flexiblas']
    library_dirs = ['/usr/lib64']
    language = c
    define_macros = [('HAVE_CBLAS', None)]

  FOUND:
    libraries = ['flexiblas', 'flexiblas']
    library_dirs = ['/usr/lib64']
    language = c
    define_macros = [('HAVE_CBLAS', None)]

non-existing path in 'scipy/signal/windows': 'tests'
non-existing path in 'scipy/sparse': 'sparsetools/sparsetools_impl.h'
non-existing path in 'scipy/sparse': 'sparsetools/bsr_impl.h'
non-existing path in 'scipy/sparse': 'sparsetools/csc_impl.h'
non-existing path in 'scipy/sparse': 'sparsetools/csr_impl.h'
non-existing path in 'scipy/sparse': 'sparsetools/other_impl.h'
[makenpz] generating scipy/special/tests/data/boost.npz
[makenpz] generating scipy/special/tests/data/gsl.npz
[makenpz] generating scipy/special/tests/data/local.npz
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/pep517/in_process/_in_process.py", line 363, in <module>
    main()
  File "/usr/lib/python3.8/site-packages/pep517/in_process/_in_process.py", line 345, in main
    json_out['return_val'] = hook(**hook_input['kwargs'])
  File "/usr/lib/python3.8/site-packages/pep517/in_process/_in_process.py", line 261, in build_wheel
    return _build_backend().build_wheel(wheel_directory, config_settings,
  File "/tmp/build-env-m9a9wxsh/lib/python3.8/site-packages/setuptools/build_meta.py", line 221, in build_wheel
    return self._build_with_temp_dir(['bdist_wheel'], '.whl',
  File "/tmp/build-env-m9a9wxsh/lib/python3.8/site-packages/setuptools/build_meta.py", line 207, in _build_with_temp_dir
    self.run_setup()
  File "/tmp/build-env-m9a9wxsh/lib/python3.8/site-packages/setuptools/build_meta.py", line 258, in run_setup
    super(_BuildMetaLegacyBackend,
  File "/tmp/build-env-m9a9wxsh/lib/python3.8/site-packages/setuptools/build_meta.py", line 150, in run_setup
    exec(compile(code, __file__, 'exec'), locals())
  File "setup.py", line 631, in <module>
    setup_package()
  File "setup.py", line 627, in setup_package
    setup(**metadata)
  File "/tmp/build-env-m9a9wxsh/lib64/python3.8/site-packages/numpy/distutils/core.py", line 137, in setup
    config = configuration()
  File "setup.py", line 537, in configuration
    config.add_subpackage('scipy')
  File "/tmp/build-env-m9a9wxsh/lib64/python3.8/site-packages/numpy/distutils/misc_util.py", line 1033, in add_subpackage
    config_list = self.get_subpackage(subpackage_name, subpackage_path,
  File "/tmp/build-env-m9a9wxsh/lib64/python3.8/site-packages/numpy/distutils/misc_util.py", line 999, in get_subpackage
    config = self._get_configuration_from_setup_py(
  File "/tmp/build-env-m9a9wxsh/lib64/python3.8/site-packages/numpy/distutils/misc_util.py", line 941, in _get_configuration_from_setup_py
    config = setup_module.configuration(*args)
  File "scipy/setup.py", line 26, in configuration
    config.add_subpackage('_lib')
  File "/tmp/build-env-m9a9wxsh/lib64/python3.8/site-packages/numpy/distutils/misc_util.py", line 1033, in add_subpackage
    config_list = self.get_subpackage(subpackage_name, subpackage_path,
  File "/tmp/build-env-m9a9wxsh/lib64/python3.8/site-packages/numpy/distutils/misc_util.py", line 999, in get_subpackage
    config = self._get_configuration_from_setup_py(
  File "/tmp/build-env-m9a9wxsh/lib64/python3.8/site-packages/numpy/distutils/misc_util.py", line 941, in _get_configuration_from_setup_py
    config = setup_module.configuration(*args)
  File "scipy/_lib/setup.py", line 25, in configuration
    check_boost_submodule()
  File "scipy/_lib/setup.py", line 9, in check_boost_submodule
    raise RuntimeError("Missing the `boost` submodule! Run `git submodule "
RuntimeError: Missing the `boost` submodule! Run `git submodule update --init` to fix this.

SciPy/NumPy/Python version information

scipy 1.7.3

Metadata

Metadata

Assignees

No one assigned

    Labels

    Build issuesIssues with building from source, including different choices of architecture, compilers and OSMesonItems related to the introduction of Meson as the new build system for SciPyinvalidCan't be reproduced, or is not actionable

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions