This task ticket, organizing the steps necessary to eventually provide sagelib via PyPI, has been superseded by Meta-ticket #29705: Modularize sagelib into separate distributions (distutils packages).

. . . . . . . . . . . . . . . . .

It would be used by a Python user as follows. Let's assume a user has already installed every package that sage-the-distribution provides on their distribution.
Then a simple

pip install sagelib

would install sagelib in the user's Python installation. Then the user could do from sage.all import *.

. . . . . . . . . . . . . . . . .

Here are the first steps:

• Meta-ticket: Clean up src/setup.py to bring it to standard setuptools behavior #21508: Clean up src/setup.py to bring it to standard distutils behavior
  • Make sagelib setup.py self-contained and independent of SAGE_ROOT #21480: Make sagelib setup.py self-contained, independent of SAGE_ROOT
  • Use custom build_ext to compile Cython code #21600: Use custom build_ext to compile Cython code
  • Cleaning up stale installed files in setup() #21604: Cleaning up stale installed files in setup()
  • src/setup.py: Disentangle cleaning of stale installed files in build directory and in install directory #21654: Disentangle cleaning of stale installed files in build directory and in install directory
  • Fix sagelib sdist (src/setup.py sdist) #21516: Fix sagelib sdist (src/setup.py sdist)
  • Make src/setup.py respect --build-base and --inplace, independent of SAGE_CYTHONIZED #21535: Make src/setup.py respect --build-base and --inplace, independent of SAGE_CYTHONIZED
  • build the sage library in place #12659: build the sage library in place
  • Make sure src/setup.py respects --prefix, --install-base, --root, --user #21573: Make sure src/setup.py respects --install-base and --root
  • Don't use make for autogenerated modules in sagelib #21613: Make setup.py not depend on make (follow-up: fix incompatibility with py3 in autogen/pari #22044, setup.py: run_autogen is ran too late #22094, setup.py: generate auto-generated files in setup() #22106)
  • Don't hard-code the path to thebe.js #22061: Don't hard-code the path to thebe.js (dup: Fix symbolic link to thebe.js #21527: Fix symbolic link to thebe.js)
  • Install Jupyter kernel in the correct prefix #25546: Install Jupyter kernel in the correct prefix
  • sage_bootstrap: Add command "sage -package list --output={install-requires,requirements,pipfile,debian,....}" #29041: at ./bootstrap time, generate src/requirements.txt, src/constraints.txt, src/setup.cfg [install_requires] from build/pkgs
• Testsuite for src/setup.py #21678: Testsuite for src/setup.py
• Install SAGE_SRC/ext/ in package data of sage package by setup.py, not SAGE_LOCAL/share/sage/ext/ by build/make/Makefile #21785: Installation of SAGE_SRC/ext/ in SAGE_LOCAL/share/sage/ext/ should be done by setup.py, not build/make/Makefile
• Use package_data instead of data_files in setup.py #20108: Use package_data instead of data_files in setup.py
• Add a separate "build_cython" command to setup.py #21682: Add a separate "cythonize" command to setup.py
• Install src/bin/* scripts via setup.py (scripts, console_scripts) #21569: Install src/bin/* scripts via setup.py (scripts, console_scripts)
• Move non-scripts of src/bin/ elsewhere (and also move their install location) #21570: Move non-scripts of src/bin/ elsewhere (and also move their install location)
• Install COPYING.txt #21571: Install COPYING.txt in SAGE_LOCAL and use it from misc/copying.py
• Improve Cython debugging #21509: Install cython_debug somewhere in SAGE_LOCAL
• Move runtime documentation python modules into src/sage #21732: Move runtime documentation python modules into src/sage
• Python package sage_conf: Provides optional configuration information for sagelib #29038: Python package sage_conf: Provides optional configuration information for sagelib

This defines milestone 1. sagelib is now a well-behaved Python package. It can be built and installed as follows (without invoking sage -sh):

export SAGE_LOCAL=/path/to/local/hierarchy/populated/by/sage/distribution
export SAGE_PKGS=/path/to/sage/distribution/source/directory/build/pkgs
$SAGE_LOCAL/bin/python setup.py install   # or pip install . 

. . . . . . . . . . . . . . . . .

Next steps:

• Remove the dependency on the environment variable SAGE_PKGS (Replace is_package_installed with Features #20382, ...)

This defines milestone 2. sagelib can now be built and installed as follows (without invoking sage -sh):

export SAGE_LOCAL=/path/to/local/hierarchy/populated/by/sage/distribution
$SAGE_LOCAL/bin/python setup.py install   # or pip install . 

. . . . . . . . . . . . . . . . .

Next steps:

• Remove the dependency on the SAGE_LOCAL environment variable
  • hardwired paths in src/sage #15105: hardwired paths in src/sage
  • Clean up unnecessary flint include directories #21803: env.py, misc/cython.py: Define and use FLINT_INCLUDE_DIR
  • Add an autoconf configure script for sagelib #29119: Add an autoconf configure script for sagelib
  • ...

This defines milestone 3. If SAGE_LOCAL is not set, then sagelib will discover system packages and Python packages installed in standard places.

python setup.py install   # or pip install . 

At this point, sagelib will be a standard pip-installable Python source package, ready for upload to PyPI.

Binary packages (using wheels etc.) is beyond the scope of this ticket.

. . . . . . . . . . . . . . . . .

See also:

• Meta-meta-ticket: Build, packaging, testing improvements #29133 META-META-TICKET: Build, packaging, testing improvements

. . . . . . . . . . . . . . References . . . . . . . . . . . . . . .

• https://docs.python.org/2/library/distutils.html
• https://setuptools.readthedocs.io/en/latest/setuptools.html#command-reference
• https://docs.python.org/2/distutils/sourcedist.html
• https://docs.python.org/2/install/
• https://pip.pypa.io/en/stable/reference/pip_install/#pip-install-examples
• http://askubuntu.com/questions/802544/is-sudo-pip-install-still-a-broken-practice
• http://stackoverflow.com/questions/33004708/osx-el-capitan-sudo-pip-install-oserror-errno-1-operation-not-permitted/33004920#33004920
• https://packaging.python.org/science/
• https://packaging.python.org/current/
• https://packaging.python.org/key_projects/#setuptools
• https://setuptools.readthedocs.io/en/latest/setuptools.html#development-mode
• https://www.cac.cornell.edu/wiki/index.php?title=Python_Distutils_Tips
• https://blog.ionelmc.ro/2014/05/25/python-packaging/
• http://stackoverflow.com/questions/7522250/how-to-include-package-data-with-setuptools-distribute

CC: @jdemeyer @williamstein @vbraun @videlec @dimpase @kiwifb @embray @nexttime @sagetrac-aenge @nthiery @miguelmarco @kwankyu @robertwb @infinity0 @tobihan @defeo @slel @timokau @kevinywlui @isuruf @jhpalmieri

Component: build

Keywords: pip, PyPI

Reviewer: Dima Pasechnik

Issue created by migration from https://trac.sagemath.org/ticket/21507