src/bin/sage-env (with generated configuration in src/bin/sage-env-config; installed in $SAGE_LOCAL/bin) is used in all of the following contexts:

1. Sage-the-distribution while building spkgs,
2. Sage-the-distribution for building sagelib,
3. for environment variables needed to start Sage,
4. as variables by Sage at runtime (through sage.env), and
5. for environment variables affecting subprocesses of Python invoked by sagelib modules.

This ticket proposes to split the configuration according to these 5 contexts. This will make the installation of Sage more modular and flexible.

1./2. Create build/bin/sage-build-env(with generated configuration in build/bin/sage-build-env-config) for the build-time environment variables for sage-the-distribution (spkg and sagelib). It is not installed in $SAGE_LOCAL.
b. In build/make/Makefile.in and the scripts generated by build/bin/sage-spkg, source build/bin/sage-build-env in addition to src/bin/sage-env. This is #29052.
c. Gradually, we will move settings from src/bin/sage-env[-config] to build/bin/sage-build-env[-config] that are known to be only needed for 1./2. (Note that not all compiler-related environment variables can be moved exclusively to 1./2. – some need to be added to 5 as well for some some calls to compilers needed at runtime to support things like %cython (?) and sage.calculus.desolvers.desolve_mintides.) As well as users' use of pip to install additional Python packages!
d. Eventually, we will remove the call to src/bin/sage-env from the build scripts.

2./3. Obtain sagelib's build-time configuration such as src/setup.py's library_dirs and runtime information (sage.env) from sage-config (introduced in #29038).
a. In src/Makefile, poison the SAGE_LOCAL variable.

• make sagelib a script package #29411: make sagelib a script package
• pkgconf: Update to 1.8.0, remove runtime dep on environment variable SAGE_LOCAL #29779: pkg-config installed from SPKG pkgconf should not depend on environment variable SAGE_LOCAL
  b. Pass SAGE_PKGS (if still needed at all - see src/setup.py: Replace use of is_package_installed_and_updated by sage_conf or features #28815 or Meta-ticket: Modularize sagelib into separate distributions (pip-installable packages) sagemath-... for Sage 10.x #29705) via sage-config instead of from src/Makefile.

3./4./5. Using sage_conf (Python module and script sage-config, #29038), make sage.all fully functional when imported from a Python, without setting any environment variables (sage-env).
a. Initially, phase out src/bin/sage-env-config by using sage-config instead.

• Clean up src/bin/sage-env-config.in: Move logic to src/bin/sage-env, move non-environment configuration variables to sage_conf.py #29384: Clean up src/bin/sage-env-config.in: Move logic to src/bin/sage-env, move non-environment configuration variables to sage_conf.py
• Clean-up for src/bin/sage-env, move src/bin/sage-clone-source, src/bin/sage-sdist to build/bin #29825: Clean-up for src/bin/sage-env
  b. Set sage.env variables via the Python module sage_conf instead of relying on information from environment variables. Example: CYSIGNALS_CRASH_DAYS
• sage.env: Give values from sage_conf precedence over environment variables #34236 sage.env: Give values from sage_conf precedence over environment variables
  c. Set environment variables that are needed only by subprocesses invoked by sagelib in the environment of these subprocesses, rather than relying on them begin set in sage-env. This environment could be provided by a variable sage_conf.SAGE_SUBPROCESS_ENV and/or more invidual variables. Example: R_PROFILE
• Use configuration variable MAXIMA instead of hardcoding "maxima" #30563: Use configuration variable MAXIMA instead of hardcoding "maxima"
• Meta-ticket: Set environment for subprocesses invoked by Sage #30818 Set environment for subprocesses invoked by Sage
  d. Finally, remove use of sage-env for all purposes except for sage -sh.

Aspects of downstream sage packaging:

• The goal is to enable downstream sage packagers to use a whole unmodified src directory to build and install sagelib.
  • more precisely, to use the pip-installable source package created by build/pkgs/sagelib/spkg-src (Build sagelib from build/pkgs/sagelib/src, fix setup.py sdist, add spkg-src and tox.ini #29950); see also sage-conf_pypi: API for incremental builds #30036 (pip-installable sage)
• Downstream packagers could provide their own "implementation" (one Python module sage_conf.py) of the configuration module defined by Python package sage_conf: Provides optional configuration information for sagelib #29038, instead of patching sources. They would not necessarily use the reference implementation in build/pkgs/sage_conf, which is intended for use with sage-the-distribution.

Binary packages:

• version of package sage_conf for relocatable binary distributions #31417 version of package sage_conf for relocatable binary distributions

pip-installable Sage

• pip-installable version of package sage_conf - installs non-Python bits of the Sage distribution in ~/.sage/ #29039 pip-installable version of package sage_conf - installs non-Python bits of the Sage distribution in ~/.sage/
• relocatable wheel version of package sage_conf #31396 relocatable wheel version of package sage_conf

More related tickets:

• cleaning of src/bin as described in Move non-scripts of src/bin/ elsewhere (and also move their install location) #21570, Install src/bin scripts by sagelib's setup.py, not make #21559.
• Discover SAGE_SCRIPTS_DIR to make $SAGE_LOCAL/bin/sage work from any directory, in an environment without SAGE_* variables #25486: Discover SAGE_SCRIPTS_DIR to make $SAGE_LOCAL/bin/sage work from any directory, in an environment without SAGE_* variables - this adds a SAGE_ROOT variable to sage-env-config
• src/bin/sage-env, src/bin/sage-env-config.in: Remove PYTHON_FOR_VENV #30724: src/bin/sage-env, src/bin/sage-env-config.in: Remove PYTHON_FOR_VENV
• Meta-meta-ticket: Build, packaging, testing improvements #29133 META-META-TICKET: Build, packaging, testing improvements

Depends on #29038
Depends on #29052

CC: @embray @jdemeyer @nexttime @kiwifb @dimpase @isuruf @antonio-rojas @infinity0 @timokau @orlitzky @jhpalmieri @tobiasdiez

Component: build

Keywords: sd109

Author: Matthias Koeppe

Issue created by migration from https://trac.sagemath.org/ticket/21707