#27824 made $SAGE_LOCAL a venv if a suitable system python3 is found.

In this ticket, we add support for installing builds of Sage with several python versions, sharing the non-Python packages in SAGE_LOCAL:

• keeping install records for Python packages (recognized by having install-requires.txt or requirements.txt) in $SAGE_VENV/var/lib/sage/{installed,scripts,wheels}, separate from those of non-Python packages ($SAGE_LOCAL/var/lib/sage/{installed,scripts})
• likewise for $SAGE_LOCAL/var/tmp/sage/build
• here $SAGE_VENV defaults to $SAGE_LOCAL, but can be overridden to an arbitrary directory that will be used as the wheel-building venv, for example $SAGE_LOCAL/var/lib/sage/venv/$PYTHON_TAG.

Configuration:

• This is activated by running, for example,
  ./configure --with-python=/usr/bin/python3.8 --with-sage-venv="$SAGE_LOCAL/var/lib/sage/venv/python3.8".

As of this ticket, we are able to establish the venv in an arbitrary configured directory instead of SAGE_LOCAL. This is already useful for #31396 to create a wheel-building venv that will not be packaged as part of SAGE_LOCAL.

In follow-up tickets, we gain the full functionality that facilitates testing with several Python versions without having to rebuild the Sage distribution. This depends on:

• Repackage pynac as a pip-installable package #30534 Repackage pynac as a pip-installable package

Follow-ups:

• We also want to support ./configure --with-sage-venv=no which would suppress making any venv (./configure --with-sage-venv=none #30896)

• Support the tricky case: When system python3 is not in use and SAGE_VENV != SAGE_LOCAL, then both a real $SAGE_LOCAL/bin/python3 needs to be built and a venv in SAGE_VENV created... or should python3 be installed in $SAGE_VENV?)

• Optional: A file build/pkgs/SPKG/trees could override the install tree determination - this would allow us to for example install a package such as jupyter_core both into something like SAGE_NOTEBOOK_VENV and SAGE_VENV...

CC: @dimpase @jdemeyer @embray @vbraun @jhpalmieri

Component: build

Keywords: sd111

Author: Matthias Koeppe

Branch/Commit: d68e861

Reviewer: John Palmieri

Issue created by migration from https://trac.sagemath.org/ticket/29013