Skip to content

Parallel docbuild, cythonization: Use GNU make's POSIX jobserver protocol #30369

@mkoeppe

Description

@mkoeppe

The correct way out of the mess with recommending MAKE='make -j8' instead of make -j8 (which we do because of the outdated GNU make 3.81 on macOS - see #21610, #30345) is to use GNU make's POSIX job server protocol - https://www.gnu.org/software/make/manual/html_node/POSIX-Jobserver.html

  • Create a wrapper script in Python that tries to acquire >= A, <= B jobs from the jobserver within <= T wall clock seconds, sets the environment variable SAGE_NUM_THREADS to min(C, NUM_JOBS_RECEIVED), and invokes an arbitrary script.

  • Use this script as a wrapper for the sagelib build and docbuild.

  • Replace the recommendation MAKE='make -j8' make by the familiar make -j8.

  • In the top-level Makefile, filter out -j so that also users who continue to follow the old recommendation MAKE='make -j8' make benefit from the improvements, and to get rid of the "disabling jobserver" messages.

CC: @dimpase @orlitzky @jhpalmieri @Etn40ff

Component: build

Issue created by migration from https://trac.sagemath.org/ticket/30369

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions