Skip to content

Conversation

tornaria
Copy link
Contributor

The implementation of sage.misc.package_dir.read_distributions uses the function Cython.Utils.open_source_file() to open a file instead of io.open(). This function was introduced in #29701 (b582789).

The only difference between open_source_file() and io.open() is that the former will open the file as binary to check the encoding. But all our files should be utf-8 so we don't need this trick.

In case I’m mistaken and there is a reason this trick is really necessary (@mkoeppe?) it seems better to include a version of open_source_file(), for two main reasons: (a) we don’t pull the whole of Cython just for a trivial function; (b) changes in Cython don’t affect sagemath (this is not a documented function of Cython, afaict)

📝 Checklist

  • The title is concise, informative, and self-explanatory.
  • The description explains in detail what this PR is about.
  • I have linked a relevant issue or discussion.

@mkoeppe
Copy link
Contributor

mkoeppe commented Feb 11, 2024

The change to just use open is fine with me.

@tornaria
Copy link
Contributor Author

With this PR, I can build a sdist with just setuptools as here:

./bootstrap sagelib
cd pkgs/sagemath-standard
python -m venv venv_for_sdist
venv_for_sdist/bin/pip install build
venv_for_sdist/bin/pip install setuptools
venv_for_sdist/bin/python -m build -snx

Note that I have to ignore (-x) because the build-dependencies have not been changed (they should, because the build-system requires are supposed to be just what is required to run the build-system; for build wheel requirements see https://peps.python.org/pep-0517/#get-requires-for-build-wheel).

@mkoeppe
Copy link
Contributor

mkoeppe commented Feb 11, 2024

AFAIK setuptools does not support specifying requires-for-build-{sdist,wheel,editable}.

We can of course put our own PEP-517/660 build_meta in there if we really want to.

@tornaria
Copy link
Contributor Author

AFAIK setuptools does not support specifying requires-for-build-{sdist,wheel,editable}.

We can of course put our own PEP-517/660 build_meta in there if we really want to.

See: https://peps.python.org/pep-0517/#get-requires-for-build-wheel and https://peps.python.org/pep-0517/#get-requires-for-build-sdist.

I assume we are aiming for pep-517 build and running setup.py directly is not supported.

@mkoeppe
Copy link
Contributor

mkoeppe commented Feb 11, 2024

And PEP-660 for modern editable.

@tornaria
Copy link
Contributor Author

Can you suggest how to build a sdist without the symlink or the PYTHONPATH hack?

@mkoeppe
Copy link
Contributor

mkoeppe commented Feb 11, 2024

I'll prepare a branch.

The implementation of sage.misc.package_dir.read_distributions uses the
function `Cython.Utils.open_source_file()` to open a file instead of
`io.open()`. This function was introduced in sagemath#29701 (b582789).

The only difference between `open_source_file()` and `io.open()` is that
the former will open the file as binary to check the encoding. But all
our files should be utf-8 so we don't need this trick.
@tornaria tornaria force-pushed the sagemath-standard-sdist branch from 17c1c7d to 487a02f Compare February 11, 2024 01:25
@mkoeppe
Copy link
Contributor

mkoeppe commented Feb 11, 2024

@tornaria
Copy link
Contributor Author

That's:

* [pkgs/sagemath-standard: Remove self-reference at build time #37292](https://github.com/sagemath/sage/pull/37292)

What is this supposed to fix?

@mkoeppe
Copy link
Contributor

mkoeppe commented Feb 11, 2024

It fixes the "circular reference" that you complained about in #37287 (comment)

@tornaria
Copy link
Contributor Author

Can you suggest how to build a sdist without the symlink or the PYTHONPATH hack?

@tornaria tornaria closed this Feb 11, 2024
@tornaria tornaria reopened this Feb 11, 2024
@mkoeppe
Copy link
Contributor

mkoeppe commented Feb 11, 2024

We seem to be miscommunicating. You build the sdist by making the build requirements declared per PEP-518 available. For example by building wheels of sage-setup and then sagemath-environment in the required versions and making them available for python -m build -s

@tornaria
Copy link
Contributor Author

It fixes the "circular reference" that you complained about in #37287 (comment)

Nothing to do with this PR.

You were about to tell me how to build a sdist of sagemath-standard from the github repo, but instead you've just turned one obstacle (sage-setup) into two (sage-setup + sagemath-environment).

@mkoeppe
Copy link
Contributor

mkoeppe commented Feb 11, 2024

They are not obstacles, they are build dependencies. Where's the difficulty in building them?

@tornaria
Copy link
Contributor Author

They are not obstacles, they are build dependencies. Where's the difficulty in building them?

They are NOT independent. The idea is that build-system dependencies should be readily available so I can, you know, use them to build the package.

Here I cannot modify and build from source if I need to change sage-setup in sync with sagemath-standard because when I try to build sagemath-standard the build frontend will try to build sage-setup from pypi instead of my version.

Copy link

Documentation preview for this PR (built with commit 487a02f; changes) is ready! 🎉

@mkoeppe
Copy link
Contributor

mkoeppe commented Feb 11, 2024

The idea is that build-system dependencies should be readily available

Whose idea would that be?

@mkoeppe
Copy link
Contributor

mkoeppe commented Feb 11, 2024

Here I cannot modify and build from source if I need to change sage-setup in sync with sagemath-standard because when I try to build sagemath-standard the build frontend will try to build sage-setup from pypi instead of my version.

You already know how to do this. If you don't want to use the venv that is automatically provisioned by build, you make your own venv with the versions that you want to use. Then you run python -m build -nx.

@tornaria
Copy link
Contributor Author

Here I cannot modify and build from source if I need to change sage-setup in sync with sagemath-standard because when I try to build sagemath-standard the build frontend will try to build sage-setup from pypi instead of my version.

You already know how to do this. If you don't want to use the venv that is automatically provisioned by build, you make your own venv with the versions that you want to use. Then you run python -m build -nx.

Have you tried it? As I explained already it doesnt work:

$ python -m build
* Creating venv isolated environment...
* Installing packages in isolated environment... (cypari2 >=2.1.1, cysignals >=1.10.2, cython >=3.0, != 3.0.3, <4.0, gmpy2 ~=2.1.b999, jinja2 >=3.0, jupyter_core >=4.6.3, memory_allocator, numpy >=1.19, pkgconfig, pplpy >=0.8.6, sage-conf ~= 10.3b7, sage-setup ~= 10.3b7, sage_setup[autogen], sagemath-environment ~= 10.3b7, setuptools >= 68.1.1, wheel >=0.36.2)
...
ERROR: Could not find a version that satisfies the requirement sagemath-environment~=10.3b7 (from versions: none)
ERROR: No matching distribution found for sagemath-environment~=10.3b7

@mkoeppe
Copy link
Contributor

mkoeppe commented Feb 11, 2024

Sounds like you need to learn about PIP_FIND_LINKS?

@tornaria
Copy link
Contributor Author

Also: you asked me to remove the symlink,, and I did. I asked politely for an alternative and you just hijacked the issue to push forward completely unrelated stuff.

@tornaria
Copy link
Contributor Author

The idea is that build-system dependencies should be readily available

Whose idea would that be?

PEP-518: "The purpose of this PEP though, is to specify the minimal set of requirements for the build system to simply begin execution."

@mkoeppe
Copy link
Contributor

mkoeppe commented Feb 11, 2024

If you set this PR to "needs review", I'll be happy to set it to "positive review" immediately. It's a simple enough change.

@mkoeppe
Copy link
Contributor

mkoeppe commented Feb 11, 2024

There you go.

@tornaria
Copy link
Contributor Author

If you set this PR to "needs review", I'll be happy to set it to "positive review" immediately. It's a simple enough change.

I "asked for review" in the github interface, but I forgot that even that is NIH.

@mkoeppe
Copy link
Contributor

mkoeppe commented Feb 11, 2024

Yes, we were clearly miscommunicating.

Start from scratch? Feel free to pick a place.

@tornaria
Copy link
Contributor Author

Sounds like you need to learn about PIP_FIND_LINKS?

Can't find it anywhere. If it's a pip thing, let me remind you that we agreed the aim is to pep-517 build, not "pip" build.

@tornaria
Copy link
Contributor Author

Yes, we were clearly miscommunicating.

Start from scratch? Feel free to pick a place.

github issues are definitely not a good place to build a community.

I wish #sagemath would be used, but alas it's not.

@mkoeppe
Copy link
Contributor

mkoeppe commented Feb 11, 2024

I wish #sagemath would be used, but alas it's not.

I can be on Zulip if necessary.

@mkoeppe
Copy link
Contributor

mkoeppe commented Feb 11, 2024

Sounds like you need to learn about PIP_FIND_LINKS?

Can't find it anywhere. If it's a pip thing, let me remind you that we agreed the aim is to pep-517 build, not "pip" build.

PEP-518 does not specify how exactly the declared requirements are to be satisfied.
It is the implementation of the frontend that does.
pypa/build uses pip to provision the declared requirements.
So the environment variables equivalent to pip command-line options can be used to control stuff.

@mkoeppe
Copy link
Contributor

mkoeppe commented Feb 11, 2024

@vbraun vbraun merged commit 81a374d into sagemath:develop Feb 13, 2024
@tornaria tornaria deleted the sagemath-standard-sdist branch February 28, 2025 20:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants