Skip to content

Meson: add sage cli #39015

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Apr 2, 2025
Merged

Meson: add sage cli #39015

merged 8 commits into from
Apr 2, 2025

Conversation

tobiasdiez
Copy link
Contributor

@tobiasdiez tobiasdiez commented Nov 20, 2024

Meson currently doesn't install any of the scripts in src/bin. This is because

  • they mostly don't make sense for meson - especially doctests, sage packages interactions and various other developer tools shouldn't be installed for normal users
  • they rely on tricky environment variable manipulation
  • they use bash and thus do not work on Windows

Here, we reimplement a very small subset of the sage cli functionality in Python without any env hacks. At the moment only --version, the interactive sage shell, --notebook and -c are implemented

📝 Checklist

  • The title is concise and informative.
  • The description explains in detail what this PR is about.
  • I have linked a relevant issue or discussion.
  • I have created tests covering the changes.
  • I have updated the documentation and checked the documentation preview.

⌛ Dependencies

@antonio-rojas
Copy link
Contributor

I'd say sage -n to run the notebook directly and sage -c to run a single command should definitely be kept. I personally use sage -t a lot for running tests (I know it can be replaced with python -m sage.doctest, but the shortcut is quite handy)

@antonio-rojas
Copy link
Contributor

antonio-rojas commented Nov 22, 2024

The current version seems to work fine to launch sage, but fails at exit. Running exit gives

sage: ERROR:prompt_toolkit.buffer:Loading history failed
Traceback (most recent call last):
  File "/usr/lib/python3.12/site-packages/prompt_toolkit/buffer.py", line 405, in load_history_done
    f.result()
  File "/usr/lib/python3.12/site-packages/prompt_toolkit/buffer.py", line 393, in load_history
    async for item in self.history.load():
  File "/usr/lib/python3.12/site-packages/prompt_toolkit/history.py", line 60, in load
    self._loaded_strings = list(self.load_history_strings())
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/IPython/terminal/interactiveshell.py", line 190, in load_history_strings
    for __, ___, cell in self.shell.history_manager.get_tail(
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'get_tail'

Unhandled exception in event loop:
  File "/usr/lib/python3.12/site-packages/prompt_toolkit/buffer.py", line 393, in load_history
    async for item in self.history.load():
  File "/usr/lib/python3.12/site-packages/prompt_toolkit/history.py", line 60, in load
    self._loaded_strings = list(self.load_history_strings())
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/IPython/terminal/interactiveshell.py", line 190, in load_history_strings
    for __, ___, cell in self.shell.history_manager.get_tail(
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Exception 'NoneType' object has no attribute 'get_tail'
Press ENTER to continue...

and trying with CTRL+D gives

sage:                                                                                                                                                     
/usr/lib/python3.12/site-packages/sage/misc/banner.py:66: DeprecationWarning: Use sage.version instead.
See https://github.com/sagemath/sage/issues/1 for details.
  a("\n│ %-66s │\n" % version())

and displays the banner again. Pressing CTRL+D once again finally exits.

Copy link

github-actions bot commented Nov 29, 2024

Documentation preview for this PR (built with commit 1a872e3; changes) is ready! 🎉
This preview will update shortly after each push to this PR.

@tornaria
Copy link
Contributor

I'd say sage -n to run the notebook directly and sage -c to run a single command should definitely be kept. I personally use sage -t a lot for running tests (I know it can be replaced with python -m sage.doctest, but the shortcut is quite handy)

+1

Also sage <file> to run a file. This includes preparsing *.sage files, and compiling *.pyx files.

+100 on cleaning up the scripts


I agree sage -t is useful. The current cli has some quirks... maybe worth a cleanup at some point. With this PR and the new (meson) build I get:

$ python -m sage.doctest --all
Running doctests...
...
Doctesting entire Sage library.
Skipping '/tmp/tmpdir-tornaria/build-via-sdist-l1daqoy5/sagemath-10.6b6/src/sage/../../src/sage' because it does not have one of the recognized file name extensions
No files to doctest

so it seems the paths are not quite right, and also the build paths are leaking.

Similar experiment using the sagemath_standard wheel works fine (both using sage -t and python -m sage.doctest).


Build paths also leak on introspection. For example, if one does ZZ? you'll see the build file path reported. In fact, the file integer_ring.pyx is not shipped so no path can be shown here. For the same reason, doing ZZ?? will not show the source (this is unfortunate IMO).

OTOH, something like random? shows the correct path (the correct one inside the venv where I installed this wheel) and random?? shows the source.

Comment on lines 30 to 32
from sage.misc.superseded import deprecation

deprecation(1, "Use sage.version instead.")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you remove this?

Comment on lines 39 to 40
if __name__ == "__main__":
sys.exit(main())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this needs to go in __main__.py so that python -m sage.cli works.

Comment on lines 28 to 30
if not input_args:
InteractiveShellCmd(CliOptions()).run()

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if not input_args:
InteractiveShellCmd(CliOptions()).run()

@tornaria
Copy link
Contributor

tornaria commented Mar 7, 2025

If you can fix the 2 or 3 suggestions above, we can get this in even if it's still missing critical functionality so the new package is usable. Changes suggested:

  • remove the deprecation (move to a separate PR where the users of that are also adjusted)
  • remove the two lines in my last suggestion so the thing doesn't run "twice"
  • make python -m sage.cli work
    The first two are trivial (just remove a few lines). The third one should be easy and desirable (so we can "play" with the cli easily even with sagemath-standard).

@tobiasdiez
Copy link
Contributor Author

I'd say sage -n to run the notebook directly and sage -c to run a single command should definitely be kept. I personally use sage -t a lot for running tests (I know it can be replaced with python -m sage.doctest, but the shortcut is quite handy)

Took me a bit longer than expected to come back to this, sorry. The two options -n and -c are now reimplemented in the new interface. I'm not planning to add sage -t since this a) should be replaced by pytest in the near future and b) is mostly relevant for devs and not actual users (and devs can just run the local ./sage -t in the root).

The other mentioned issues (exit issues, python -m cli support, etc) should be fixed as well now.

@tornaria
Copy link
Contributor

tornaria commented Mar 9, 2025

Took me a bit longer than expected to come back to this, sorry. The two options -n and -c are now reimplemented in the new interface.

Thanks, this is looking good.

I'm not planning to add sage -t since this a) should be replaced by pytest in the near future and b) is mostly relevant for devs and not actual users (and devs can just run the local ./sage -t in the root).

a) As much as I agree with on the need to move to standard tooling (like pytest), we have 20 years of doctests, they are not going to be moved to pytest any time soon.

b) Devs are users, in fact the main users from my pov.

c) Here's one important usage: I can write a script in a single *.sage (or *.py), and I can run

$ sage test.sage
[runs the script]

and my script can have functions with doctests and I can easily test them with

$ sage -t test.sage
[runs the tests]

This works with the current sage script, and should keep working. I think it's ok if we make some (minimal) changes in the cli to improve (make it easier to maintain and remove a lot of warts), but removing important functionality is a difficult sell...

The other mentioned issues (exit issues, python -m cli support, etc) should be fixed as well now.

Nice. I'll make some comments on the code later, but this is looking good. I am still using sagemath-standard with this PR merged in, and I am using alias sage-cli='python -m sage.cli' to play with this.

@dimpase
Copy link
Member

dimpase commented Mar 23, 2025

Took me a bit longer than expected to come back to this, sorry. The two options -n and -c are now reimplemented in the new interface.

Thanks, this is looking good.

I'm not planning to add sage -t since this a) should be replaced by pytest in the near future and b) is mostly relevant for devs and not actual users (and devs can just run the local ./sage -t in the root).

a) As much as I agree with on the need to move to standard tooling (like pytest), we have 20 years of doctests, they are not going to be moved to pytest any time soon.

it's not that anything needs to be moved.
Pytest can test docstrings in python files just fine.
With cython it's a bit harder, but most probably nothing needs to be changed, either.

b) Devs are users, in fact the main users from my pov.

c) Here's one important usage: I can write a script in a single *.sage (or *.py), and I can run

$ sage test.sage
[runs the script]

and my script can have functions with doctests and I can easily test them with

$ sage -t test.sage
[runs the tests]

This works with the current sage script, and should keep working. I think it's ok if we make some (minimal) changes in the cli to improve (make it easier to maintain and remove a lot of warts), but removing important functionality is a difficult sell...

we can make sage -t to just call pytest.

Copy link
Member

@dimpase dimpase left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Please switch this to Positive review

@dimpase
Copy link
Member

dimpase commented Mar 24, 2025

the failing

sage -t --warn-long 5.0 --random-seed=306843490678697062143679095807789091604 src/sage/rings/polynomial/plural.pyx  # SignalError in doctesting framework

isn't relevant here as far as I can see.

vbraun pushed a commit to vbraun/sage that referenced this pull request Apr 1, 2025
sagemathgh-39015: Meson: add sage cli
    
<!-- ^ Please provide a concise and informative title. -->
<!-- ^ Don't put issue numbers in the title, do this in the PR
description below. -->
<!-- ^ For example, instead of "Fixes sagemath#12345" use "Introduce new method
to calculate 1 + 2". -->
<!-- v Describe your changes below in detail. -->
<!-- v Why is this change required? What problem does it solve? -->
<!-- v If this PR resolves an open issue, please link to it here. For
example, "Fixes sagemath#12345". -->

Meson currently doesn't install any of the scripts in `src/bin`. This is
because
- they mostly don't make sense for meson - especially doctests, sage
packages interactions and various other developer tools shouldn't be
installed for normal users
- they rely on tricky environment variable manipulation
- they use bash and thus do not work on Windows

Here, we reimplement a very small subset of the sage cli functionality
in Python without any env hacks. At the moment only `--version`, the
interactive sage shell, `--notebook` and `-c` are implemented

### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->

- [ ] The title is concise and informative.
- [ ] The description explains in detail what this PR is about.
- [ ] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation and checked the documentation
preview.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on. For example,
-->
<!-- - sagemath#12345: short description why this is a dependency -->
<!-- - sagemath#34567: ... -->
    
URL: sagemath#39015
Reported by: Tobias Diez
Reviewer(s): Dima Pasechnik, Gonzalo Tornaría
@vbraun vbraun merged commit 40dd00e into sagemath:develop Apr 2, 2025
25 of 26 checks passed
@tobiasdiez tobiasdiez deleted the meson-cli branch April 3, 2025 06:47
@antonio-rojas
Copy link
Contributor

The sage-ipython command is used in sage/interfaces/sage0.py for the Sage pexpect interface, in order to start a sage session without any bells and whistles. What's the plan here? Maybe the sage command could forward all unused parameters to ipython?

@tobiasdiez
Copy link
Contributor Author

Based on a quick look, I would say in sage0 one could simply call python with an init script that sets up a SageTerminalApp with the necessary options.

Maybe the sage command could forward all unused parameters to ipython?

The disadvantage here would be that one doesn't get a meaningful error if one mistypes a command (e.g. sage -n0tebook would then call ipython with -n0tebook). Perhaps an idea would be to have a special cmd arg --ipython which then accepts all parameter and passes them along.

@antonio-rojas
Copy link
Contributor

It would be good to still have a way to trigger the simple mode from the command line. This also affects some third-party UIs (eg. KDE's cantor [1]), which communicate with the sage process via terminal output and thus need to disable all coloring/banner/etc. Currently there's no clear way how to achieve that with meson-built sage.

[1] https://invent.kde.org/education/cantor/-/blob/v24.12.3/src/backends/sage/sagesession.cpp?ref_type=tags#L107

@antonio-rojas
Copy link
Contributor

antonio-rojas commented Apr 4, 2025

Another issue: the missing sage-cleaner executable breaks running the test suite, many of the modules fail with cryptic Bad exit: 1 errors.

I will open an issue to track all test suite regression in the meson build, I assume these aren't caught by the CI because the executables are still present in the source tree.

@antonio-rojas
Copy link
Contributor

I will open an issue to track all test suite regression in the meson build, I assume these aren't caught by the CI because the executables are still present in the source tree.

Done: #39872

@williamstein
Copy link
Contributor

version() is the first command I ever type into most sage notebooks, so bummer it is deprecated.

Second, the deprecation prints out "Use sage.version instead." which isn't as clear as it claims to be:

/tmp/ipykernel_4061/2131152398.py:1: DeprecationWarning: Use sage.version instead.
See https://github.com/sagemath/sage/issues/39015 for details.
  version()

Unfortunately, doing "sage.version" doesn't really work as advertised and in fact what works is "sage.version.version" (not as a function), which takes several tries to figure out:

image

Finally, having version is standard in libraries and was in sage, but got broken. I reopened that ticket.

@dimpase
Copy link
Member

dimpase commented Aug 20, 2025

in the context of notebooks, it's not clear to me what version() should mean. A notebook created in one jupyter kerner version might not work in another.

@tobiasdiez
Copy link
Contributor Author

The wording is perhaps not the best, but a general reference to the module sage.version was intended. For example, you might want to use the full version banner or the simple version string, depending on your needs. version() was very misleading in this regard.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants