Thanks for doing this. These changes look good.

Actually right now the best thing would just be for mpmath to hold off on making any prereleases though. Right now things seem to be working and there is a risk that further changes might cause further problems. I don't know what strange things people are doing downstream with packaging tools...

I would like to get a sympy release out with an upper cap on mpmath: sympy/sympy#26413 sympy/sympy#26273

Then it should be possible for mpmath to put out prereleases without anything breaking.

right now the best thing would just be for mpmath to hold off on making any prereleases though

Ok. Though, I think changes aren't too big:

• 22859b0 Merge pull request Support binary/hexadecimal string output #711 from skirpichev/hex
• afb829b Merge pull request Misc fixes #759 from skirpichev/misc
• d294c04 Merge pull request Improve pickling support #761 from skirpichev/pickling
• 3512b74 Merge pull request Misc fixes #763 from skirpichev/misc
• 4c1b435 Merge pull request Misc fixes #767 from skirpichev/misc
• aa1c0b9 Merge pull request Update README.rst #770 from ChefAharoni/patch-1

Then it should be possible for mpmath to put out prereleases without anything breaking.

Unless someone will install sympy pre-release...

Then it should be possible for mpmath to put out prereleases without anything breaking.

Unless someone will install sympy pre-release...

I think we just need to ensure that each release of sympy whether it is a full release or a prerelease has an upper cap on mpmath and is fully tested with that capped version.

I want to cap mpmath < 1.4 so that a mpmath 1.3.1 release would still be picked up in case there is a need to put out 1.3.1 e.g. for Python 3.13 support or something like that.

we just need to ensure that each release of sympy whether it is a full release or a prerelease has an upper cap on mpmath

Limiting upper versions does make sense for stable releases. But for pre-releases - I don't think so, unless you have many deps. People should install pre-releases only intentionally and with understanding of consequences.

People should install pre-releases only intentionally and with understanding of consequences.

The purpose of installing the prerelease is to test in advance what can be expected to happen once the final release is released so there should not be any difference in behaviour. I think that a pre-release of e.g. sympy should have version constraints that match what the future final release would be expected to have. For example if I pip install sympy==1.13.0rc1 then I would hope that this gets the same version of mpmath that pip install sympy==1.13.0 would be expected to get if there are no changes between rc1 and final.

I think that a pre-release of e.g. sympy should have version constraints that match what the future final release would be expected to have.

That depends, e.g. clearly make sense for late rc-releases. But alpha- and beta-releases - also pre-releases c.f. pypi point of view.

I would say that the alpha/beta releases are where you bump the dependency constraints.

If someone wants to use pip install --pre then they can install prereleases of everything transitively and deal with the bleeding edge consequences. If they just track prereleases of sympy though then the bumped version constraints would be part of what they are tracking.

Ok, I did tests for mpq in domains - that works too. The only missing thing from sympy/sympy@eef227f is from_str() changes.

It's support for underscores, 9585bb9. Obviously, it's a compatibility break. I think it's a good idea to support this, but we could implement that differently. The current code uses gmpy2 approach: just filter out underscores. This accepts valid python strings, but also much more. Instead, we could choose to be strictly python-compatible. Let me know if it worth to reopen #613 for discussion. But see also aleaxit/gmpy#210.

Edit:
We also have following differences in imported names:

In [4]: mpmath_names = [_ for _ in dir(mpmath) if not _.startswith('__')]

In [5]: mpmath_libmp_names = [_ for _ in dir(mpmath.libmp) if not _.startswith('__')]

In [6]: set(mpmath_names_130) - set(mpmath_names)
Out[6]: {'_ctx_mp', 'doctests', 'math2', 'runtests'}

In [7]: set(mpmath_libmp_names_130) - set(mpmath_libmp_names)
Out[7]: {'MPZ_TYPE', 'STRICT', 'from_bstr', 'normalize1', 'sage', 'to_bstr'}

In [8]: set(mpmath_names) - set(mpmath_names_130)
Out[8]: {'libfp', 'lower_gamma', 'rank', 'upper_gamma'}

In [9]: set(mpmath_libmp_names) - set(mpmath_libmp_names_130)
Out[9]: {'MPQ', 'giant_steps'}

Probably, it worth restoring math2 as a deprecated alias for libfp. Other removed names in the core namespace - related to testing.

I don't think it worth restoring names in the libmp: these aren't used in sympy or in other projects (as it seems from my GH search).

I did also tests for mpmath.libmp imports:

$ python ~/find-libmp.py .  # sympy 1.12
mpmath.libmp imports:
['ComplexResult', 'MPZ', 'MPZ_ONE', 'NoConvergence', '_infs_nan', '_normalize', 'backend', 'bin_to_radix', 'bitcount', 'catalan_fixed', 'dps_to_prec', 'fhalf', 'finf', 'fnan', 'fninf', 'fnone', 'fone', 'from_float', 'from_int', 'from_man_exp', 'from_rational', 'from_str', 'fzero', 'giant_steps', 'ifac', 'ifib', 'int_types', 'isqrt', 'libhyper', 'libintmath', 'libmpc', 'libmpf', 'mpc_abs', 'mpc_exp', 'mpc_pow', 'mpc_pow_int', 'mpc_pow_mpf', 'mpc_sqrt', 'mpf_abs', 'mpf_add', 'mpf_atan', 'mpf_atan2', 'mpf_ceil', 'mpf_cmp', 'mpf_cos', 'mpf_cosh_sinh', 'mpf_div', 'mpf_e', 'mpf_eq', 'mpf_exp', 'mpf_floor', 'mpf_ge', 'mpf_gt', 'mpf_le', 'mpf_log', 'mpf_lt', 'mpf_mod', 'mpf_mul', 'mpf_neg', 'mpf_pi', 'mpf_pow', 'mpf_pow_int', 'mpf_shift', 'mpf_sin', 'mpf_sqrt', 'mpf_sub', 'normalize', 'numeral', 'phi_fixed', 'prec_to_dps', 'repr_dps', 'round_nearest', 'sqrtrem', 'to_float', 'to_int', 'to_pickable', 'to_rational', 'to_str']
$ python ~/find-libmp.py .  # diofant v0.14
missing in latest namespace:
['_infs_nan', '_normalize']
mpmath.libmp imports:
['ComplexResult', 'MPZ', 'MPZ_ONE', 'NoConvergence', '_infs_nan', 'backend', 'bitcount', 'dps_to_prec', 'fhalf', 'finf', 'fnan', 'fninf', 'fnone', 'fone', 'from_Decimal', 'from_float', 'from_int', 'from_man_exp', 'from_rational', 'from_str', 'fzero', 'gammazeta', 'giant_steps', 'ifac', 'ifib', 'int_types', 'libintmath', 'libmpc', 'libmpf', 'mpc_abs', 'mpc_exp', 'mpc_pow', 'mpc_pow_int', 'mpc_pow_mpf', 'mpc_sqrt', 'mpf_abs', 'mpf_add', 'mpf_atan', 'mpf_atan2', 'mpf_bernoulli', 'mpf_ceil', 'mpf_cmp', 'mpf_cos', 'mpf_cosh_sinh', 'mpf_div', 'mpf_e', 'mpf_eq', 'mpf_exp', 'mpf_floor', 'mpf_ge', 'mpf_gt', 'mpf_le', 'mpf_log', 'mpf_lt', 'mpf_mod', 'mpf_mul', 'mpf_neg', 'mpf_pi', 'mpf_pow', 'mpf_pow_int', 'mpf_shift', 'mpf_sin', 'mpf_sqrt', 'mpf_sub', 'normalize', 'pi_fixed', 'prec_to_dps', 'repr_dps', 'round_nearest', 'sqrtrem', 'to_float', 'to_int', 'to_rational', 'to_str']
missing in latest namespace:
['_infs_nan']
$ python ~/find-libmp.py .  # diofant latest
mpmath.libmp imports:
['ComplexResult', 'MPZ', 'MPZ_ONE', 'NoConvergence', 'dps_to_prec', 'fhalf', 'finf', 'fnan', 'fninf', 'fnone', 'fone', 'from_float', 'from_int', 'from_man_exp', 'from_rational', 'from_str', 'fzero', 'giant_steps', 'ifac', 'ifib', 'int_types', 'mpc_abs', 'mpc_exp', 'mpc_pow', 'mpc_pow_int', 'mpc_pow_mpf', 'mpc_sqrt', 'mpf_abs', 'mpf_add', 'mpf_atan', 'mpf_atan2', 'mpf_bernoulli', 'mpf_ceil', 'mpf_cmp', 'mpf_cos', 'mpf_div', 'mpf_eq', 'mpf_exp', 'mpf_floor', 'mpf_ge', 'mpf_gt', 'mpf_le', 'mpf_log', 'mpf_lt', 'mpf_mod', 'mpf_mul', 'mpf_neg', 'mpf_pi', 'mpf_pow', 'mpf_pow_int', 'mpf_shift', 'mpf_sin', 'mpf_sqrt', 'mpf_sub', 'normalize', 'prec_to_dps', 'repr_dps', 'round_nearest', 'sqrtrem', 'to_float', 'to_int', 'to_rational', 'to_str']
missing in latest namespace:
[]

I doubt, that importing from libmp's submodules is a good idea, especially private names. That's a thing I would like to deprecate. What do you think? Unfortunately, I doubt we can issue a warning in runtime in this case.

import ast
import os
import sys

import mpmath


sys.setrecursionlimit(10000)  # for sympy

libmp_imports = set()


class ImportVisitor(ast.NodeVisitor):
    def __init__(self):
        self.names = set()

    def visit_ImportFrom(self, node):
        # from mpmath.libmp import foo, bar
        if node.module == 'mpmath.libmp':
            for n in node.names:
                libmp_imports.add(n.name)
        if node.module and node.module.startswith('mpmath.libmp.'):
            # from mpmath.libmp.libmpf import foo
            for n in node.names:
                libmp_imports.add(n.name)
            m = node.module
            m = m.replace('mpmath.libmp.', '')
            assert '.' not in m
            libmp_imports.add(m)
        # from mpmath import foo, bar, libmp
        if node.module == 'mpmath':
            for n in node.names:
                if n.name == 'libmp':
                    self.names.add(n.asname if n.asname else n.name)

    def visit_Import(self, node):
        # import mpmath.libmp
        # import mpmath.libmp as libm
        # import mpmath.libmp.libmpf
        # import mpmath.libmp.backend as b
        for n in node.names:
            if n.name == 'mpmath.libmp':
                self.names.add(n.asname if n.asname else n.name)
            elif n.name.startswith('mpmath.libmp.'):
                self.names.add(n.asname if n.asname else n.name)

    def visit_Attribute(self, node):
        v = node.value
        if isinstance(v, ast.Name):
            if v.id in self.names:
                a = node.attr
                assert a in dir(mpmath.libmp)
                libmp_imports.add(a)
        if isinstance(v, ast.Attribute):
            if isinstance(v.value, ast.Name):
                if v.value.id == 'mpmath' and v.attr == 'libmp':
                    a = node.attr
                    assert a in dir(mpmath.libmp)
                    libmp_imports.add(a)


for root, sdirs, files in os.walk(sys.argv[1]):
    for fname in files:
        if fname.endswith('.py'):
            with open(os.path.join(root, fname), 'r') as file:
                source = file.read()
            tree = ast.parse(source)
            ImportVisitor().visit(tree)

print("mpmath.libmp imports:")
print(sorted(libmp_imports))

print("missing in latest namespace:")
print(sorted(_ for _ in libmp_imports if _ not in dir(mpmath.libmp)))

What do you think?

I think that sympy reaches into mpmath's internals too much and it is inevitable that if mpmath is actively developed then future versions of mpmath are not going to be compatible with current versions of sympy. We can coordinate this to keep sympy up to date with latest mpmath changes but we need sympy releases to have a version cap like mpmath < 1.4.

The first thing to consider though before deprecating or removing things is what will ultimately be the public API that should be used instead. Currently the awkward thing about mpmath's documented API is that it is all based on contexts and the idea of a global precision. What is nice about the libmp functions is that they are pure functions that take an explicit precision argument.

We can coordinate this to keep sympy up to date with latest mpmath changes

As a first step for the sympy master, I suggest something like diofant/diofant@1dd72fb: all required libmp's imports (including the giant_steps()) could be imported from the mpmath.libmp namespace. (from mpmath.libmp import foo, bar or import mpmath.libmp as libmp)

that it is all based on contexts and the idea of a global precision.

And contexts serve both to hold some attributes (like precision or rounding mode) and as namespaces for functions (namespaces, where we actually have same set of mathematical functions for all contexts).

What is nice about the libmp functions is that they are pure functions that take an explicit precision argument.

Yet the namespace of libmp is huge:

>>> len(libmp_names)  # not prefixed by '_'
320

While e.g. SymPy (v1.12) is using:

>>> len(sympy_names)  # from above script, maybe I miss something
78

Not all functions have common naming pattern (e.g. mpf_foo), common arguments convention (say, function arguments, precision, optional rounding mode), have different defaults for prec/rnd, etc...

I think we could start with something, that fits SymPy's needs, with few functions being deprecated (e.g. to/from_pickable() or bitcount()) and treat the rest of libmp as private stuff.

Ok, let me know when I can release a new alpha.

The latest sympy prerelease (sympy 1.12.1rc1) pins mpmath < 1.4.0 so it should be fine. Go ahead and put out prereleases if you want to get them out.

Thanks.

v1.4.0a1 released, I hope sympy/sympy#26273 is fixed.

It should be fine as far as prereleases go. Thanks Sergey.