Skip to content

Conversation

jakelishman
Copy link
Member

Summary

This fixes several places in quantum_info that will be affected by the casting-rule changes coming in Numpy 2.01. There are several other places where the casting rules will change the output types of certain arrays due to the removal of Python-scalar value-dependent casting, but those do not appear to have an observable effect on our behaviour, so I have left them.

It is also becoming invalid to set a scalar entry in a Numpy array with an array-like with a non-zero number of dimensions, even if the sequence is of length 1.

Details and comments

For reference, I found these by running the test suite with Numpy 1.26.0 and the environment variable NPY_PROMOTION_STATE=weak set. I also ran with that set to weak_and_warn to see if there were other places that looked suspicious, but that is (predictably) very noisy, and none of the related changes jumped out at me as particularly problematic.

Running with Numpy 1.26.0 (on macOS) runs afoul of #10305, so it's possible I missed a couple of things, but I don't think so. Unfortunately, a bug in Numpy 1.24.4 means that np.complex128(1j) * opflow.Z (as a MWE) segfaults the Python interpreter when running in NPY_PROMOTION_STATE=weak, so it's pretty hard work to disentangle the test suite from that well enough to use the old Numpy version with the new casting rules to find errors (plus they've been changing a little over time).

Footnotes

  1. https://numpy.org/neps/nep-0050-scalar-promotion.html

This fixes several places in `quantum_info` that will be affected by the
casting-rule changes coming in Numpy 2.0[^1].  There are several other
places where the casting rules will change the output types of certain
arrays due to the removal of Python-scalar value-dependent casting, but
those do not appear to have an observable effect on our behaviour, so I
have left them.

It is also becoming invalid to set a scalar entry in a Numpy array with
an array-like with a non-zero number of dimensions, even if the sequence
is of length 1.

[^1]: https://numpy.org/neps/nep-0050-scalar-promotion.html
@jakelishman jakelishman added type: qa Issues and PRs that relate to testing and code quality mod: quantum info Related to the Quantum Info module (States & Operators) Changelog: None Do not include in changelog labels Sep 25, 2023
@jakelishman jakelishman added this to the 0.45.0 milestone Sep 25, 2023
@jakelishman jakelishman requested review from ikkoham and a team as code owners September 25, 2023 13:27
@qiskit-bot
Copy link
Collaborator

One or more of the the following people are requested to review this:

  • @Qiskit/terra-core
  • @ikkoham

Copy link
Contributor

@ikkoham ikkoham left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. I didn't know the NumPy 2. Note that pauli_table.py will be removed soon, but this PR is fine.

@jakelishman
Copy link
Member Author

jakelishman commented Sep 29, 2023

Thanks Hamamura-san. Yeah, I'm hoping that PauliTable will be gone from main before NumPy 2 is actually available too, but the 0.45 and then 0.46 releases of Qiskit will still be supported after the release of NumPy 2, so we'll need them to continue working with it.

If it helps - we're tracking our complete progress on NumPy 2 support in #10889.

@jakelishman jakelishman added this pull request to the merge queue Sep 29, 2023
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Sep 29, 2023
@jakelishman jakelishman added this pull request to the merge queue Sep 29, 2023
Merged via the queue into Qiskit:main with commit 92d7080 Sep 29, 2023
@jakelishman jakelishman deleted the numpy/casting branch September 29, 2023 13:31
rupeshknn pushed a commit to rupeshknn/qiskit that referenced this pull request Oct 9, 2023
This fixes several places in `quantum_info` that will be affected by the
casting-rule changes coming in Numpy 2.0[^1].  There are several other
places where the casting rules will change the output types of certain
arrays due to the removal of Python-scalar value-dependent casting, but
those do not appear to have an observable effect on our behaviour, so I
have left them.

It is also becoming invalid to set a scalar entry in a Numpy array with
an array-like with a non-zero number of dimensions, even if the sequence
is of length 1.

[^1]: https://numpy.org/neps/nep-0050-scalar-promotion.html
jakelishman added a commit to jakelishman/qiskit-terra that referenced this pull request Mar 12, 2024
This commit brings the Qiskit test suite to a passing state (with all
optionals installed) with Numpy 2.0.0b1, building on previous commits
that handled much of the rest of the changing requirements:

- Qiskitgh-10890
- Qiskitgh-10891
- Qiskitgh-10892
- Qiskitgh-10897
- Qiskitgh-11023

Notably, this commit did not actually require a rebuild of Qiskit,
despite us compiling against Numpy; it seems to happen that the C API
stuff we use via `rust-numpy` (which loads the Numpy C extensions
dynamically during module initialisation) hasn't changed.
jakelishman added a commit to jakelishman/qiskit-terra that referenced this pull request Apr 15, 2024
This commit brings the Qiskit test suite to a passing state (with all
optionals installed) with Numpy 2.0.0b1, building on previous commits
that handled much of the rest of the changing requirements:

- Qiskitgh-10890
- Qiskitgh-10891
- Qiskitgh-10892
- Qiskitgh-10897
- Qiskitgh-11023

Notably, this commit did not actually require a rebuild of Qiskit,
despite us compiling against Numpy; it seems to happen that the C API
stuff we use via `rust-numpy` (which loads the Numpy C extensions
dynamically during module initialisation) hasn't changed.

The main changes are:

- adapting to the changed `copy=None` and `copy=False` semantics in
  `array` and `asarray`.
- making sure all our implementers of `__array__` accept both `dtype`
  and `copy` arguments.

Co-authored-by: Lev S. Bishop <18673315+levbishop@users.noreply.github.com>
jakelishman added a commit to jakelishman/qiskit-terra that referenced this pull request Apr 16, 2024
This commit brings the Qiskit test suite to a passing state (with all
optionals installed) with Numpy 2.0.0b1, building on previous commits
that handled much of the rest of the changing requirements:

- Qiskitgh-10890
- Qiskitgh-10891
- Qiskitgh-10892
- Qiskitgh-10897
- Qiskitgh-11023

Notably, this commit did not actually require a rebuild of Qiskit,
despite us compiling against Numpy; it seems to happen that the C API
stuff we use via `rust-numpy` (which loads the Numpy C extensions
dynamically during module initialisation) hasn't changed.

The main changes are:

- adapting to the changed `copy=None` and `copy=False` semantics in
  `array` and `asarray`.
- making sure all our implementers of `__array__` accept both `dtype`
  and `copy` arguments.

Co-authored-by: Lev S. Bishop <18673315+levbishop@users.noreply.github.com>
github-merge-queue bot pushed a commit that referenced this pull request Apr 25, 2024
* Finalise support for Numpy 2.0

This commit brings the Qiskit test suite to a passing state (with all
optionals installed) with Numpy 2.0.0b1, building on previous commits
that handled much of the rest of the changing requirements:

- gh-10890
- gh-10891
- gh-10892
- gh-10897
- gh-11023

Notably, this commit did not actually require a rebuild of Qiskit,
despite us compiling against Numpy; it seems to happen that the C API
stuff we use via `rust-numpy` (which loads the Numpy C extensions
dynamically during module initialisation) hasn't changed.

The main changes are:

- adapting to the changed `copy=None` and `copy=False` semantics in
  `array` and `asarray`.
- making sure all our implementers of `__array__` accept both `dtype`
  and `copy` arguments.

Co-authored-by: Lev S. Bishop <18673315+levbishop@users.noreply.github.com>

* Update `__array__` methods for Numpy 2.0 compatibility

As of Numpy 2.0, implementers of `__array__` are expected and required
to have a signature

    def __array__(self, dtype=None, copy=None): ...

In Numpys before 2.0, the `copy` argument will never be passed, and the
expected signature was

    def __array__(self, dtype=None): ...

Because of this, we have latitude to set `copy` in our implementations
to anything we like if we're running against Numpy 1.x, but we should
default to `copy=None` if we're running against Numpy 2.0.

The semantics of the `copy` argument to `np.array` changed in Numpy 2.0.
Now, `copy=False` means "raise a `ValueError` if a copy is required" and
`copy=None` means "copy only if required".  In Numpy 1.x, `copy=False`
meant "copy only if required".  In _both_ Numpy 1.x and 2.0,
`ndarray.astype` takes a `copy` argument, and in both, `copy=False`
means "copy only if required".  In Numpy 2.0 only, `np.asarray` gained a
`copy` argument with the same semantics as the `np.array` copy argument
from Numpy 2.0.

Further, the semantics of the `__array__` method changed in Numpy 2.0,
particularly around copying.  Now, Numpy will assume that it can pass
`copy=True` and the implementer will handle this.  If `copy=False` is
given and a copy or calculation is required, then the implementer is
required to raise `ValueError`.  We have a few places where the
`__array__` method may (or always does) calculate the array, so in all
these, we must forbid `copy=False`.

With all this in mind: this PR sets up all our implementers of
`__array__` to either default to `copy=None` if they will never actually
need to _use_ the `copy` argument within themselves (except perhaps to
test if it was set by Numpy 2.0 to `False`, as Numpy 1.x will never set
it), or to a compatibility shim `_numpy_compat.COPY_ONLY_IF_NEEDED` if
they do naturally want to use it with those semantics.  The pattern

    def __array__(self, dtype=None, copy=_numpy_compat.COPY_ONLY_IF_NEEDED):
        dtype = self._array.dtype if dtype is None else dtype
        return np.array(self._array, dtype=dtype, copy=copy)

using `array` instead of `asarray` lets us achieve all the desired
behaviour between the interactions of `dtype` and `copy` in a way that
is compatible with both Numpy 1.x and 2.x.

* fixing numerical issues on mac-arm

* Change error to match Numpy

---------

Co-authored-by: Lev S. Bishop <18673315+levbishop@users.noreply.github.com>
Co-authored-by: Sebastian Brandhofer <148463728+sbrandhsn@users.noreply.github.com>
github-merge-queue bot pushed a commit that referenced this pull request Jun 18, 2024
* Finalise support for Numpy 2.0

This commit brings the Qiskit test suite to a passing state (with all
optionals installed) with Numpy 2.0.0b1, building on previous commits
that handled much of the rest of the changing requirements:

- gh-10890
- gh-10891
- gh-10892
- gh-10897
- gh-11023

Notably, this commit did not actually require a rebuild of Qiskit,
despite us compiling against Numpy; it seems to happen that the C API
stuff we use via `rust-numpy` (which loads the Numpy C extensions
dynamically during module initialisation) hasn't changed.

The main changes are:

- adapting to the changed `copy=None` and `copy=False` semantics in
  `array` and `asarray`.
- making sure all our implementers of `__array__` accept both `dtype`
  and `copy` arguments.

Co-authored-by: Lev S. Bishop <18673315+levbishop@users.noreply.github.com>

* Update `__array__` methods for Numpy 2.0 compatibility

As of Numpy 2.0, implementers of `__array__` are expected and required
to have a signature

    def __array__(self, dtype=None, copy=None): ...

In Numpys before 2.0, the `copy` argument will never be passed, and the
expected signature was

    def __array__(self, dtype=None): ...

Because of this, we have latitude to set `copy` in our implementations
to anything we like if we're running against Numpy 1.x, but we should
default to `copy=None` if we're running against Numpy 2.0.

The semantics of the `copy` argument to `np.array` changed in Numpy 2.0.
Now, `copy=False` means "raise a `ValueError` if a copy is required" and
`copy=None` means "copy only if required".  In Numpy 1.x, `copy=False`
meant "copy only if required".  In _both_ Numpy 1.x and 2.0,
`ndarray.astype` takes a `copy` argument, and in both, `copy=False`
means "copy only if required".  In Numpy 2.0 only, `np.asarray` gained a
`copy` argument with the same semantics as the `np.array` copy argument
from Numpy 2.0.

Further, the semantics of the `__array__` method changed in Numpy 2.0,
particularly around copying.  Now, Numpy will assume that it can pass
`copy=True` and the implementer will handle this.  If `copy=False` is
given and a copy or calculation is required, then the implementer is
required to raise `ValueError`.  We have a few places where the
`__array__` method may (or always does) calculate the array, so in all
these, we must forbid `copy=False`.

With all this in mind: this PR sets up all our implementers of
`__array__` to either default to `copy=None` if they will never actually
need to _use_ the `copy` argument within themselves (except perhaps to
test if it was set by Numpy 2.0 to `False`, as Numpy 1.x will never set
it), or to a compatibility shim `_numpy_compat.COPY_ONLY_IF_NEEDED` if
they do naturally want to use it with those semantics.  The pattern

    def __array__(self, dtype=None, copy=_numpy_compat.COPY_ONLY_IF_NEEDED):
        dtype = self._array.dtype if dtype is None else dtype
        return np.array(self._array, dtype=dtype, copy=copy)

using `array` instead of `asarray` lets us achieve all the desired
behaviour between the interactions of `dtype` and `copy` in a way that
is compatible with both Numpy 1.x and 2.x.

---------

Co-authored-by: Lev S. Bishop <18673315+levbishop@users.noreply.github.com>
Co-authored-by: Raynel Sanchez <87539502+raynelfss@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Changelog: None Do not include in changelog mod: quantum info Related to the Quantum Info module (States & Operators) type: qa Issues and PRs that relate to testing and code quality
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants