Readout mitigator classes #6485

ShellyGarion · 2021-05-30T09:32:09Z

Summary

#6484

Add Readout Mitigator classes bases on the current code in ignis.

Details and comments

BaseReadoutMitgator
CorrelatedReadoutMitigator
LocalReadoutMitigator
Update qiskit.result classes only if necessary (Counts, ProbDitribution, QuasiDistribution)
Add tests

co-authored by @gadial

nonhermitian · 2021-06-29T11:26:23Z

qiskit/result/distributions/quasi.py

@@ -123,3 +123,11 @@ def hex_probabilities(self):
                format ``"0x1a"``
        """
        return {hex(key): value for key, value in self.items()}
+
+    def stddev(self, shots):


This does not work here. The runtime mitigation method will soon return the mitigation_overhead that, together with the number of shots, gives an estimate (usually exact) of this value.

Thank you @nonhermitian for the feedback! So where should I add the stddev function?
Should I add it to the ReadoutMitigator class?

It makes sense but needs to be computed differently.

nonhermitian · 2021-07-14T10:15:07Z

Here is the basic way M3 works:

# Grab a mitigator instance and target a given backend
mit = mthree.M3Mitigation(backend)

# calibrate over the desired number of qubits (can be renamed to something more generic)
mit.tensored_cals_from_system(qubits)

# apply correction
quasi_probs = mit.apply_correction(real_counts, qubits=measured_qubits)

nonhermitian · 2021-07-16T12:57:57Z

The M3 code is up here #6748. So thoughts on this PR:

I think the mitigation classes should be simpler. Namely, the bare minimum one needs is:

mit = Mitigator(backend)
mit.calibrations_from_system(qubits)
mitigated_values = mit.apply_correction(raw_counts, qubits, **kwargs)

The mitigator itself should in charge of executing the calibration circuits because at some point the backends should just compute this info on a regular basis and cache it at which point calibrations_from_system might just be an API call, or another method like calibrations_from_remote may exist.

Things like plotting the A-matrix should not be here, just simply return an array for the given matrix.

Returning the mitigation matrix should not be in the base class as computing it is optional depending on the solution method.

I am also not a big fan of having expectation_value attached to the mitigator as it is not really a property of the mitigator but rather of the return values and additional info attached to them. I do however understand that the CTMP works this was, but, in my opinion, that should be refactored to return quasi-probs like the others do.

qiskit/quantum_info/mitigation/base_readout_mitigator.py

This commit migrates the measurement mitigation code from qiskit-ignis into qiskit-terra for use with the QuantumInstance class. The QuantumInstance class's usage of the measurement mitigation from ignis is the one thing blocking us from deprecating qiskit-ignis completely. By embedding the code the quantum instance depends on inside qiskit.utils users of QuantumInstance (and therefore qiskit.algorithms) can construct and use measurement mitigation in it's current form. The use of this migrated module is only supported for use with the QuantumInstance class and is explicitly documented as internal/private except for how it gets used by the QuantumInstance. There is ongoing work to create a standardized mitigation API in Qiskit#6485 and Qiskit#6748, this does not preclude that work, but we should adapt this as part of those efforts to use the standardized interface. Ideally this would have been made a private interface and not exposed it as user facing (in deference to the standardization effort), but unfortunately the QuantumInstance expects classes of these classes as it's public interface for selecting a mitigation technique which means users need to be able to use the classes. However, as only the classes are public interfaces we can adapt this as we come up with a standardized mitigation interface and rewrite the internals of this and how the QuantumInstance leverages mitigators to use the new interface. A good follow-up here would be to adapt the mitigator selection kwarg to deprecate the use of classes and then we can make things explicitly private in the migrated code and wait for Qiskit#6495 and Qiskit#6748 to be ready for our user facing API. I opted to not include that in this PR to minimize changes to just what we migrated from ignis and update usage of old ignis classes to rely on the migrated version.

ShellyGarion · 2021-08-26T10:04:03Z

@nonhermitian @chriseclectic @mtreinish - I updated this PR following your comments and discussions.
Now, the BaseReadoutMitgator class contains only 2 methods: quasi_probabilities and expectation_value.
I also didn't change the qiskit.result classes, except of adding the methods num_qubits and shots to Counts, which I think are useful (but can be removed if you don't like them).
Could you please review the code and let me know if you agree with the design of the base class (which should also be relevant for the M3Mitigation)?

chriseclectic

Some initial comments

qiskit/quantum_info/mitigation/base_readout_mitigator.py

qiskit/quantum_info/mitigation/complete_readout_mitigator.py

chriseclectic · 2021-08-31T22:32:46Z

qiskit/quantum_info/mitigation/base_readout_mitigator.py

+    _HAS_MATPLOTLIB = False
+
+
+class BaseReadoutMitigator(ABC):


We might want to include something in the base class for what physical qubits it is defined on.

could you please elaborate?

qiskit/quantum_info/mitigation/complete_readout_mitigator.py

chriseclectic · 2021-08-31T22:34:33Z

qiskit/quantum_info/mitigation/complete_readout_mitigator.py

+
+        # Compute variance
+        sq_expval = (coeffs ** 2).dot(probs)
+        variance = (sq_expval - expval ** 2) / shots


I think this method under-estimates error since it doesn't take into account the mitigation overhead from the CTMP paper. In that paper they use an upper bound for the variance based on the A-matrix which comes from uncertainty in the estimates of the true error probabilities.

Does the CTMP mitigation overhead estimator work for all mitigation methods? If so - perhaps we should add mitigation_overhead to the base class?

I took the code for the complete mitigator method from the previous code in ignis - does it mean that the code there is not accurate enough?
https://github.com/Qiskit/qiskit-ignis/blob/f0728b6b6785b68693a0d31c54c51f9826ddde34/qiskit/ignis/mitigation/expval/utils.py#L244-#L246

I see that there is another function stddev_upper_bound that relies on gamma:
https://github.com/Qiskit/qiskit-ignis/blob/f0728b6b6785b68693a0d31c54c51f9826ddde34/qiskit/ignis/mitigation/expval/base_meas_mitigator.py#L155

so perhaps we could return only gamma and not the entire expectation_value ?

I added the functions _stddev_upper_bound and _gamma

qiskit/result/counts.py

ShellyGarion · 2021-09-05T07:49:49Z

qiskit/quantum_info/mitigation/base_readout_mitigator.py

+
+    def quasi_probabilities(
+        self, data: Counts, qubits: Iterable[int] = None, shots: Optional[int] = None
+    ) -> (QuasiDistribution, Dict[str, float]):


@chriseclectic - Is it better to return the standard deviation also as a QuasiDistribution object (instead of a Dict)?

gadial · 2021-09-19T15:23:34Z

qiskit/quantum_info/mitigation/complete_readout_mitigator.py

+        else:
+            qubits = tuple(sorted(qubits))
+
+        # Check for cached mitigation matrix


Why not use a built-in Python caching mechanism such as the @lru_cache decorator?

Should discuss where we can store the cached mitigation matrix(es)

* Migrate measure mitigation from ignis for QuantumInstance This commit migrates the measurement mitigation code from qiskit-ignis into qiskit-terra for use with the QuantumInstance class. The QuantumInstance class's usage of the measurement mitigation from ignis is the one thing blocking us from deprecating qiskit-ignis completely. By embedding the code the quantum instance depends on inside qiskit.utils users of QuantumInstance (and therefore qiskit.algorithms) can construct and use measurement mitigation in it's current form. The use of this migrated module is only supported for use with the QuantumInstance class and is explicitly documented as internal/private except for how it gets used by the QuantumInstance. There is ongoing work to create a standardized mitigation API in #6485 and #6748, this does not preclude that work, but we should adapt this as part of those efforts to use the standardized interface. Ideally this would have been made a private interface and not exposed it as user facing (in deference to the standardization effort), but unfortunately the QuantumInstance expects classes of these classes as it's public interface for selecting a mitigation technique which means users need to be able to use the classes. However, as only the classes are public interfaces we can adapt this as we come up with a standardized mitigation interface and rewrite the internals of this and how the QuantumInstance leverages mitigators to use the new interface. A good follow-up here would be to adapt the mitigator selection kwarg to deprecate the use of classes and then we can make things explicitly private in the migrated code and wait for #6495 and #6748 to be ready for our user facing API. I opted to not include that in this PR to minimize changes to just what we migrated from ignis and update usage of old ignis classes to rely on the migrated version. * Update releasenotes/notes/ignis-mitigators-70492690cbcf99ca.yaml Co-authored-by: Steve Wood <40241007+woodsp-ibm@users.noreply.github.com> * Finish release notes * Move API stability warning to the top * Apply suggestions from code review Co-authored-by: Jake Lishman <jake@binhbar.com> * Remove unecessary test subclassing * Fix skip logic in algorithm mitigation tests * Fix exception handling on missing ignis in algorithms * Assert deprecation message for ignis use mentions ignis * Make filters module properly private * Fix lint Co-authored-by: Steve Wood <40241007+woodsp-ibm@users.noreply.github.com> Co-authored-by: Jake Lishman <jake@binhbar.com>

test/python/quantum_info/test_mitigators.py

ShellyGarion · 2021-10-06T06:24:24Z

should move the code out of quantum_info to qiskit/mitigation
should discuss how it integrates with Expectation value re-design Expectation value re-design #6864

Attempting to fix cyclic dependancy error

test/python/result/generate_data.py

qiskit/result/mitigation/local_readout_mitigator.py

releasenotes/notes/readout-mitigation-classes-2ef175e232d791ae.yaml

qiskit/result/counts.py

Attempting to fix cyclic dependancy error

test/python/result/generate_data.py

* Migrate measure mitigation from ignis for QuantumInstance This commit migrates the measurement mitigation code from qiskit-ignis into qiskit-terra for use with the QuantumInstance class. The QuantumInstance class's usage of the measurement mitigation from ignis is the one thing blocking us from deprecating qiskit-ignis completely. By embedding the code the quantum instance depends on inside qiskit.utils users of QuantumInstance (and therefore qiskit.algorithms) can construct and use measurement mitigation in it's current form. The use of this migrated module is only supported for use with the QuantumInstance class and is explicitly documented as internal/private except for how it gets used by the QuantumInstance. There is ongoing work to create a standardized mitigation API in Qiskit#6485 and Qiskit#6748, this does not preclude that work, but we should adapt this as part of those efforts to use the standardized interface. Ideally this would have been made a private interface and not exposed it as user facing (in deference to the standardization effort), but unfortunately the QuantumInstance expects classes of these classes as it's public interface for selecting a mitigation technique which means users need to be able to use the classes. However, as only the classes are public interfaces we can adapt this as we come up with a standardized mitigation interface and rewrite the internals of this and how the QuantumInstance leverages mitigators to use the new interface. A good follow-up here would be to adapt the mitigator selection kwarg to deprecate the use of classes and then we can make things explicitly private in the migrated code and wait for Qiskit#6495 and Qiskit#6748 to be ready for our user facing API. I opted to not include that in this PR to minimize changes to just what we migrated from ignis and update usage of old ignis classes to rely on the migrated version. * Update releasenotes/notes/ignis-mitigators-70492690cbcf99ca.yaml Co-authored-by: Steve Wood <40241007+woodsp-ibm@users.noreply.github.com> * Finish release notes * Move API stability warning to the top * Apply suggestions from code review Co-authored-by: Jake Lishman <jake@binhbar.com> * Remove unecessary test subclassing * Fix skip logic in algorithm mitigation tests * Fix exception handling on missing ignis in algorithms * Assert deprecation message for ignis use mentions ignis * Make filters module properly private * Fix lint Co-authored-by: Steve Wood <40241007+woodsp-ibm@users.noreply.github.com> Co-authored-by: Jake Lishman <jake@binhbar.com>

…kit#6867) * Migrate measure mitigation from ignis for QuantumInstance This commit migrates the measurement mitigation code from qiskit-ignis into qiskit-terra for use with the QuantumInstance class. The QuantumInstance class's usage of the measurement mitigation from ignis is the one thing blocking us from deprecating qiskit-ignis completely. By embedding the code the quantum instance depends on inside qiskit.utils users of QuantumInstance (and therefore qiskit.algorithms) can construct and use measurement mitigation in it's current form. The use of this migrated module is only supported for use with the QuantumInstance class and is explicitly documented as internal/private except for how it gets used by the QuantumInstance. There is ongoing work to create a standardized mitigation API in Qiskit/qiskit#6485 and Qiskit/qiskit#6748, this does not preclude that work, but we should adapt this as part of those efforts to use the standardized interface. Ideally this would have been made a private interface and not exposed it as user facing (in deference to the standardization effort), but unfortunately the QuantumInstance expects classes of these classes as it's public interface for selecting a mitigation technique which means users need to be able to use the classes. However, as only the classes are public interfaces we can adapt this as we come up with a standardized mitigation interface and rewrite the internals of this and how the QuantumInstance leverages mitigators to use the new interface. A good follow-up here would be to adapt the mitigator selection kwarg to deprecate the use of classes and then we can make things explicitly private in the migrated code and wait for Qiskit/qiskit#6495 and Qiskit/qiskit#6748 to be ready for our user facing API. I opted to not include that in this PR to minimize changes to just what we migrated from ignis and update usage of old ignis classes to rely on the migrated version. * Update releasenotes/notes/ignis-mitigators-70492690cbcf99ca.yaml Co-authored-by: Steve Wood <40241007+woodsp-ibm@users.noreply.github.com> * Finish release notes * Move API stability warning to the top * Apply suggestions from code review Co-authored-by: Jake Lishman <jake@binhbar.com> * Remove unecessary test subclassing * Fix skip logic in algorithm mitigation tests * Fix exception handling on missing ignis in algorithms * Assert deprecation message for ignis use mentions ignis * Make filters module properly private * Fix lint Co-authored-by: Steve Wood <40241007+woodsp-ibm@users.noreply.github.com> Co-authored-by: Jake Lishman <jake@binhbar.com>

ShellyGarion requested review from chriseclectic and a team as code owners May 30, 2021 09:32

ShellyGarion added mod: quantum info Related to the Quantum Info module (States & Operators) Changelog: New Feature Include in the "Added" section of the changelog labels May 30, 2021

ShellyGarion changed the title ~~Readout mitigator classes~~ [WIP] Readout mitigator classes May 30, 2021

ShellyGarion mentioned this pull request Jun 15, 2021

Add Support to TensoredMeasFitter in QuantumInstance #6570

Merged

nonhermitian reviewed Jun 29, 2021

View reviewed changes

nonhermitian reviewed Jul 16, 2021

View reviewed changes

qiskit/quantum_info/mitigation/base_readout_mitigator.py Outdated Show resolved Hide resolved

mtreinish mentioned this pull request Aug 4, 2021

Migrate measure mitigation from ignis for QuantumInstance #6867

Merged

chriseclectic added this to the 0.19 milestone Aug 24, 2021

ShellyGarion requested review from nonhermitian and mtreinish August 26, 2021 09:13

chriseclectic reviewed Aug 31, 2021

View reviewed changes

ShellyGarion removed request for nonhermitian and mtreinish September 5, 2021 07:24

ShellyGarion commented Sep 5, 2021

View reviewed changes

gadial reviewed Sep 19, 2021

View reviewed changes

ShellyGarion commented Oct 5, 2021

View reviewed changes

test/python/quantum_info/test_mitigators.py Outdated Show resolved Hide resolved

ShellyGarion force-pushed the readout_mitigator branch 3 times, most recently from ac638fd to 6549830 Compare October 11, 2021 11:48

ShellyGarion and others added 6 commits December 1, 2021 08:17

update import in mitigation files

053b7ab

Attempting to fix cyclic dependancy error

2263b9e

Merge pull request #23 from gadial/readout_mitigator

aa3e5d0

Attempting to fix cyclic dependancy error

update import

3ceefe9

update import

a4384b5

Merge branch 'main' into readout_mitigator

af92ca8

mtreinish requested changes Dec 1, 2021

View reviewed changes

gadial and others added 7 commits December 1, 2021 17:22

Attempting to fix cyclic dependancy error

acc648b

Merge pull request #25 from gadial/readout_mitigator

cc46d75

Attempting to fix cyclic dependancy error

style

2963d90

updates following review comments

cf2725b

remove generate data functions

f26d49a

add more details in classes docstrings

f35e2e4

add more details in release notes

baa8e50

ShellyGarion requested review from chriseclectic and mtreinish December 2, 2021 07:42

chriseclectic reviewed Dec 2, 2021

View reviewed changes

test/python/result/generate_data.py Outdated Show resolved Hide resolved

move test_data to test_mitigators

e37bd3d

ShellyGarion requested a review from chriseclectic December 2, 2021 17:29

chriseclectic added 2 commits December 2, 2021 13:46

Update release note

ca9edcb

Merge branch 'main' into readout_mitigator

163f3cd

mtreinish approved these changes Dec 2, 2021

View reviewed changes

mtreinish added the automerge label Dec 2, 2021

chriseclectic approved these changes Dec 2, 2021

View reviewed changes

mergify bot merged commit ae55844 into Qiskit:main Dec 2, 2021

ShellyGarion mentioned this pull request Dec 3, 2021

Readout Mitigator Classes #6484

Closed

peendebak mentioned this pull request Nov 20, 2022

Fix shots argument of local_readout_mitigator #9155

Merged

Readout mitigator classes #6485

Readout mitigator classes #6485

Uh oh!

Conversation

ShellyGarion commented May 30, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Details and comments

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nonhermitian commented Jul 14, 2021

Uh oh!

nonhermitian commented Jul 16, 2021

Uh oh!

Uh oh!

ShellyGarion commented Aug 26, 2021

Uh oh!

chriseclectic left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ShellyGarion Oct 12, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ShellyGarion Sep 5, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ShellyGarion commented Oct 6, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ShellyGarion commented May 30, 2021 •

edited

Loading

ShellyGarion Oct 12, 2021 •

edited

Loading

ShellyGarion Sep 5, 2021 •

edited

Loading

ShellyGarion commented Oct 6, 2021 •

edited

Loading