Add benchmarks for converter functions #628
Conversation
This commit adds benchmarks for the converter functions. The converter functions are normally quick to execute but are commonly used as part of other operations. For example circuit_to_dag and dag_to_circuit get called under the covers during transpile() calls. Individually benchmarking the performance of them is useful so we can isolate and track their performance, even if it's normally only a small percentage of a larger operation. The benchmarks here just use the large qasm circuit example as a type of worst case example because it has >3k gates. In the future we can add additional types of circuits such as ones with many registers, etc to try and test additional worst case edge cases.
There was a problem hiding this comment.
👍 on the concept, and I agree we should be splitting our benchmarks into synthetic benchmarks (targeting smaller components, edge cases, using our understanding of the implementation, ...) and application benchmarks (that closely cover what users commonly run).
Instead of test_eoh_qasm, can we build synthetic circuits from circuit.random.random_circuit? That way we can track with e.g. depth/width (test_eoh_qasm isn't great as either an application benchmark or a synthetic benchmark) . If there are other cases we'd like to cover (specifying a gate set so we can look at performance for gates that require many steps to reach the target basis), maybe we can add an issue in terra.
Otherwise, looks good.
|
@kdk I switched this to use |
* Add benchmarks for converter functions This commit adds benchmarks for the converter functions. The converter functions are normally quick to execute but are commonly used as part of other operations. For example circuit_to_dag and dag_to_circuit get called under the covers during transpile() calls. Individually benchmarking the performance of them is useful so we can isolate and track their performance, even if it's normally only a small percentage of a larger operation. * Pivot to use random circuit The benchmarks now use a randomly generated circuit (with a fixed seed for consistency) of varying sizes to test different cases. In the future we can add additional types of circuits to try and test additional worst case edge cases. * Fix lint * Fix lint for real * Add comment about skips
* Add benchmarks for converter functions This commit adds benchmarks for the converter functions. The converter functions are normally quick to execute but are commonly used as part of other operations. For example circuit_to_dag and dag_to_circuit get called under the covers during transpile() calls. Individually benchmarking the performance of them is useful so we can isolate and track their performance, even if it's normally only a small percentage of a larger operation. * Pivot to use random circuit The benchmarks now use a randomly generated circuit (with a fixed seed for consistency) of varying sizes to test different cases. In the future we can add additional types of circuits to try and test additional worst case edge cases. * Fix lint * Fix lint for real * Add comment about skips
Summary
This commit adds benchmarks for the converter functions. The converter
functions are normally quick to execute but are commonly used as part of
other operations. For example circuit_to_dag and dag_to_circuit get
called under the covers during transpile() calls. Individually
benchmarking the performance of them is useful so we can isolate and
track their performance, even if it's normally only a small percentage
of a larger operation. The benchmarks here just use the large qasm
circuit example as a type of worst case example because it has >3k
gates. In the future we can add additional types of circuits such as
ones with many registers, etc to try and test additional worst case edge
cases.
Details and comments
This was inspired by: Qiskit/qiskit#3176 which
should improve performance but there aren't any direct measurement of
that in the benchmark suite.