Sandia National Laboratories has developed a new architecture-agnostic benchmark for measuring quantum performance across both physical- and logical-qubit systems, with input from Quantinuum and NVIDIA. Named QUOPS (Quantum Universal Operations Performance System), it can be applied to different architectures, codes, modalities, and levels of fault tolerance.
Sandia shared early results in a preprint posted ahead of IEEE Quantum Week, reporting initial QUOPS benchmarks for Quantum Processing Units (QPUs) from Google, IBM and Quantinuum. The scientific paper is available on arXiv.org and QUOPS can also be run on Github, while NVIDIA has also made the QUOPS reference implementation available in NVIDIA CUDA-Q.
“At Sandia, we can’t wait to see fault-tolerant quantum computers helping to solve problems of national importance for the Department of Energy and the United States,” said Timothy Proctor, co-director of Sandia National Laboratories’ Quantum Performance Laboratory. “Our mission right now is to bring that about sooner, by accelerating our industry partners’ progress. To do that, we, and other quantum computing stakeholders, have to be able to track and forecast the growth of quantum computer abilities. We created QUOPS to do exactly that, and we’re excited to see it used by quantum computing vendors and customers.”
A summary of the architecture has been published by Quantinuum, where the company stated that QUOPS runs the same randomized workloads across different computational shapes, measures whether each workload succeeds, identifies the boundary of a system’s capability region, and reports two summary metrics:
- Q: the largest benchmark circuit size that passes the success threshold inside a utility-motivated region. Size is defined as 2*(width)*(depth).
- Ω: the effective operations per second at that point.
The result is a direct measure of how much computation a system can perform and how quickly it can do so. Together, these measurements provide a two-dimensional view of capability while reducing system performance to a common currency, i.e., quantum operations.
According to Quantinuum, component-level metrics remain essential for engineering; qubit count, two-qubit fidelity, and gate speed can reveal control errors, crosstalk, leakage, connectivity constraints, and other system limitations, but do not necessarily predict system-level performance.
QUOPS, said Quantinuum, will not replace every quantum benchmark but instead “serves as a common system-level yardstick that can make roadmaps more comparable, procurement more objective, and progress easier to track.”
In its blog post, Quantinuum has also called on vendors to report QUOPS metrics (Q, Ω) and capability regions alongside existing metrics, buyers and agencies to consider QUOPS thresholds in RFPs, and researchers to contribute fault-tolerant architectures and resource estimates.
