Introducing BenchQ, The Quantum Resource Estimation Tool We Developed With DARPA

We recently announced that, together with our partners, we have been awarded the second phase of the Defense Advanced Research Projects Agency (DARPA) Quantum Benchmarking program. So we thought we’d take a moment to share the progress we made in the first phase and what we’re hoping to accomplish in the second phase.  

But first things first:  

What Is the Quantum Benchmarking Program?

DARPA created the Quantum Benchmarking program to answer some important questions, including:  

• What are the high-utility problems that future quantum computers might solve? 
• What are the quantum computing resources required to unlock this utility? 

Zapata AI was a natural fit for the program. Not only did we have a long history of developing algorithms for quantum computers, we also had the Orquestra® platform.

In our work with customers, we use Orquestra to run Industrial Generative AI applications on different hardware backends. But the same functionality makes Orquestra an ideal environment for predicting the performance of quantum algorithms for various hardware modalities. Orquestra can also scale up compute on the cloud for large benchmarking calculations, and its collaboration tools are well suited for sharing data with the large group of collaborators involved in the DARPA program. 

Because of this fit, we were awarded contracts for the 18-month Phase I of the program in 2022 as participants in both Technical Area 1 (TA1) and Technical Area 2 (TA2). Our work on both TA1 and TA2 has since been awarded to continue on in Phase II. 

In TA1, the goal was to identify high-value use cases and create performance benchmarks for each of these use cases, which could be used to compare the performance of different quantum hardware devices. For this technical area we were a subcontractor of L3Harris with support from BBVA, bp, Copernic Catalysts, Mitsubishi Chemical, and academic partners at the University of Toronto. We announced the delivery of the first 30 quantum computing challenge scenarios back in May. 

In TA2, the goal was to create software tools that could make hardware-specific resource estimates for quantum computers. We worked on this alongside our partners at Aalto University, IonQ, the University of Technology Sydney, and the University of Texas at Dallas. The end result was BenchQ — an open-source tool for resource estimation and benchmarking for quantum computing applications. 

Once we had developed a first version of BenchQ, the next step was to show how it could help us understand when quantum computers will solve utility-scale problems. To do that, we used it to estimate the resources required for an industrially valuable problem in chemistry. 

When Will Quantum Computers Be Useful for Chemistry?

Chemistry is one of the most promising fields where quantum computers are expected to make an impact. Since quantum computers are based on the same quantum physics that governs the behavior of atoms and molecules, they can also theoretically simulate the uncertainty of quantum physics more accurately than classical computers. This unique property means that quantum computers could one day support the development of new battery materials, new drugs, new industrial catalysts, and much more.  

One of the key problems that quantum computers could help solve is ground state energy estimation (GSEE). For most valuable applications of quantum computing in chemistry, you would need to estimate the ground state energy of the molecules in question. Once you have an estimate of the ground state energy, you can extrapolate a lot of other properties of a molecule or chemical reaction you may be interested in. As a result, GSEE has been the focus of a lot of research in the quantum computing community. That made it an ideal problem for demonstrating resource estimation on BenchQ. 

The computational resources required to calculate the GSE increase with the size and complexity of the molecule, with larger molecules requiring more computational resources. In our work in TA1, we identified a list of molecules that would be valuable for industry — as catalysts, specifically — but are too big to calculate the GSE using classical computers. We then used BenchQ to estimate the quantum resources that would be required to calculate the GSE for these molecules. 

On the left, you can see the number of physical qubits that would be required to calculate the GSE for each of the catalysts identified in TA1, and on the right you can see the hours required to calculate the GSE for each molecule. On the X-axis in both plots, the number of spin orbitals corresponds to the size and complexity of the molecule. The dotted line represents the utility threshold, or the point where the molecule becomes too big to calculate using a classical computer.  

We looked at both trapped ion and superconducting quantum computing architectures, two popular designs for quantum computers. Special thanks to IonQ, who helped to develop the models for how these future quantum computers would perform. As you can see, these models predict that trapped Ion computers need fewer qubits to calculate GSE, but with a tradeoff of a longer runtime. You get a similar tradeoff of fewer qubits for longer runtime if you use the low-depth quantum circuits we developed for GSEE last summer.  

Based on these results, we would need millions of physical qubits to get a valuable calculation for GSEE as things stand today — far beyond the capabilities of today’s quantum devices. However, it’s important to note that these predictions are based on current hardware specifications and do not account for improvements in runtime speed or gate fidelity that are expected by hardware providers. These improvements could drastically reduce the resources required for a valuable GSEE calculation. Our hope is that BenchQ will help reduce these resource requirements further by enabling the quantum community to identify any inefficiencies in our current algorithms or hardware.

Next Steps 

Our work with BenchQ is just getting started. Our plan now is to generate resource estimates for more application instances — in other words, industrially relevant problems that are hard to solve classically. As the work in TA1 continues to generate new application instances, we will use BenchQ to estimate the resources required for these applications.  

We’ll also be improving the compilation tools in BenchQ so we can make more detailed resource estimates and accommodate larger problems. These more detailed estimates will help developers across the quantum stack improve their tools and better explore the design space of the quantum problem to find a more efficient way to solve it.  

We’re excited to share BenchQ and hope it will be a valuable resource for the quantum computing community. We invite hardware developers to use BenchQ as a tool to optimize the design of their architectures for different applications and invite software developers to use it to optimize their algorithms for different quantum devices. Our hope is that BenchQ will help the community identify bottlenecks and gaps that we can work to overcome and bring the world closer to the age of practical quantum advantage. 

We welcome any feedback on the software, if you have any suggestions please create an issue in the BenchQ GitHub or star the repo if you’re a fan. 

Finally, we’d like to thank our awesome team members at Aalto University, IonQ, the University of Technology Sydney, and the University of Texas at Dallas. Without their contributions, none of this important work would have been possible. We’re looking forward to building on our work in Phase I as we improve BenchQ and broaden community adoption in Phase II.  

If you’d like to use BenchQ, you can run pip install benchq in your terminal or you can check out the Github repo here.