Co-Author:

Kaitlin Gili

Despite being powerful and widespread algorithms, unsupervised machine learning models known as generative models still come with many open challenges, thus representing an active research topic within the classical and quantum machine learning community. An outstanding question is how to evaluate their performance after training. More specifically, defining and accurately measuring their generalization capabilities – the model’s ability to generate novel high-quality data – remains an ongoing challenge. In this work, we construct a simple and unambiguous approach to generalization, and propose adequate metrics to quantify such capabilities. Using our sample-based metrics, any generative model can be evaluated on the same ground on a concrete well-defined framework: from state-of-the-art classical models like Generative Adversarial Networks (GANs), to quantum-inspired models such as Tensor Network Born Machines (TNBMs) and quantum models such as Quantum Circuit Born Machines (QCBMs).

Our results show that these metrics provide a robust assessment of generalization capabilities and can diagnose trainability issues, e.g., mode collapse and overfitting. Remarkably, they can be used to compare generative models in different domains: we demonstrate that our TNBM instance has up to a 68x enhancement in generating unseen and valid samples compared to GANs, and a ratio of 61:2 for generating samples with better quality than those observed in the training set.

As generalization is the gold standard for measuring the quality of a generative model, we foresee our framework and metrics as valuable tools for rigorously defining practical quantum advantage in this domain. We hope this work incites both quantum and classical ML experts to use this framework to enhance the design and the evaluation of their models, in this now quantitative race towards demonstrating practical quantum advantage in generative modeling.

*Marta Mauri and Kaitlin Gili contributed equally to this work as first author.