Quantum extension of variational Bayes inference

Abstract

Variational Bayes (VB) inference is one of the most important algorithms in machine learning and widely used in engineering and industry. However, VB is known to suffer from the problem of local optima. In this Letter, we generalize VB by using quantum mechanics, and propose a new algorithm, which we call quantum annealing variational Bayes (QAVB) inference. We then show that QAVB drastically improve the performance of VB by applying them to a clustering problem described by a Gaussian mixture model. Finally, we discuss an intuitive understanding on how QAVB works well.Introdction.-Machine learning gathers considerable attention in a wide range of fields, and much effort is devoted to develop effective algorithms. Variational Bayes (VB) inference [1-6] is one of the most fundamental methods in machine learning, and widely used for parameter estimation and model selection. In particular, VB has succeeded to compensate some disadvantages of the expectation-maximization (EM) algorithm [5][6][7], which is a well-used approach for maximum likelihood estimation. For example, overfitting, which is often occurred in EM, is greatly moderated in VB. Furthermore, a variant of VB based on classical statistical mechanics, which we call simulated annealing variational Bayes (SAVB) inference in this paper, was proposed [8] and has been getting popular in many fields due to its effectiveness. However, it is also known that VB and SAVB often fail to estimate appropriate parameters of an assumed model depending on prior distributions and initial conditions.In the field of physics, the study of quantum computation and how to exploit it for machine learning are getting popular. For example, while experimentalists are intensively developing quantum machines [9-13], theorists are developing quantum error correction schemes [14-18] and quantum algorithms [19][20][21][22][23][24][25][26][27][28][29]. In particular, the study of quantum annealing (QA) has a history for more than two decades [22][23][24][25] and is still progressing [26].In this Letter, by focusing on QA and VB, we devise a quantum-mechanically inspired algorithm that works on a classical computer in practical time and achieves a considerable improvement over VB and SAVB. More specifically, we introduce the mathematical mechanism of quantum fluctuations into VB, and propose a new algorithm, which we call quantum annealing variational Bayes (QAVB) inference. To demonstrate the performance of QAVB, we consider a clustering problem and employ a Gaussian mixture model, which is one of important applications of VB. Then, we see that QAVB succeeds in estimation with high probability while VB and SAVB do not. This fact is noteworthy because our algorithm is one of the few algorithms that can obtain a global optimum of non-convex optimization in practical computational time without using random numbers.Problem setting of VB.-For preparation of a quantum extension of VB, we briefly review the problem setting of VB [1][2][3][4][5][6]. First, we summarize the definitions of variable...

Quantum Speedup of Natural Gradient for Variational Bayes

“…The result in (24) says that the algorithm does stop at a stationary point of the target L(λ). The result in (25) says that, under the strong convexity condition, the algorithm converges to the minimum point λ * .…”

“…Kerenidis and Prakash [23] use building blocks of HHL for Euclidean gradient descent for affine transformations. Miyahara and Sughiyama [24] perform mean-field VB via quantum annealing -an alternative to gradient descent optimization.…”

Quantum expectation-maximization algorithm

“…In Refs. [15,16], we have succeeded in improving the performances of the EM algorithm and VB. However, the aim of Refs.…”

“…The expectation-maximization (EM) algorithm [9,10,12] and variational Bayes (VB) inference [9,10] with the GMM are often used to improve the clustering, since the general GMM can deal with a wider class of data sets. Recently, one of the authors proposed quantum-inspired algorithms for the EM algorithm [13][14][15] and VB [16]. In Refs.…”

“…However, the aim of Refs. [15,16] is to make use of quantum fluctuations as a numerical tool and not to provide a quantum speedup over a classical algorithm; as a result, the computational costs are almost the same.…”