The Presence and Absence of Barren Plateaus in Tensor-network Based Machine Learning

Abstract

Tensor networks are efficient representations of high-dimensional tensors with widespread applications in quantum many-body physics. Recently, they have been adapted to the field of machine learning, giving rise to an emergent research frontier that has attracted considerable attention. Here, we study the trainability of tensor-network based machine learning models by exploring the landscapes of different loss functions, with a focus on the matrix product states (also called tensor trains) architecture. In particular, we rigorously prove that barren plateaus (i.e., exponentially vanishing gradients) prevail in the training process of the machine learning algorithms with global loss functions. Whereas, for local loss functions the gradients with respect to variational parameters near the local observables do not vanish as the system size increases. Therefore, the barren plateaus are absent in this case and the corresponding models could be efficiently trainable. Our results reveal a crucial aspect of tensor-network based machine learning in a rigorous fashion, which provide a valuable guide for both practical applications and theoretical studies in the future.

Subtleties in the trainability of quantum machine learning models

“…In this work we study the trainability and the existence of barren plateaus in QML models. Our work represents a general treatment that goes beyond previous analysis of gradient scaling and trainability in specific QML models [49][50][51][52][53][54][55][56]. Our main results are two-fold.…”

“…Finally, the presence of a BP can be diagnosed by inspecting the eigenvalues of the Hessian and the FI matrix, as these become exponentially vanishing with the system size [42,52] (see also Section III C below). Despite tremendous progress in understanding BPs for VQAs, there are only a few results which analyze the existence of BPs in QML settings [49][50][51][52][53][54][55]. In fact, while in the VQA community there is a consensus that certain properties of an algorithm should be avoided (such as global measurements), these same properties are still widely used in the QML literature (e.g., global parity measurements as those in Eqs.…”

Recent advances for quantum classifiers

“…In addition to the above two studies, we also mention that barren plateaus for variational quantum circuits have also been investigated from other aspects in the literature [103][104][105][106][107][108][109][110][111][112][113][114][115][116][117][118]. For instance, in Ref.…”

“…However, the barren plateaus present the situation that the cost function with respect to the parameters is too flat in most of the space except a small area around the optimal point, thus it is nearly impossible to converge to the optimal point. Nowadays, this phenomenon has attracted broad interest across communities, with many works analyzing the reasons why barren plateau occurs and designing strategies to avoid it [101][102][103][104][105][106][107][108][109][110][111][112][113][114][115][116][117][118]. The reasons underlying this phenomenon include the poor design of the quantum circuits [101], non-negligible noises [104], massive entanglement [106], and inappropriate choice of cost functions [102,118].…”

“…Nowadays, this phenomenon has attracted broad interest across communities, with many works analyzing the reasons why barren plateau occurs and designing strategies to avoid it [101][102][103][104][105][106][107][108][109][110][111][112][113][114][115][116][117][118]. The reasons underlying this phenomenon include the poor design of the quantum circuits [101], non-negligible noises [104], massive entanglement [106], and inappropriate choice of cost functions [102,118]. Second, the vulnerability of quantum classifiers concerns their performance against adversarial attacks.…”