Mitigating Barren Plateaus with Transfer-learning-inspired Parameter Initializations

Liu, Huanyu; Chen, Zhaoyun; Sun, Tai‐Ping; Wu, Yu-Chun; Han, Yu; Guo, Guo-Ping

doi:10.48550/arxiv.2112.10952

Cited by 3 publications

(4 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [22], the proposed method uses the identity block strategy to limit the effective depth of the circuits used to calculate the first parameter update to avoid the QNN being stuck in a barren plateau at the start of training. Further, [23] proposes a parameter initialization strategy that transfers the small pre-trained layer blocks to the target model stacking by multiple identical basic blocks. This idea is based on transfer learning.…”

Section: Mitigating the Barren Plateaumentioning

confidence: 99%

See 1 more Smart Citation

LAWS: Look Around and Warm-Start Natural Gradient Descent for Quantum Neural Networks

Tao¹,

Wu²,

Xia³

et al. 2022

Preprint

View full text Add to dashboard Cite

Variational quantum algorithms (VQAs) have recently received significant attention from the research community due to their promising performance in Noisy Intermediate-Scale Quantum computers (NISQ). However, VQAs run on parameterized quantum circuits (PQC) with randomly initialized parameters are characterized by barren plateaus (BP) where the gradient vanishes exponentially in the number of qubits. In this paper, we first review quantum natural gradient (QNG), which is one of the most popular algorithms used in VQA, from the classical first-order optimization point of view. Then, we proposed a Look Around Warm-Start QNG (LAWS) algorithm to mitigate the widespread existing BP issues. LAWS is a combinatorial optimization strategy taking advantage of model parameter initialization and fast convergence of QNG. LAWS repeatedly reinitializes parameter search space for the next iteration parameter update. The reinitialized parameter search space is carefully chosen by sampling the gradient close to the current optimal. Moreover, we present a unified framework (WS-SGD) for integrating parameter initialization techniques into the optimizer. We provide the convergence proof of the proposed framework for both convex and non-convex objective functions based on Polyak-Lojasiewicz (PL) condition. Our experiment results show that the proposed algorithm could mitigate the BP and have better generalization ability in quantum classification problems.

show abstract

Section: Mitigating the Barren Plateaumentioning

confidence: 99%

“…To address the BP issue, gradient rescaling [20], [21], PQC's parameter initialization [22], [23], and gradient-free optimizations [24] have been extensively studied. Our work is also motivated by addressing the BP issue.…”

Section: Introductionmentioning

confidence: 99%

LAWS: Look Around and Warm-Start Natural Gradient Descent for Quantum Neural Networks

Tao¹,

Wu²,

Xia³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Other methods make use of tailored distributions of the initial circuits parameters and carefully designed circuits architectures [21][22][23][24][25][26][27]. Yet, only a handful of configurations offer trainability guarantees and robustness against barren plateaus [28,29].…”

Section: Introductionmentioning

confidence: 99%

Efficient estimation of trainability for variational quantum circuits

Heyraud¹,

Li²,

Donatella³

et al. 2023

Preprint

View full text Add to dashboard Cite

“…Many methods have been suggested to mitigate BPs in the literature. Some of these methods suggest to use different ansätze or cost functions [18,19], determining a better initial point to start the optimization [20][21][22][23], determining the step size during the optimization based on the ansatz [24], correlating parameters of the ansatz (e.g., restricting the directions of rotation) [10,25], or combining multiple methods [26,27].…”

Section: Introductionmentioning

confidence: 99%

Classical Splitting of Parametrized Quantum Circuits

Tüysüz¹,

Clemente²,

Crippa³

et al. 2022

Preprint

View full text Add to dashboard Cite

Barren plateaus appear to be a major obstacle to using variational quantum algorithms to simulate large-scale quantum systems or replace traditional machine learning algorithms. They can be caused by multiple factors such as expressivity, entanglement, locality of observables, or even hardware noise. We propose classical splitting of ansätze or parametrized quantum circuits to avoid barren plateaus. Classical splitting is realized by splitting an N qubit ansatz to multiple ansätze that consists of O(log N ) qubits. We show that such an ansatz can be used to avoid barren plateaus. We support our results with numerical experiments and perform binary classification on classical and quantum datasets. Then, we propose an extension of the ansatz that is compatible with variational quantum simulations. Finally, we discuss a speed-up for gradient-based optimization and hardware implementation, robustness against noise and parallelization, making classical splitting an ideal tool for noisy intermediate scale quantum (NISQ) applications.

show abstract

Mitigating Barren Plateaus with Transfer-learning-inspired Parameter Initializations

Cited by 3 publications

References 47 publications

LAWS: Look Around and Warm-Start Natural Gradient Descent for Quantum Neural Networks

LAWS: Look Around and Warm-Start Natural Gradient Descent for Quantum Neural Networks

Efficient estimation of trainability for variational quantum circuits

Classical Splitting of Parametrized Quantum Circuits

Contact Info

Product

Resources

About