Quantum Architecture Search via Continual Reinforcement Learning

Ye, Esther; Chen, Samuel Yen-Chi

doi:10.48550/arxiv.2112.05779

Cited by 4 publications

(6 citation statements)

References 85 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are different technologies to design the different architectures of the circuit by machine learning [88] or reinforcement learning [89,90].…”

Section: Parameter Layermentioning

confidence: 99%

Unentangled quantum reinforcement learning agents in the OpenAI Gym

Hsiao¹,

Du²,

Chiang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Classical reinforcement learning (RL) has generated excellent results in different regions ; however, its sample inefficiency remains a critical issue. In this paper, we provide concrete numerical evidence that the sample efficiency (the speed of convergence) of quantum RL could be better than that of classical RL, and for achieving comparable learning performance, quantum RL could use much (at least one order of magnitude) fewer trainable parameters than classical RL. Specifically, we employ the popular benchmarking environments of RL in the OpenAI Gym, and show that our quantum RL agent converges faster than classical fully-connected neural networks (FCNs) in the tasks of CartPole and Acrobot under the same optimization process. We also successfully train the first quantum RL agent that can complete the task of LunarLander in the OpenAI Gym. Our quantum RL agent only requires a single-qubit-based variational quantum circuit without entangling gates, followed by a classical neural network (NN) to post-process the measurement output. Finally, we could accomplish the aforementioned tasks on the real IBM quantum machines. To the best of our knowledge, none of the earlier quantum RL agents could do that.

show abstract

“…There are different technologies to design the different architectures of the circuit by machine learning [88] or reinforcement learning [89,90].…”

Section: Parameter Layermentioning

confidence: 99%

Unentangled quantum reinforcement learning agents in the OpenAI Gym

Hsiao¹,

Du²,

Chiang³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Ref. [35] studied quantum state preparation for changing environment by training a reinforcement learning agent that utilizes past policies learned from previous environments.…”

Section: Related Workmentioning

confidence: 99%

“…Ref. [35] proposed a continual reinforcement learning framework to deal with the constantly changing environment. In this framework, the agent leverages past policies learned from previous noise environments to generate a state preparation quantum circuit for a new noise environment.…”

Section: Introductionmentioning

confidence: 99%

Continual learning of quantum state classification with gradient episodic memory

Situ¹,

Lu²,

Mao³

et al. 2022

Preprint

View full text Add to dashboard Cite

Continual learning is one of the many areas of machine learning research. For the goal of strong artificial intelligence that can mimic human-level intelligence, AI systems would have the ability to adapt to ever-changing scenarios and learn new knowledge continuously without forgetting previously acquired knowledge. A phenomenon called catastrophic forgetting emerges when a machine learning model is trained across multiple tasks. The model's performance on previously learned tasks may drop dramatically during the learning process of the newly seen task. Some continual learning strategies have been proposed to address the catastrophic forgetting problem. Recently, continual learning has also been studied in the context of quantum machine learning. By leveraging the elastic weight consolidation method, a single quantum classifier can perform multiple tasks after being trained consecutively on those tasks. In this work, we incorporate the gradient episodic memory method to train a variational quantum classifier. The gradient of the current task is projected to the closest gradient, avoiding the increase of the loss at previous tasks, but allowing the decrease. We use six quantum state classification tasks to benchmark this method. Numerical simulation results show that better performance is obtained compared to the elastic weight consolidation method. Furthermore, positive transfer of knowledge to previous tasks is observed, which means the classifier's performance on previous tasks is enhanced rather than compromised while learning a new task.

show abstract

“…In the case of some VQAs, to address the challenges of finding the architecture of ansatz, methods have been introduced that draw on the insight and techniques of machine learning [8][9][10][11][12], such as a process of automating the architecture engineering of quantum circuits is known as quantum architecture search (QAS) [9,13,14]. Recent studies have strongly suggested that double deep Q-networks (DDQN) in reinforcement learning (RL) can successfully solve QAS problems [8,10], performance improvement in QAOA variants [15] as well as the task of quantum compiling [16].…”

Section: Introductionmentioning

confidence: 99%

Enhancing variational quantum state diagonalization using reinforcement learning techniques

Kundu,

Bedełek,

Ostaszewski

et al. 2024

New J. Phys.

View full text Add to dashboard Cite

The variational quantum algorithms are crucial for the application of NISQ computers. Such algorithms require short quantum circuits, which are more amenable to implementation on near-term hardware, and many such methods have been developed. One of particular interest is the so-called variational quantum state diagonalization method, which constitutes an important algorithmic subroutine and can be used directly to work with data encoded in quantum states. In particular, it can be applied to discern the features of quantum states, such as entanglement properties of a system, or in quantum machine learning algorithms. In this work, we tackle the problem of designing a very shallow quantum circuit, required in the quantum state diagonalization task, by utilizing reinforcement learning (RL). We use a novel encoding method for the RL-state, a dense reward function, and an $\epsilon$-greedy policy to achieve this. We demonstrate that the circuits proposed by the reinforcement learning methods are shallower than the standard variational quantum state diagonalization algorithm and thus can be used in situations where hardware capabilities limit the depth of quantum circuits. The methods we propose in the paper can be readily adapted to address a wide range of variational quantum algorithms.

show abstract

Quantum Architecture Search via Continual Reinforcement Learning

Cited by 4 publications

References 85 publications

Unentangled quantum reinforcement learning agents in the OpenAI Gym

Unentangled quantum reinforcement learning agents in the OpenAI Gym

Continual learning of quantum state classification with gradient episodic memory

Enhancing variational quantum state diagonalization using reinforcement learning techniques

Contact Info

Product

Resources

About