Quantum Reinforcement Learning

Abstract

Abstract-The key approaches for machine learning, especially learning in unknown probabilistic environments are new representations and computation mechanisms. In this paper, a novel quantum reinforcement learning (QRL) method is proposed by combining quantum theory and reinforcement learning (RL). Inspired by the state superposition principle and quantum parallelism, a framework of value updating algorithm is introduced. The state (action) in traditional RL is identified as the eigen state (eigen action) in QRL. The state (action) set can be represented with a quantum superposition state and the eigen state (eigen action) can be obtained by randomly observing the simulated quantum state according to the collapse postulate of quantum measurement. The probability of the eigen action is determined by the probability amplitude, which is parallelly updated according to rewards. Some related characteristics of QRL such as convergence, optimality and balancing between exploration and exploitation are also analyzed, which shows that this approach makes a good tradeoff between exploration and exploitation using the probability amplitude and can speed up learning through the quantum parallelism. To evaluate the performance and practicability of QRL, several simulated experiments are given and the results demonstrate the effectiveness and superiority of QRL algorithm for some complex problems. The present work is also an effective exploration on the application of quantum computation to artificial intelligence.Index Terms-quantum reinforcement learning, state superposition, collapse, probability amplitude, Grover iteration.

Quantum speed-up for unsupervised learning

“…Learning algorithms based on quantum primitives have already been developed in supervised (Anguita et al 2003;Ezhov and Berman 2003) and reinforcement learning (Dong et al 2005). However, not much work has been done yet concerning unsupervised learning, with the exception of a quantum algorithm for the minimal spanning tree (Dürr et al 2004), which can also be used to perform clustering although this was not these authors' intention.…”

Hybrid MDP based integrated hierarchical Q-learning

“…For example, fuzzy logic, grey theory and quantum computation are adopted for the generalization and speedup of RL [12][13][14][15][16][17][18]. Various factors are studied to explore the performance improvement for RL [19,20].…”

Superposition-Inspired Reinforcement Learning and Quantum Reinforcement Learning

“…Taking advantage of quantum computation, the algorithm integration inspired by quantum characteristics will not only improve the performance of existing algorithms on traditional computers, but also promote the development of related research areas such as quantum computer and machine learning. According to our recent research results (Dong et al, 2005a;Dong et al, 2006a;Dong et al, 2006b;Chen et al, 2006a;Chen et al, 2006c;Dong et al, 2007a;Dong et al, 2007b), in this chapter the RL methods based on quantum theory are introduced following the developing roadmap from SuperpositionInspired Reinforcement Learning (SIRL) to Quantum Reinforcement Learning (QRL). As for SIRL methods we concern mainly about the exploration policy.…”

“…In this way, the simulated experiments show that SIRL may accelerate the learning process and allow avoiding the locally optimal policies. When SIRL is extended to quantum mechanical systems, QRL theory is proposed naturally (Dong et al, 2005a, Dong et al, 2007b. In a QRL system, the state value can be represented with quantum state and be obtained by randomly observing the quantum state, which will lead to state collapse according to quantum measurement postulate.…”