Quantum Enhancements for Deep Reinforcement Learning in Large Spaces

Jerbi, Sofiène; Trenkwalder, Lea M.; Nautrup, Hendrik Poulsen; Briegel, Hans J.; Dunjko, Vedran

doi:10.1103/prxquantum.2.010328

Cited by 65 publications

(77 citation statements)

References 71 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the field of quantum machine learning (QML), applications of VQCs to standard machine learning tasks have achieved various degrees of success. Prominent examples include function approximation [13,[43][44][45], classification [13,14,[46][47][48][49][50][51][52][53][54][55][56][57][58][59][60][61][62][63], generative modeling [64][65][66][67][68], deep RL [29][30][31][32][33][69][70][71][72], sequence modeling [43,[73][74][75][76], speech recognition [77], metric and embedding learning [78,79], transfer learning [50,80] and federated learning [...…”

Section: Variational Quantum Circuitsmentioning

confidence: 99%

See 1 more Smart Citation

Variational quantum reinforcement learning via evolutionary optimization

Chen

Huang

Hsing

et al. 2022

Mach. Learn.: Sci. Technol.

View full text Add to dashboard Cite

Recent advance in classical reinforcement learning (RL) and quantum computation (QC) points to a promising direction of performing RL on a quantum computer. However, potential applications in quantum RL are limited by the number of qubits available in modern quantum devices. Here we present two frameworks of deep quantum RL tasks using a gradient-free evolution optimization: First, we apply the amplitude encoding scheme to the Cart-Pole problem, where we demonstrate the quantum advantage of parameter saving using the amplitude encoding; Second, we propose a hybrid framework where the quantum RL agents are equipped with a hybrid tensor network-variational quantum circuit (TN-VQC) architecture to handle inputs of dimensions exceeding the number of qubits. This allows us to perform quantum RL on the MiniGrid environment with 147-dimensional inputs. The hybrid TN-VQC architecture provides a natural way to perform efficient compression of the input dimension, enabling further quantum RL applications on noisy intermediate-scale quantum devices.

show abstract

Section: Variational Quantum Circuitsmentioning

confidence: 99%

“…We will leave this for future investigation. For a more detailed review of recent developments in quantum RL, we refer interested readers to [69,113].…”

Section: Relevant Studiesmentioning

confidence: 99%

Variational quantum reinforcement learning via evolutionary optimization

Chen

Huang

Hsing

et al. 2022

Mach. Learn.: Sci. Technol.

View full text Add to dashboard Cite

show abstract

“…Recent developments in VQA provide a framework to design near-term quantum applications in various scenarios [10,11]. VQA-based applications include solving quantum chemistry problems [12] as well as machine learning (ML) tasks such as: function approximation [13][14][15][16], classification [14,[17][18][19][20][21][22][23][24][25][26][27][28][29][30][31][32][33][34], generative modeling [35][36][37][38][39], deep reinforcement learning [40][41][42][43][44][45][46][47][48][49], sequence modeling [13,[50][51][52], speech recognition [53], metric and embedding learning [54,55...…”

Section: Introductionmentioning

confidence: 99%

Quantum Architecture Search via Continual Reinforcement Learning

Ye¹,

Chen²

2021

Preprint

View full text Add to dashboard Cite

Quantum computing has promised significant improvement in solving difficult computational tasks over classical computers. Designing quantum circuits for practical use, however, is not a trivial objective and requires expert-level knowledge. To aid this endeavor, this paper proposes a machine learning-based method to construct quantum circuit architectures. Previous works have demonstrated that classical deep reinforcement learning (DRL) algorithms can successfully construct quantum circuit architectures without encoded physics knowledge. However, these DRLbased works are not generalizable to settings with changing device noises, thus requiring considerable amounts of training resources to keep the RL models up-to-date. With this in mind, we incorporated continual learning to enhance the performance of our algorithm. In this paper, we present the Probabilistic Policy Reuse with deep Q-learning (PPR-DQL) framework to tackle this circuit design challenge. By conducting numerical simulations over various noise patterns, we demonstrate that the RL agent with PPR was able to find the quantum gate sequence to generate the two-qubit Bell state faster than the agent that was trained from scratch. The proposed framework is general and can be applied to other quantum gate synthesis or control problems-including the automatic calibration of quantum devices.

show abstract

“…Various results show quantum advantages for quantum-enhanced agents interacting with a classical environment. In this way, improvements on the deliberation time of an agent [12][13][14] or a better performance via variatonal quantum circuits [15] can be achieved.…”

Section: Introductionmentioning

confidence: 99%

Performance analysis of a hybrid agent for quantum-accessible reinforcement learning

Hamann,

Wölk

2021

Preprint

View full text Add to dashboard Cite

In the last decade quantum machine learning has provided fascinating and fundamental improvements to supervised, unsupervised and reinforcement learning. In reinforcement learning, a so-called agent is challenged to solve a task given by some environment. The agent learns to solve the task by exploring the environment and exploiting the rewards it gets from the environment. For some classical task environments, such as deterministic strictly epochal environments, an analogue quantum environment can be constructed which allows to find rewards quadratically faster by applying quantum algorithms. In this paper, we analytically analyze the behavior of a hybrid agent which combines this quadratic speedup in exploration with the policy update of a classical agent. This leads to a faster learning of the hybrid agent compared to the classical agent. We demonstrate that if the classical agent needs on average J rewards and T c epochs to learn how to solve the task, the hybrid agent will take T q ≤ α T c J epochs on average. Here, α denotes a constant which is independent of the problem size. Additionally, we prove that if the environment allows for maximally αokmax sequential coherent interactions, e.g. due to noise effects, an improvement given by T q ≈ αo T c/4kmax is still possible.

show abstract

Quantum Enhancements for Deep Reinforcement Learning in Large Spaces

Cited by 65 publications

References 71 publications

Variational quantum reinforcement learning via evolutionary optimization

Variational quantum reinforcement learning via evolutionary optimization

Quantum Architecture Search via Continual Reinforcement Learning

Performance analysis of a hybrid agent for quantum-accessible reinforcement learning

Contact Info

Product

Resources

About