Quantum Multi-Agent Reinforcement Learning via Variational Quantum Circuit Design

Yun, Won Joon; Kwak, Yunseok; Kim, Jae Pyoung; Cho, Hyunhee; Jung, Soyi; Park, Jihong; Kim, Joongheon

doi:10.48550/arxiv.2203.10443

Cited by 3 publications

(5 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The small number of trainable parameters in the RL/MARL regime acts as a vulnerability for the classical neural network. In [30], [31], it is proven that QRL and QMARL can achieve similar performance to a classical RL/MARL. In the results of this paper, the classical neural network with fewer parameters yields lower performance due to two reasons; 1) index embedding on observation, and 2) parameter-shared policy.…”

Section: F Discussionmentioning

confidence: 96%

Quantum Multiagent Actor–Critic Neural Networks for Internet-Connected Multirobot Coordination in Smart Factory Management

Yun

Kim

Jung

et al. 2023

IEEE Internet Things J.

View full text Add to dashboard Cite

As one of the latest fields of interest in both academia and industry, quantum computing has garnered significant attention. Among various topics in quantum computing, variational quantum circuits (VQC) have been noticed for their ability to carry out quantum deep reinforcement learning (QRL). This paper verifies the potential of QRL, which will be further realized by implementing quantum multi-agent reinforcement learning (QMARL) from QRL, especially for Internet-connected autonomous multi-robot control and coordination in smart factory applications. However, the extension is not straightforward due to the non-stationarity of classical MARL. To cope with this, the centralized training and decentralized execution (CTDE) QMARL framework is proposed under the Internet connection. A smart factory environment with the Internet of Things (IoT)-based multiple agents is used to show the efficacy of the proposed algorithm. The simulation corroborates that the proposed QMARL-based autonomous multi-robot control and coordination performs better than the other frameworks.

show abstract

Section: F Discussionmentioning

confidence: 96%

Quantum Multiagent Actor–Critic Neural Networks for Internet-Connected Multirobot Coordination in Smart Factory Management

Yun

Kim

Jung

et al. 2023

IEEE Internet Things J.

View full text Add to dashboard Cite

show abstract

“…The quantum circuit uses the three-layer structure of 1-qubit rotation data encoding, variational layers with entanglement gates, and measurement. The paper applies QRL to multi-agent problems and extends the original proposal on quantum CTDE by Yun et al [Yun+22]. An additional step is introduced for the training procedure, resulting in a meta-learning approach.…”

Section: Algorithmicmentioning

confidence: 94%

“…It is not completely clear, whether this two-step training procedure is beneficial in a general setup. (see [Yun+22])…”

Section: Algorithmicmentioning

confidence: 99%

See 1 more Smart Citation

A Survey on Quantum Reinforcement Learning

Meyer¹,

Ufrecht²,

Periyasamy³

et al. 2022

Preprint

View full text Add to dashboard Cite

Quantum reinforcement learning is an emerging field at the intersection of quantum computing and machine learning. While we intend to provide a broad overview of the literature on quantum reinforcement learning (our interpretation of this term will be clarified below), we put particular emphasis on recent developments. With a focus on already available noisy intermediate-scale quantum devices, these include variational quantum circuits acting as function approximators in an otherwise classical reinforcement learning setting. In addition, we survey quantum reinforcement learning algorithms based on future fault-tolerant hardware, some of which come with a provable quantum advantage. We provide both a birds-eye-view of the field, as well as summaries and reviews for selected parts of the literature.

show abstract

“…Several improved quantum policy gradient algorithms have been proposed, including actor-critic [24], soft actor-critic (SAC) [9], [25], deep deterministic policy gradient (DDPG) [26], quantum asynchronous advantage actor-critic (A3C) [27], and generative adversarial RL [28], aiming to enhance the efficiency and effectiveness of QRL methods. QRL has found applications in quantum control [29] and has been extended to multi-agent settings with promising results [30]- [32]. Chen et al [10] explored the use of evolutionary optimization in QRL, employing parallel training and selection of the bestperforming agents as parents for the next generation.…”

Section: Related Workmentioning

confidence: 99%

Quantum Deep Q-Learning with Distributed Prioritized Experience Replay

Chen

2023

2023 IEEE International Conference on Quantum Computing and Engineering (QCE)

View full text Add to dashboard Cite

This paper introduces the QDQN-DPER framework to enhance the efficiency of quantum reinforcement learning (QRL) in solving sequential decision tasks. The framework incorporates prioritized experience replay, asynchronous training and novel matrix loss into the training algorithm to reduce the high sampling complexities. Numerical simulations demonstrate that QDQN-DPER outperforms the baseline distributed quantum Q-learning with the same model architecture. The proposed framework holds potential for more complex tasks while maintaining training efficiency.

show abstract

Quantum Multi-Agent Reinforcement Learning via Variational Quantum Circuit Design

Cited by 3 publications

References 0 publications

Quantum Multiagent Actor–Critic Neural Networks for Internet-Connected Multirobot Coordination in Smart Factory Management

Quantum Multiagent Actor–Critic Neural Networks for Internet-Connected Multirobot Coordination in Smart Factory Management

A Survey on Quantum Reinforcement Learning

Quantum Deep Q-Learning with Distributed Prioritized Experience Replay

Contact Info

Product

Resources

About