Reinforcement Learning Approaches to Coordination in Cooperative Multi-agent Systems

Kapetanakis, Spiros; Kudenko, Daniel; Strens, Malcolm J. A.

doi:10.1007/3-540-44826-8_2

Cited by 25 publications

(7 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(c) RL enables SUs to explore new operating environment and exploit the knowledge gained so far. An issue is that, SUs may converge to a sub-optimal joint action when one of these conditions happens: actions with severe negative rewards exist or multiple high performance actions exist [30]. Using RL helps to solve such issue found in game-based approaches since RL has the flexibility to fine tune its policy as time progresses.…”

Section: Q4mentioning

confidence: 98%

Application of reinforcement learning for security enhancement in cognitive radio networks

Ling

Yau

Qadir

et al. 2015

Applied Soft Computing

View full text Add to dashboard Cite

Please cite this article in press as: M.H. Ling, et al., Application of reinforcement learning for security enhancement in cognitive radio networks, Appl. Soft Comput. J. (2015), http://dx. a b s t r a c tCognitive radio network (CRN) enables unlicensed users (or secondary users, SUs) to sense for and opportunistically operate in underutilized licensed channels, which are owned by the licensed users (or primary users, PUs). Cognitive radio network (CRN) has been regarded as the next-generation wireless network centered on the application of artificial intelligence, which helps the SUs to learn about, as well as to adaptively and dynamically reconfigure its operating parameters, including the sensing and transmission channels, for network performance enhancement. This motivates the use of artificial intelligence to enhance security schemes for CRNs. Provisioning security in CRNs is challenging since existing techniques, such as entity authentication, are not feasible in the dynamic environment that CRN presents since they require pre-registration. In addition these techniques cannot prevent an authenticated node from acting maliciously. In this article, we advocate the use of reinforcement learning (RL) to achieve optimal or near-optimal solutions for security enhancement through the detection of various malicious nodes and their attacks in CRNs. RL, which is an artificial intelligence technique, has the ability to learn new attacks and to detect previously learned ones. RL has been perceived as a promising approach to enhance the overall security aspect of CRNs. RL, which has been applied to address the dynamic aspect of security schemes in other wireless networks, such as wireless sensor networks and wireless mesh networks can be leveraged to design security schemes in CRNs. We believe that these RL solutions will complement and enhance existing security solutions applied to CRN To the best of our knowledge, this is the first survey article that focuses on the use of RL-based techniques for security enhancement in CRNs.

show abstract

Section: Q4mentioning

confidence: 98%

Application of reinforcement learning for security enhancement in cognitive radio networks

Ling

Yau

Qadir

et al. 2015

Applied Soft Computing

View full text Add to dashboard Cite

show abstract

“…For example, the Q-value of 𝒟 𝑗 's 𝑘 th action, i.e., A major challenge for the selection of actions is to strike a balance between exploring and exploiting. In our scheme, we have chosen the Boltzmann strategy; each agent chooses an action to perform in the next iteration with a probability that is based on its current estimate of the usefulness of that action [15], [17].…”

Section: The Proposed Spectrum Allocation Scheme For the Iomt Paradigmmentioning

confidence: 99%

Learning and Game Based Spectrum Allocation Model for Internet of Medical Things (IoMT) Platform

Kim

2023

IEEE Access

View full text Add to dashboard Cite

The Internet of Medical Things (IoMT) paradigm provides pervasive healthcare services inhome monitoring networks. Nowadays, these services play an imperative part in the life of human beings. However, excessive requirements of health services result in insufficient spectrum resources and service delays. In this study, a novel spectrum allocation scheme is proposed for the IoMT system platform. The main challenge of our scheme is to effectively share the limited spectrum resource while dynamically handling different service requests. To achieve a mutually desirable solution for multiple IoMT devices, our proposed scheme is designed as a bi-level control algorithm using the ideas of multi-agent reinforcement learning (MARL) and the Balakrishnan-G𝑜́mez-Vohra (BGV) solution. At the first level, each IoMT device selects its salient point according to the MARL model. At the second level, the spectrum resource is distributed through the BGV solution, which is implemented by considering the selected salient point of each device. Through the sequential interactions of intelligent devices, our bi-level control approach can effectively guide individual IoMT devices to choose cooperation strategies while optimizing the spectrum allocation process. Finally, numerical results show the effectiveness of our proposed scheme through the comparisons with benchmark protocols. We demonstrate the performance improvement of our method in terms of the normalized device payoff, IoMT system throughput and device fairness.

show abstract

“…If the users do not fetch the content from the neighboring SBSs, it will fetch it from the core network directly. Thus, the problem of each SBSs is to learn an optimal caching policy so as to minimize the expected transmission delay of the network, which is modeled in an MAMAB framework with stateless setting [128]. In this case, the SBS is an agent.…”

Section: B Content Cachingmentioning

confidence: 99%

“…The centralized critic network is deployed at the central core network to estimate an action-value function to train an actor for each SBS network through using all the SBSs' experiences. The evaluation experiments conducted within an ultra-dense network with a large number of BSs reveal that the proposed algorithm can achieve a better fetching cost and cache hit rate compared with non-cooperative multiagent actor-critic [138], (LRU) [131], and DRL [128].…”

Section: B Content Cachingmentioning

confidence: 99%

Applications of Multi-Agent Reinforcement Learning in Future Internet: A Comprehensive Survey

Li¹,

Zhu²,

Luong³

et al. 2021

Preprint

View full text Add to dashboard Cite

Future Internet involves several emerging technologies such as 5G and beyond 5G networks, vehicular networks, unmanned aerial vehicle (UAV) networks, and Internet of Things (IoTs). Moreover, future Internet becomes heterogeneous and decentralized with a large number of involved network entities. Each entity may need to make its local decision to improve the network performance under dynamic and uncertain network environments. Standard learning algorithms such as single-agent Reinforcement Learning (RL) or Deep Reinforcement Learning (DRL) have been recently used to enable each network entity as an agent to learn an optimal decision-making policy adaptively through interacting with the unknown environments. However, such an algorithm fails to model the cooperations or competitions among network entities, and simply treats other entities as a part of the environment that may result in the non-stationarity issue. Multi-agent Reinforcement Learning (MARL) allows each network entity to learn its optimal policy by observing not only the environments, but also other entities' policies. As a result, MARL can significantly improve the learning efficiency of the network entities, and it has been recently used to solve various issues in the emerging networks. In this paper, we thus review the applications of MARL in the emerging networks. In particular, we provide a tutorial of MARL and a comprehensive survey of applications of MARL in next generation Internet. In particular, we first introduce single-agent RL and MARL. Then, we review a number of applications of MARL to solve emerging issues in future Internet. The issues consist of network access, transmit power control, computation offloading, content caching, packet routing, trajectory design for UAV-aided networks, and network security issues. Finally, we discuss the challenges, open issues, and future directions related to the applications of MARL in future Internet.

show abstract

Reinforcement Learning Approaches to Coordination in Cooperative Multi-agent Systems

Cited by 25 publications

References 3 publications

Application of reinforcement learning for security enhancement in cognitive radio networks

Application of reinforcement learning for security enhancement in cognitive radio networks

Learning and Game Based Spectrum Allocation Model for Internet of Medical Things (IoMT) Platform

Applications of Multi-Agent Reinforcement Learning in Future Internet: A Comprehensive Survey

Contact Info

Product

Resources

About