Distributed Reinforcement Learning Frameworks for Cooperative Retransmission in Wireless Networks

Naddafzadeh-Shirazi, Ghasem; Kong, Peng‐Yong; Tham, Chen-Khong

doi:10.1109/tvt.2010.2059055

Cited by 38 publications

(25 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The eNB broadcasts f t * Prea,0 , and backlogged IoT devices attempt communication in the tth TTI; 9 Update δ t+1 :=D t RACH,0 −D t−1 RACH,0 . 3) LE-URC for Multiple CE Groups: We slightly revise the introduced single-parameter single-group LE-URC approach (given in Section III.B) to dynamically configure resource for multiple CE groups.…”

Section: Le-urcmentioning

confidence: 99%

Reinforcement Learning for Real-Time Optimization in NB-IoT Networks

Jiang

Deng

Nallanathan

et al. 2019

IEEE J. Select. Areas Commun.

View full text Add to dashboard Cite

NarrowBand-Internet of Things (NB-IoT) is an emerging cellular-based technology that offers a range of flexible configurations for massive IoT radio access from groups of devices with heterogeneous requirements. A configuration specifies the amount of radio resource allocated to each group of devices for random access and for data transmission. Assuming no knowledge of the traffic statistics, there exists an important challenge in "how to determine the configuration that maximizes the long-term average number of served IoT devices at each Transmission Time Interval (TTI) in an online fashion". Given the complexity of searching for optimal configuration, we first develop real-time configuration selection based on the tabular Q-learning (tabular-Q), the Linear Approximation based Q-learning (LA-Q), and the Deep Neural Network based Q-learning (DQN) in the single-parameter single-group scenario. Our results show that the proposed reinforcement learning based approaches considerably outperform the conventional heuristic approaches based on load estimation (LE-URC) in terms of the number of served IoT devices. This result also indicates that LA-Q and DQN can be good alternatives for tabular-Q to achieve almost the same performance with much less training time. We further advance LA-Q and DQN via Actions Aggregation (AA-LA-Q and AA-DQN) and via Cooperative Multi-Agent learning (CMA-DQN) for the multi-parameter multi-group scenario, thereby solve the problem that Q-learning agents do not converge in high-dimensional configurations. In this scenario, the superiority of the proposed Q-learning approaches over the conventional LE-URC approach significantly improves with the increase of configuration dimensions, and the CMA-DQN approach outperforms the other approaches in both throughput and training efficiency. Partnership Project (3GPP) partners have standardized a new radio access technology, namely NarrowBand-IoT (NB-IoT) [1]. NB-IoT is expected to provide reliable wireless access for IoT devices with various

show abstract

Section: Le-urcmentioning

confidence: 99%

Reinforcement Learning for Real-Time Optimization in NB-IoT Networks

Jiang

Deng

Nallanathan

et al. 2019

IEEE J. Select. Areas Commun.

View full text Add to dashboard Cite

show abstract

“…They are different from conventional channel selection strategies by taking advantage of the sense information around external environment to make a decision in the channel selection process. The intelligent methods can be broadly divided into two categories: the game-based category that focuses on applying game theoretical tools to make a real-time decision [5][6][7][12][13][14][15][16][17][18][19][20][21][22][23] and the learning-based category that focuses on introducing reinforcement learning techniques to select optimized resource [8][9][10][11][24][25][26][27][28][29][30][31].…”

Section: Related Workmentioning

confidence: 99%

“…In order to implement self-decision and self-learning, the approaches based on game theory and reinforcement learning have been introduced to improve channel selection or other resource allocation problems, for example, [5][6][7][8][9][10][11]. Game theory has been a powerful tool to model decentralized networks to obtain an equilibrium state.…”

Section: Introductionmentioning

confidence: 99%

“…Nie and Haykin [8] provide a classical reinforcement learning framework to solve the channel assignment problem. Naddafzadeh-Shirazi et al [9] and Zhou et al [10] investigate reinforcement learning schemes to help secondary users capture the state of the primary user and learn the satisfactory feedback to improve its own utility. Zame et al [11] design a statistical count learning scheme to make secondary stations learn from and coordinate their own histories, while simultaneously teaching other stations about these histories of counter.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Optimal Channel Selection Based on Online Decision and Offline Learning in Multichannel Wireless Sensor Networks

Qiao

Zhao

Huang

et al. 2017

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

We propose a channel selection strategy with hybrid architecture, which combines the centralized method and the distributed method to alleviate the overhead of access point and at the same time provide more flexibility in network deployment. By this architecture, we make use of game theory and reinforcement learning to fulfill the optimal channel selection under different communication scenarios. Particularly, when the network can satisfy the requirements of energy and computational costs, the online decision algorithm based on noncooperative game can help each individual sensor node immediately select the optimal channel. Alternatively, when the network cannot satisfy the requirements of energy and computational costs, the offline learning algorithm based on reinforcement learning can help each individual sensor node to learn from its experience and iteratively adjust its behavior toward the expected target. Extensive simulation results validate the effectiveness of our proposal and also prove that higher system throughput can be achieved by our channel selection strategy over the conventional off-policy channel selection approaches.

show abstract

“…It is shown that DVF-based learning outperforms the independent learner-based learning algorithm, especially under the condition of high sensor node densities. In [129], a learning algorithm based on the exchange of both the instantaneous reward and the estimated local state-value is proposed for the joint power control and relay selection in a distributed cooperative network. The proposed learning scheme is featured by weighting over both the instantaneous reward and the estimated local state-value that are shared by the neighbor nodes, and thus is called learning with the Distributed Reward and Value (DRV) function.…”

Section: A Applications Of Distributed Learning Based On the Model Omentioning

confidence: 99%

A Survey on Applications of Model-Free Strategy Learning in Cognitive Wireless Networks

Wang

Kwasinski

Niyato

et al. 2016

IEEE Commun. Surv. Tutorials

View full text Add to dashboard Cite

Abstract-The framework of cognitive wireless radio is expected to endow the wireless devices with the cognitionintelligence ability, with which they can efficiently learn and respond to the dynamic wireless environment. In many practical scenarios, the complexity of network dynamics makes it difficult to determine the network evolution model in advance. As a result, the wireless decision-making entities may face a black-box network control problem and the model-based network management mechanisms will be no longer applicable. In contrast, model-free learning has been considered as an efficient tool for designing control mechanisms when the model of the system environment or the interaction between the decision-making entities is not available as a-priori knowledge. With model-free learning, the decision-making entities adapt their behaviors based on the reinforcement from their interaction with the environment and are able to (implicitly) build the understanding of the system through trial-and-error mechanisms. Such characteristics of model-free learning is highly in accordance with the requirement of cognition-based intelligence for devices in cognitive wireless networks. Recently, model-free learning has been considered as one key implementation approach to adaptive, self-organized network control in cognitive wireless networks. In this paper, we provide a comprehensive survey on the applications of the stateof-the-art model-free learning mechanisms in cognitive wireless networks. According to the system models that those applications are based on, a systematic overview of the learning algorithms in the domains of single-agent system, multi-agent systems and multi-player games is provided. Furthermore, the applications of model-free learning to various problems in cognitive wireless networks are discussed with the focus on how the learning mechanisms help to provide the solutions to these problems and improve the network performance over the existing model-based, non-adaptive methods. Finally, a broad spectrum of challenges and open issues is discussed to offer a guideline for the future research directions.

show abstract

Distributed Reinforcement Learning Frameworks for Cooperative Retransmission in Wireless Networks

Cited by 38 publications

References 10 publications

Reinforcement Learning for Real-Time Optimization in NB-IoT Networks

Reinforcement Learning for Real-Time Optimization in NB-IoT Networks

Optimal Channel Selection Based on Online Decision and Offline Learning in Multichannel Wireless Sensor Networks

A Survey on Applications of Model-Free Strategy Learning in Cognitive Wireless Networks

Contact Info

Product

Resources

About