Reinforcement Learning-Based Multiaccess Control and Battery Prediction With Energy Harvesting in IoT Systems

Chu, Man; Li, Hang; Liao, Xuewen; Cui, Shuguang

doi:10.1109/jiot.2018.2872440

Cited by 113 publications

(48 citation statements)

References 39 publications

(78 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, the proposed architecture, algorithm and mechanism are also promising to be applied over other large-scale network control problems in the future networks. For example, for the access control problem in the IoT system studied in [34], one could first cluster the IoT devices into different groups and then determine their access control policies respectively. Meanwhile, the strategy of learning under multiple behavior policies and the safeguard mechanism can be exploited to further improve the learning efficiency and the online performance.…”

Section: Discussionmentioning

confidence: 99%

Load Balancing for Ultradense Networks: A Deep Reinforcement Learning-Based Approach

Wang

et al. 2019

IEEE Internet Things J.

Self Cite

View full text Add to dashboard Cite

In this paper, we propose a deep reinforcement learning (DRL) based mobility load balancing (MLB) algorithm along with a two-layer architecture to solve the large-scale load balancing problem for ultra-dense networks (UDNs). Our contribution is three-fold. First, this work proposes a two-layer architecture to solve the large-scale load balancing problem in a self-organized manner. The proposed architecture can alleviate the global traffic variations by dynamically grouping small cells into self-organized clusters according to their historical loads, and further adapt to local traffic variations through intra-cluster load balancing afterwards. Second, for the intra-cluster load balancing, this paper proposes an off-policy DRL-based MLB algorithm to autonomously learn the optimal MLB policy under an asynchronous parallel learning framework, without any prior knowledge assumed over the underlying UDN environments. Moreover, the algorithm enables joint exploration with multiple behavior policies, such that the traditional MLB methods can be used to guide the learning process thereby improving the learning efficiency and stability. Third, this work proposes an offlineevaluation based safeguard mechanism to ensure that the online system can always operate with the optimal and well-trained MLB policy, which not only stabilizes the online performance but also enables the exploration beyond current policies to make full use of machine learning in a safe way. Empirical results verify that the proposed framework outperforms the existing MLB methods in general UDN environments featured with irregular network topologies, coupled interferences, and random user movements, in terms of the load balancing performance.

show abstract

Section: Discussionmentioning

confidence: 99%

Load Balancing for Ultradense Networks: A Deep Reinforcement Learning-Based Approach

Wang

et al. 2019

IEEE Internet Things J.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Moreover, the users' content request distributions and data correlation will change as time elapses. Traditional learning approaches such as [26] must re-implement the learning process as the users' content request distribution and data correlation change. However, the ESN-based transfer RL algorithm can transform the already learned resource block allocation policy into the new resource block allocation policy that must be learned as the users' content request distribution and data correlation change so as to improve the convergence speed.…”

Section: Echo State Network For Self-organizing Resource Allocamentioning

confidence: 99%

Data Correlation-Aware Resource Management in Wireless Virtual Reality (VR): An Echo State Transfer Learning Approach

Chen

Saad

Yin

et al. 2019

IEEE Trans. Commun.

View full text Add to dashboard Cite

Providing seamless connectivity for wireless virtual reality (VR) users has emerged as a key challenge for future cloud-enabled cellular networks. In this paper, the problem of wireless VR resource management is investigated for a wireless VR network in which VR contents are sent by a cloud to cellular small base stations (SBSs). The SBSs will collect tracking data from the VR users, over the uplink, in order to generate the VR content and transmit it to the end-users using downlink cellular links.For this model, the data requested or transmitted by the users can exhibit correlation, since the VR users may engage in the same immersive virtual environment with different locations and orientations. As such, the proposed resource management framework can factor in such spatial data correlation, so as to better manage uplink and downlink traffic. This potential spatial data correlation can be factored into the resource allocation problem to reduce the traffic load in both uplink and downlink. In the downlink, the cloud can transmit 360 • contents or specific visible contents (e.g., user field of view) that are extracted from the original 360 • contents to the users according to the users' data correlation so as to reduce the backhaul traffic load. In the uplink, each SBS can associate with the users that have similar tracking information so as to reduce the tracking data size. This data correlation-aware resource management problem is formulated as an optimization problem whose goal is to maximize the users' successful A preliminary version of this work was published in [1].

show abstract

“…The procedure described by (14), (15) and (16) is called the fictitious play procedure. As described in (14), at the m th iteration, a node attempts to learn the Nash maximizer, F * m , given that its belief about the distribution of the nodes across the states isπ m .…”

Section: Mf-marl For Distributed Power Controlmentioning

confidence: 99%

“…Next, at the (m + 1) th iteration, each node attempts to learn the Nash maximizer, F * m+1 . A discrete-time MFG is said to have FPP if and only if the procedure described by (14), (15) and (16) converges.…”

Section: Mf-marl For Distributed Power Controlmentioning

confidence: 99%

Distributed Power Control for Large Energy Harvesting Networks: A Multi-Agent Deep Reinforcement Learning Approach

Sharma

Zappone

Assaad

et al. 2019

IEEE Trans. Cogn. Commun. Netw.

View full text Add to dashboard Cite

In this paper, we develop a multi-agent reinforcement learning (MARL) framework to obtain online power control policies for a large energy harvesting (EH) multiple access channel, when only causal information about the EH process and wireless channel is available. In the proposed framework, we model the online power control problem as a discrete-time mean-field game (MFG), and analytically show that the MFG has a unique stationary solution. Next, we leverage the fictitious play property of the mean-field games, and the deep reinforcement learning technique to learn the stationary solution of the game, in a completely distributed fashion. We analytically show that the proposed procedure converges to the unique stationary solution of the MFG. This, in turn, ensures that the optimal policies can be learned in a completely distributed fashion. In order to benchmark the performance of the distributed policies, we also develop a deep neural network (DNN) based centralized as well as distributed online power control schemes. Our simulation results show the efficacy of the proposed power control policies. In particular, the DNN based centralized power control policies provide a very good performance for large EH networks for which the design of optimal policies is intractable using the conventional methods such as Markov decision processes. Further, performance of both the distributed policies is close to the throughput achieved by the centralized policies.

show abstract

Reinforcement Learning-Based Multiaccess Control and Battery Prediction With Energy Harvesting in IoT Systems

Cited by 113 publications

References 39 publications

Load Balancing for Ultradense Networks: A Deep Reinforcement Learning-Based Approach

Load Balancing for Ultradense Networks: A Deep Reinforcement Learning-Based Approach

Data Correlation-Aware Resource Management in Wireless Virtual Reality (VR): An Echo State Transfer Learning Approach

Distributed Power Control for Large Energy Harvesting Networks: A Multi-Agent Deep Reinforcement Learning Approach

Contact Info

Product

Resources

About