Mobility-Aware Edge Caching and Computing in Vehicle Networks: A Deep Reinforcement Learning

Tan, Le Thanh; Hu, Rose Qingyang

doi:10.1109/tvt.2018.2867191

Cited by 330 publications

(157 citation statements)

References 42 publications

Supporting

Mentioning

148

Contrasting

Order By: Relevance

“…Considering the huge action space and high complexity with the vehicle's mobility and service delay deadline T d , a multitime scale DQN framework is proposed in [88] to minimize the system cost by the joint design of communication, caching and computing in VANET. The policy design accounts for limited storage capacities and computational resources at the vehicles and the RSUs.…”

Section: A Wireless Proactive Cachingmentioning

confidence: 99%

Applications of Deep Reinforcement Learning in Communications and Networking: A Survey

Luong

Hoang

Gong

et al. 2019

IEEE Commun. Surv. Tutorials

1,401

481

View full text Add to dashboard Cite

This paper presents a comprehensive literature review on applications of deep reinforcement learning in communications and networking. Modern networks, e.g., Internet of Things (IoT) and Unmanned Aerial Vehicle (UAV) networks, become more decentralized and autonomous. In such networks, network entities need to make decisions locally to maximize the network performance under uncertainty of network environment. Reinforcement learning has been efficiently used to enable the network entities to obtain the optimal policy including, e.g., decisions or actions, given their states when the state and action spaces are small. However, in complex and large-scale networks, the state and action spaces are usually large, and the reinforcement learning may not be able to find the optimal policy in reasonable time. Therefore, deep reinforcement learning, a combination of reinforcement learning with deep learning, has been developed to overcome the shortcomings. In this survey, we first give a tutorial of deep reinforcement learning from fundamental concepts to advanced models. Then, we review deep reinforcement learning approaches proposed to address emerging issues in communications and networking. The issues include dynamic network access, data rate control, wireless caching, data offloading, network security, and connectivity preservation which are all important to next generation networks such as 5G and beyond. Furthermore, we present applications of deep reinforcement learning for traffic routing, resource sharing, and data collection. Finally, we highlight important challenges, open issues, and future research directions of applying deep reinforcement learning.

show abstract

Section: A Wireless Proactive Cachingmentioning

confidence: 99%

Applications of Deep Reinforcement Learning in Communications and Networking: A Survey

Luong

Hoang

Gong

et al. 2019

IEEE Commun. Surv. Tutorials

1,401

481

View full text Add to dashboard Cite

show abstract

“…In this case, how to effectively allocate the limited resources is an important problem. Focusing on resource allocation in the integrated architecture, two joint optimization models were formulated in [77], [78] to determine the optimal caching and computing decisions, which were then solved by deep reinforcement learning based methods.…”

Section: A Vehicle As a Clientmentioning

confidence: 99%

Mobile Edge Intelligence and Computing for the Internet of Vehicles

2020

View full text Add to dashboard Cite

The Internet of Vehicles (IoV) is an emerging paradigm, driven by recent advancements in vehicular communications and networking. Advances in research can now provide reliable communication links between vehicles, via vehicle-to-vehicle communications, and between vehicles and roadside infrastructures, via vehicle-to-infrastructure communications. Meanwhile, the capability and intelligence of vehicles are being rapidly enhanced, and this will have the potential of supporting a plethora of new exciting applications, which will integrate fully autonomous vehicles, the Internet of Things (IoT), and the environment. These trends will bring about an era of intelligent IoV, which will heavily depend upon communications, computing, and data analytics technologies. To store and process the massive amount of data generated by intelligent IoV, onboard processing and Cloud computing will not be sufficient, due to resource/power constraints and communication overhead/latency, respectively. By deploying storage and computing resources at the wireless network edge, e.g., radio access points, the edge information system (EIS), including edge caching, edge computing, and edge AI, will play a key role in the future intelligent IoV. Such system will provide not only low-latency content delivery and computation services, but also localized data acquisition, aggregation and processing. This article surveys the latest development in EIS for intelligent IoV. Key design issues, methodologies and hardware platforms are introduced. In particular, typical use cases for intelligent vehicles are illustrated, including edge-assisted perception, mapping, and localization. In addition, various open research problems are identified.

show abstract

“…In general, cache status b f,u or cache placement depends on several factors, such as user behavior, information popularity distribution and so on, and it can be decided by several advanced cache schemes, e.g., artificial intelligencebased multi-timescale framework method [32] and deep Qlearning method [37]. Here, we assume that the cache status b f,u has been fixed according to a certain cache strategy, and the required files by the actuators, i.e., c k, f are also given in advance [19], [21].…”

Section: B Problem Formulationmentioning

confidence: 99%

Edge Cache-Assisted Secure Low-Latency Millimeter-Wave Transmission

Hao

Zeng

Sun

et al. 2020

IEEE Internet Things J.

View full text Add to dashboard Cite

In this paper, we consider an edge cache-assisted millimeter wave cloud radio access network (C-RAN). Each remote radio head (RRH) in the C-RAN has a local cache, which can prefetch and store the files requested by the actuators. Multiple RRHs form a cluster to cooperatively serve the actuators, which acquire their required files either from the local caches or from the central processor via multicast fronthaul links. For such a scenario, we formulate a beamforming design problem to minimize the secure transmission delay under transmit power constraint of each RRH. Due to the difficulty of directly solving the formulated problem, we divide it into two independent ones: i) minimizing the fronthaul transmission delay by jointly optimizing the transmit and receive beamforming; ii) minimizing the maximum access transmission delay by jointly designing cooperative beamforming among RRHs. An alternatively iterative algorithm is proposed to solve the first optimization problem. For the latter, we first design the analog beamforming based on the channel state information of the actuators. Then, with the aid of successive convex approximation and S -procedure techniques, a semidefinite program (SDP) is formulated, and an iterative algorithm is proposed through SDP relaxation. Finally, simulation results are provided to verify the performance of the proposed schemes.

show abstract

Mobility-Aware Edge Caching and Computing in Vehicle Networks: A Deep Reinforcement Learning

Cited by 330 publications

References 42 publications

Applications of Deep Reinforcement Learning in Communications and Networking: A Survey

Applications of Deep Reinforcement Learning in Communications and Networking: A Survey

Mobile Edge Intelligence and Computing for the Internet of Vehicles

Edge Cache-Assisted Secure Low-Latency Millimeter-Wave Transmission

Contact Info

Product

Resources

About