Intelligent Resource Management Based on Reinforcement Learning for Ultra-Reliable and Low-Latency IoV Communication Networks

Yang, Helin; Xie, Xianzhong; Kadoch, Michel

doi:10.1109/tvt.2018.2890686

Cited by 154 publications

(90 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A joint problem of transmission mode selection, RB allocation and power control for cellular V2X communications is formulated to maximize the sum capacity of V2I users while ensuring the latency and reliability requirements of V2V pairs. Different from [24], resource sharing among V2V pairs in different transmission modes is considered. • decentralized algorithms and achieves competitive performance compared to a centralized algorithm.…”

Section: B Contributions and Organizationmentioning

confidence: 99%

Deep-Reinforcement-Learning-Based Mode Selection and Resource Allocation for Cellular V2X Communications

Zhang

Peng

Yan

et al. 2020

IEEE Internet Things J.

191

View full text Add to dashboard Cite

Cellular vehicle-to-everything (V2X) communication is crucial to support future diverse vehicular applications. However, for safety-critical applications, unstable vehicle-to-vehicle (V2V) links and high signalling overhead of centralized resource allocation approaches become bottlenecks. In this paper, we investigate a joint optimization problem of transmission mode selection and resource allocation for cellular V2X communications. In particular, the problem is formulated as a Markov decision process, and a deep reinforcement learning (DRL) based decentralized algorithm is proposed to maximize the sum capacity of vehicle-to-infrastructure users while meeting the latency and reliability requirements of V2V pairs. Moreover, considering training limitation of local DRL models, a two-timescale federated DRL algorithm is developed to help obtain robust model. Wherein, the graph theory based vehicle clustering algorithm is executed on a large timescale and in turn the federated learning algorithm is conducted on a small timescale. Simulation results show that the proposed DRL-based algorithm outperforms other decentralized baselines, and validate the superiority of the twotimescale federated DRL algorithm for newly activated V2V pairs.

show abstract

Section: B Contributions and Organizationmentioning

confidence: 99%

Deep-Reinforcement-Learning-Based Mode Selection and Resource Allocation for Cellular V2X Communications

Zhang

Peng

Yan

et al. 2020

IEEE Internet Things J.

191

View full text Add to dashboard Cite

show abstract

“…A distributed Q-learning based spectrum allocation scheme has been proposed in [21], where D2D users learn the wireless environment and select spectrum resources autonomously to maximize their throughput while causing minimum interference to the cellular users. Since Q-learning has low convergence speed and may not always suitable to deal with continuous valued state and action spaces, an efficient transfer actor-critic (AC) RL approach has been proposed in [22] to address the intelligent resource management problem in a D2D-based Internet of Vehicle (IoV) networks. The above works can only be applied to low-dimensional state-action mapping.…”

Section: Arxiv:191209302v1 [Csni] 18 Dec 2019mentioning

confidence: 99%

Multi-Agent Deep Reinforcement Learning Based Spectrum Allocation for D2D Underlay Communications

Guo

2020

IEEE Trans. Veh. Technol.

114

View full text Add to dashboard Cite

Device-to-device (D2D) communication underlay cellular networks is a promising technique to improve spectrum efficiency. In this situation, D2D transmission may cause severe interference to both the cellular and other D2D links, which imposes a great technical challenge to spectrum allocation. Existing centralized schemes require global information, which causes a large signaling overhead. While existing distributed schemes requires frequent information exchange among D2D users and cannot achieve global optimization. In this paper, a distributed spectrum allocation framework based on multiagent deep reinforcement learning is proposed, named multiagent actor critic (MAAC). MAAC shares global historical states, actions and policies during centralized training, requires no signal interaction during execution and utilizes cooperation among users to further optimize system performance. Moreover, in order to decrease the computing complexity of the training, we further propose the neighbor-agent actor critic (NAAC) based on the neighbor users' historical information for centralized training. The simulation results show that the proposed MAAC and NAAC can effectively reduce the outage probability of cellular links, greatly improve the sum rate of D2D links and converge quickly.Index Terms-Device-to-device (D2D) communications, multiagent deep reinforcement learning, spectrum allocation.

show abstract

“…Mobile, extreme low-latency Charging price Cellular, aerial [170], [171] The real-time market frequently updates the locational marginal price to match the supply and demand, usually every five minute [172]. The peak power shifting tasks are also transmitted to the AGs to help alleviate the loading pressure.…”

Section: Mobile Evsmentioning

confidence: 99%

Energy and Information Management of Electric Vehicular Network: A Survey

Chen

Wang

Zhang

et al. 2020

IEEE Commun. Surv. Tutorials

View full text Add to dashboard Cite

The connected vehicle paradigm empowers vehicles with the capability to communicate with neighboring vehicles and infrastructure, shifting the role of vehicles from a transportation tool to an intelligent service platform. Meanwhile, the transportation electrification pushes forward the electric vehicle (EV) commercialization to reduce the greenhouse gas emission by petroleum combustion. The unstoppable trends of connected vehicle and EVs transform the traditional vehicular system to an electric vehicular network (EVN), a clean, mobile, and safe system. However, due to the mobility and heterogeneity of the EVN, improper management of the network could result in charging overload and data congestion. Thus, energy and information management of the EVN should be carefully studied. In this paper, we provide a comprehensive survey on the deployment and management of EVN considering all three aspects of energy flow, data communication, and computation. We first introduce the management framework of EVN. Then, research works on the EV aggregator (AG) deployment are reviewed to provide energy and information infrastructure for the EVN. Based on the deployed AGs, we present the research work review on EV scheduling that includes both charging and vehicle-to-grid (V2G) scheduling. Moreover, related works on information communication and computing are surveyed under each scenario. Finally, we discuss open research issues in the EVN.

show abstract

Intelligent Resource Management Based on Reinforcement Learning for Ultra-Reliable and Low-Latency IoV Communication Networks

Cited by 154 publications

References 43 publications

Deep-Reinforcement-Learning-Based Mode Selection and Resource Allocation for Cellular V2X Communications

Deep-Reinforcement-Learning-Based Mode Selection and Resource Allocation for Cellular V2X Communications

Multi-Agent Deep Reinforcement Learning Based Spectrum Allocation for D2D Underlay Communications

Energy and Information Management of Electric Vehicular Network: A Survey

Contact Info

Product

Resources

About