Virtual Relay Selection in LTE-V: A Deep Reinforcement Learning Approach to Heterogeneous Data

Abstract

The development of Long Term Evolution (LTE) enables wireless communication with high transmission rate, low latency, and wide coverage area. These outstanding features of LTE support the next generation of vehicle-to-everything (V2X) communication, which is named LTE-V. Among the various technologies in LTE-V, placing relay nodes on vehicles is a promising approach to save power and energy, and extend the transmission range. In this paper, we consider the virtual relay node selection problem. In the problem, a base station transmits data to a vehicle relay (also known as helper) who will further disseminate the received data to the vehicular subscribers nearby. The selection of the vehicle relay node is a challenging issue since the utility of the selection can only be known after this action has been made. Another challenge of this problem is that the traditional pure optimization is inapplicable due to the imperfect information available. Motivated by the recent success of Alpha Go Zero, we employ deep reinforcement learning (DRL) as a powerful tool for facing the above challenges and solving the problem without global information. We build a bridge between the traffic information and decision of relay node selection based on the reality that the utility of vehicle relay is highly correlated with the traffic density. In our work, deep convolutional neural networks are first applied to extract traffic patterns and then learn the traffic and network topology. Deep learning (DL) acts as a role to map features inside traffic and communication topology to the decisions. Then Q-Learning dynamically updates the utilities of decisions by trials in the environment. Finally, the overall rewards can be calculated to measure these decisions, and the Q function is also updated. Simulation results based on real traffic data validate that our proposed approach can achieve high utility performance.

Minimum Latency-Secure Key Transmission for Cloud-Based Internet of Vehicles Using Reinforcement Learning

“…e binding update request mainly contains the cluster vehicle ID of the updated key and the length of the new key x 2 , namely, is paper mainly considers the internal attack on the Internet of Vehicles. e attacker can be a legitimate vehicle Hash function T/T * Send/receive timestamp node in the network, or a controlled malicious vehicle node, which obtains the group key through side-channel attacks such as timing attacks and cache attacks [28], and then implements an electronic spoofing attack [29]. e act of spoofing involves hiding a message or identification so that it looks to be coming from a reliable, approved source.…”

“…This paper uses Python to build a dynamic cluster simulation scenario to verify the advantages of the low-latency IoV group key distribution management technology based on reinforcement learning in the computing delay and communication security level of encryption and decryption. Reference [ 29 ] for the setting of stimulation parameters are as follows: the group leader and vehicles within a range of 300 m form a cluster, the number of vehicle nodes in the initial cluster is 50, and the average speed of vehicles in the cluster v ∈ [0,120] km/h, and the same as that of the cluster vehicles. The number is inversely proportional [ 30 ].…”

On Topology Optimization and Routing in Integrated Access and Backhaul Networks: A Genetic Algorithm-Based Approach

“…For instance, [35]- [44] develop deep reinforcement (DR) algorithm-based solutions for topology optimization of different network configurations. Deep Q-learning is used in [45] to evaluate the cumulative transmission rate in vehicular networks. Spectrum allocation and access mode selection evaluations are considered in [46], while the potential of using K-means clustering algorithm to design ultra-reliable and low-latency wireless sensor networks is evaluated in [47].…”

Joint Buffer-Aided Hybrid-Duplex Relay Selection and Power Allocation for Secure Cognitive Networks With Double Deep Q-Network

“…In [25], a relay selection based on deep reinforcement learning was proposed to enhance the robustness. In [26], the deep Q-Learning was applied to select the virtual vehicle relay node. None of these approaches considers buffers and/or full duplex transmission at the relay, which will significantly increase the searching dimension for the learning, making it harder to converge.…”