Computer Vision Aided mmWave Beam Alignment in V2X Communications

Abstract

Visual information, captured for example by cameras, can effectively reflect the sizes and locations of the environmental scattering objects, and thereby can be used to infer communications parameters like propagation directions, receiver powers, as well as the blockage status. In this paper, we propose a novel beam alignment framework that leverages images taken by cameras installed at the mobile user. Specifically, we utilize 3D object detection techniques to extract the size and location information of the dynamic vehicles around the mobile user, and design a deep neural network (DNN) to infer the optimal beam pair for transceivers without any pilot signal overhead. Moreover, to avoid performing beam alignment too frequently or too slowly, a beam coherence time (BCT) prediction method is developed based on the vision information. This can effectively improve the transmission rate compared with the beam alignment approach with the fixed BCT. Simulation results show that the proposed vision based beam alignment methods outperform the existing LIDAR and vision based solutions, and demand for much lower hardware cost and communication overhead.

T109 Evaluation of the GEM® Premier™ 5000 with intelligent quality management 2 (IQM®2) at Beijing Tsinghua Changgung Hospital, Tsinghua University (Beijing, China)

“…Specifically, we utilize the intersection over union (IoU) to indicate the degree of the overlap of the two different 3D BBoxes, where the IoU is defined as the ratio of overlap volume to union volume of the two 3D BBoxes. Since all vehicles run on the ground and the azimuths of all vehicles are approximately parallel with the lane direction [5], [8], [14]- [15], i.e., the Y G axis, we…”

“…With the progressive development of intelligent business, such as intelligent transportation and autonomous driving, abundant types of sensors, including Radar, LIDAR and Depth/RGB cameras, will be mounted on the communications equipment or the infrastructures [3]- [4]. These sensors can be explored to assist many communications tasks, such as beam alignment [5]- [8], channel covariance matrix estimation [9], as well as the prediction for blockage, base station (BS) handover, as well as received power [10]- [11]. Among them, the visual perception by cameras has drawn much attention recently, due to its universality, low cost, and high resolution compared with the Radar and LIDAR.…”