Vision-Position Multi-Modal Beam Prediction Using Real Millimeter Wave Datasets

Charan, Gouranga; Osman, Tawfik; Hredzak, Andrew; Thawdar, Ngwe; Alkhateeb, Ahmed

doi:10.1109/wcnc51071.2022.9771835

Cited by 44 publications

(16 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Blockage prediction and BS handover prediction Requirement for MS identification and privacy concerns [14]- [15] Beam tracking [16] Beam alignment [18] Camera at MS…”

Section: Rgb Imagesmentioning

confidence: 99%

“…In [14]- [15], the previous images taken at BS and the beam sequences are utilized for beam tracking. In [16], a multi-modal beam alignment method is proposed by the conjunctive use of the images taken at BS and the location coordinates of MS.…”

Section: Introductionmentioning

confidence: 99%

“…Nevertheless, if BS is obliged to take images, then additional characteristic information of the MS is required by BS to identify the MS from all detected objects inside one image, i.e., to match the MS's radio signal with the corresponding visual information [17]. Therefore, in [13]- [16], the BS needs to rely on the previous beam sequences or the location fed back by MS to achieve the blockage/handover prediction or beam tracking/selection, which leads to inevitable communication overhead. When the MS is blocked by the surrounding objects in the camera view of BS, the visual information of the MS will be lost to cause the degradation of the beamforming performance.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Computer Vision Aided mmWave Beam Alignment in V2X Communications

Xu¹,

Gao²,

Tao³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Visual information, captured for example by cameras, can effectively reflect the sizes and locations of the environmental scattering objects, and thereby can be used to infer communications parameters like propagation directions, receiver powers, as well as the blockage status. In this paper, we propose a novel beam alignment framework that leverages images taken by cameras installed at the mobile user. Specifically, we utilize 3D object detection techniques to extract the size and location information of the dynamic vehicles around the mobile user, and design a deep neural network (DNN) to infer the optimal beam pair for transceivers without any pilot signal overhead. Moreover, to avoid performing beam alignment too frequently or too slowly, a beam coherence time (BCT) prediction method is developed based on the vision information. This can effectively improve the transmission rate compared with the beam alignment approach with the fixed BCT. Simulation results show that the proposed vision based beam alignment methods outperform the existing LIDAR and vision based solutions, and demand for much lower hardware cost and communication overhead.

show abstract

“…Blockage prediction and BS handover prediction Requirement for MS identification and privacy concerns [14]- [15] Beam tracking [16] Beam alignment [18] Camera at MS…”

Section: Rgb Imagesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Computer Vision Aided mmWave Beam Alignment in V2X Communications

Xu¹,

Gao²,

Tao³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Although the solutions can help in reducing the training overhead, relying only on location alone might result in inaccurate predictions due to the inherent errors associated with the GPS data. In [11], [12], we proposed to leverage the visual data (captured by cameras)to predict the optimal beam indices. These solutions, however, are based on synthetic data and focused on scenarios with humans, vehicles, or robots as the transmitter, where the users typically move in easy to predict mobility patterns in two dimensions.…”

Section: Introductionmentioning

confidence: 99%

“…Drones or UAVs have six degrees of freedom, three translation, and three rotation, which further increases the challenge of predicting the optimal beam index. An important question that arises is whether the promising results in [11], [12] can be achieved in reality for mmWave drones?…”

Section: Introductionmentioning

confidence: 99%

Towards Real-World 6G Drone Communication: Position and Camera Aided Beam Prediction

Charan¹,

Hredzak²,

Stoddard³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Millimeter-wave (mmWave) and terahertz (THz) communication systems typically deploy large antenna arrays to guarantee sufficient receive signal power. The beam training overhead associated with these arrays, however, make it hard for these systems to support highly-mobile applications such as drone communication. To overcome this challenge, this paper proposes a machine learning-based approach that leverages additional sensory data, such as visual and positional data, for fast and accurate mmWave/THz beam prediction. The developed framework is evaluated on a real-world multi-modal mmWave drone communication dataset comprising of co-existing camera, practical GPS, and mmWave beam training data. The proposed sensing-aided solution achieves a top-1 beam prediction accuracy of 86.32% and close to 100% top-3 and top-5 accuracies, while considerably reducing the beam training overhead. This highlights a promising solution for enabling highly mobile 6G drone communications.

show abstract