Going Beyond RF: How AI-enabled Multimodal Beamforming will Shape the NextG Standard

Abstract

Incorporating artificial intelligence and machine learning (AI/ML) methods within the 5G wireless standard promises autonomous network behavior and ultra-low-latency reconfiguration. However, the effort so far has purely focused on learning from radio frequency (RF) signals. Future standards and next-generation (nextG) networks beyond 5G will have two significant evolutions over the state-of-the-art 5G implementations: (i) massive number of antenna elements, scaling up to hundreds-tothousands in number, and (ii) inclusion of AI/ML in the critical path of the network reconfiguration process that can access sensor feeds from a variety of RF and non-RF sources. While the former allows unprecedented flexibility in 'beamforming', where signals combine constructively at a target receiver, the latter enables the network with enhanced situation awareness not captured by a single and isolated data modality. This survey presents a thorough analysis of the different approaches used for beamforming today, focusing on mmWave bands, and then proceeds to make a compelling case for considering non-RF sensor data from multiple modalities, such as LiDAR, Radar, GPS for increasing beamforming directional accuracy and reducing processing time. This so called idea of multimodal beamforming will require deep learning based fusion techniques, which will serve to augment the current RF-only and classical signal processing methods that do not scale well for massive antenna arrays. The survey describes relevant deep learning architectures for multimodal beamforming, identifies computational challenges and the role of edge computing in this process, dataset generation tools, and finally, lists open challenges that the community should tackle to realize this transformative vision of the future of beamforming.

A Literature Survey on AI-Aided Beamforming and Beam Management for 5G and 6G Systems

“…Unlike traditional CSI-based methods, optical sensor-aided beamforming methods do not require CSI measurements, and they can also simultaneously decide the best beams for both transmitter and receiving devices. In addition, the accuracy of those methods can be improved by adding GPS information or fusing it with optical and CSI data [208].…”

A Literature Survey on AI-Aided Beamforming and Beam Management for 5G and 6G Systems

“…Unlike traditional CSI-based methods, optical sensor-aided beamforming methods do not require CSI measurements, and they can also simultaneously decide the best beams for both transmitter and receiving devices. In addition, the accuracy of those methods can be improved by adding GPS information or fusing it with optical and CSI data [207].…”

Leveraging deep learning on multimodal sensor data for wireless communication: from mmWave beamforming to digital twins

“…While exhaustive searching through all candidate options ensures the beam alignment, the typical time to complete the entire procedure is in the order of ⇠10 ms for IEEE 802.11ad [8] and ⇠5 ms for 5G-NR [23] with only 30 beam pairs, respectively. To address this, we propose a beam selection framework that uses out-of-band multimodal data to identify a subset of candidate beams, which are subsequently swept to select the one that maximizes the normalized signal power [42].…”