Machine Learning on Camera Images for Fast mmWave Beamforming

Salehi, Batool; Belgiovine, Mauro; Sánchez, Sara García; Dy, Jennifer G.; Ioannidis, Stratis; Chowdhury, Kaushik R.

doi:10.1109/mass50613.2020.00049

Cited by 18 publications

(11 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To perform BSS, a deep CNN is used in [113] to classify the narrow and strongly focused beams with high reliability and low complexity. The reliable specification of transceiver locations is achieved in [114] using two CNNs in less time and high accuracy when compared to a deterministic method. Apart from the above topics, CNNs have successfully been applied for power allocation, uplink beamforming prediction, and SRM, with performance similar to that of conventional methods [115].…”

Section: Antenna Beamforming Combined With Massive Mimomentioning

confidence: 99%

A Review of the State of the Art and Future Challenges of Deep Learning-Based Beamforming

et al. 2022

View full text Add to dashboard Cite

The key objective of this paper is to explore the recent state-of-the-art artificial intelligence (AI) applications on the broad field of beamforming. Hence, a multitude of AI-oriented beamforming studies are thoroughly investigated in order to correctly comprehend and profitably interpret the AI contribution in the beamforming performance. Starting from a brief overview of beamforming, including adaptive beamforming algorithms and direction of arrival (DOA) estimation methods, our analysis probes further into the main machine learning (ML) classes, the basic neural network (NN) topologies, and the most efficient deep learning (DL) schemes. Subsequently, and based on the prior aspects, the paper explores several concepts regarding the optimal use of ML and NNs either as standalone beamforming and DOA estimation techniques or in combination with other implementations, such as ultrasound imaging, massive multipleinput multiple-output structures, and intelligent reflecting surfaces. Finally, particular attention is drawn on the realization of beamforming or DOA estimation setups via DL topologies. The survey closes with various important conclusions along with an interesting discussion on potential future aspects and promising research challenges.

show abstract

Section: Antenna Beamforming Combined With Massive Mimomentioning

confidence: 99%

A Review of the State of the Art and Future Challenges of Deep Learning-Based Beamforming

et al. 2022

View full text Add to dashboard Cite

show abstract

“…3) Image-based: This dataset is obtained by Salehi et al in [92] from a testbed composed of two Sibeam mmWave [29] antenna arrays mounted on sliders enabling horizontal movement. Using the mmWave transceivers from National Instruments, the mutual channel is measured for 13 beam directions at transmitter and receiver (169 beam configurations overall).…”

Section: B Datasetsmentioning

confidence: 99%

“…Results rreveal that the proposed 3D scene based beam selection outperforms LiDAR in accuracy, without imposing the huge cost of LiDAR sensor. While the majority of current literature uses synthetic datasets, the authors in [92] deploy a testbed using National Instruments radio at 60 GHz [29] and camera generated images to predict the best beam configuration. Their proposed method consists of two main steps, namely detection and prediction.…”

Section: Single Non-rf Modalitiesmentioning

confidence: 99%

Going Beyond RF: How AI-enabled Multimodal Beamforming will Shape the NextG Standard

Roy¹,

Salehi²,

Banou³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Incorporating artificial intelligence and machine learning (AI/ML) methods within the 5G wireless standard promises autonomous network behavior and ultra-low-latency reconfiguration. However, the effort so far has purely focused on learning from radio frequency (RF) signals. Future standards and next-generation (nextG) networks beyond 5G will have two significant evolutions over the state-of-the-art 5G implementations: (i) massive number of antenna elements, scaling up to hundreds-tothousands in number, and (ii) inclusion of AI/ML in the critical path of the network reconfiguration process that can access sensor feeds from a variety of RF and non-RF sources. While the former allows unprecedented flexibility in 'beamforming', where signals combine constructively at a target receiver, the latter enables the network with enhanced situation awareness not captured by a single and isolated data modality. This survey presents a thorough analysis of the different approaches used for beamforming today, focusing on mmWave bands, and then proceeds to make a compelling case for considering non-RF sensor data from multiple modalities, such as LiDAR, Radar, GPS for increasing beamforming directional accuracy and reducing processing time. This so called idea of multimodal beamforming will require deep learning based fusion techniques, which will serve to augment the current RF-only and classical signal processing methods that do not scale well for massive antenna arrays. The survey describes relevant deep learning architectures for multimodal beamforming, identifies computational challenges and the role of edge computing in this process, dataset generation tools, and finally, lists open challenges that the community should tackle to realize this transformative vision of the future of beamforming.

show abstract

“…Finally, to address the concern that the image preprocessing may introduce significant delay as it requires multiple forward passes, we convert the trained model to an equivalent fully convolutional network. We have previously explored such an approach in [57], which enables us to generate the entire bit map in a single forward pass.…”

Section: Appendix a Object Detection Algorithmmentioning

confidence: 99%

Deep Learning on Multimodal Sensor Data at the Wireless Edge for Vehicular Network

Salehi¹,

Reus-Muns²,

Roy³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Beam selection for millimeter-wave links in a vehicular scenario is a challenging problem, as an exhaustive search among all candidate beam pairs cannot be assuredly completed within short contact times. We solve this problem via a novel expediting beam selection by leveraging multimodal data collected from sensors like LiDAR, camera images, and GPS. We propose individual modality and distributed fusion-based deep learning (F-DL) architectures that can execute locally as well as at a mobile edge computing center (MEC), with a study on associated tradeoffs. We also formulate and solve an optimization problem that considers practical beam-searching, MEC processing and sensor-to-MEC data delivery latency overheads for determining the output dimensions of the above F-DL architectures. Results from extensive evaluations conducted on publicly available synthetic and home-grown real-world datasets reveal 95% and 96% improvement in beam selection speed over classical RF-only beam sweeping, respectively. F-DL also outperforms the state-of-the-art techniques by 20-22% in predicting top-10 best beam pairs.

show abstract

Machine Learning on Camera Images for Fast mmWave Beamforming

Cited by 18 publications

References 14 publications

A Review of the State of the Art and Future Challenges of Deep Learning-Based Beamforming

A Review of the State of the Art and Future Challenges of Deep Learning-Based Beamforming

Going Beyond RF: How AI-enabled Multimodal Beamforming will Shape the NextG Standard

Deep Learning on Multimodal Sensor Data at the Wireless Edge for Vehicular Network

Contact Info

Product

Resources

About