Deep Learning on Visual and Location Data for V2I mmWave Beamforming

Reus-Muns, Guillem; Salehi, Batool; Roy, Debashri; Jian, Tong; Wang, Zifeng; Dy, Jennifer G.; Ioannidis, Stratis; Chowdhury, Kaushik R.

doi:10.1109/msn53354.2021.00087

Cited by 14 publications

(5 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Given this possible high amount of data and the increasing dif iculty of the several tasks performed in the network, ML techniques have been adopted in several works [7,8], especially Deep Neural Networks (DNNs). DNNs are optimized for an application, in general, with supervised learning approaches, which may require a prohibitive amount of data, depending on the model being trained.…”

Section: Introductionmentioning

confidence: 99%

Simultaneous beam selection and users scheduling evaluation in a virtual world with reinforcement learning

Correa¹,

Oliveira²,

Du³

et al. 2022

ITU J-FET

View full text Add to dashboard Cite

The fifth generation of mobile networks evolved to serve applications with distinct requirements, which results in a high management complexity due to simultaneous real-time tasks. In the physical layer, code words that allow proper data exchange between the Base Station (BS) and the served users must be chosen. While, in higher layers, the BS must choose users to be served in a given transmission opportunity. There are approaches based on Machine Learning (ML) to solve these combined tasks. However, due to the high amount of possible inputs, a challenge is the availability of data to train the models. In some cases, there may not even exist a predefined optimal answer to use as a "label" for supervised approaches. In this paper, we evaluate solutions for the combined problems of beam selection and user scheduling with Reinforcement Learning (RL), which does not need labels, as a solution for problems without a predefined answer. The algorithms were proposed for Problem Statement 6 of the challenge organized by the International Telecommunication Union (ITU) in 2021, which ranked as the finalists. We compare the approaches in relation to the cumulative reward received by the agents and show a performance comparison of different RL approaches by comparing them with baselines developed for the challenge. The paper also shows how the action taken by the trained agents affect network operation by comparing the number of packets transmitted, which is highly related to the proper selection of users and code words.

show abstract

Section: Introductionmentioning

confidence: 99%

Simultaneous beam selection and users scheduling evaluation in a virtual world with reinforcement learning

Correa¹,

Oliveira²,

Du³

et al. 2022

ITU J-FET

View full text Add to dashboard Cite

show abstract

“…They extended the ViWi-BT challenge dataset into two datasets presenting the blockage-prediction and object-detection datasets. Reus-Muns et al utilized in [12] spatial information of mobile users such as location, speed, and surrounding scene images and proposed a method called channel covariance matrix to estimate the moving region containing all possible user locations at any given time and later they introduced deep learning based denoising method to reduce the error of the user locations not estimated accurately. In [13], Roy et al presented a survey related to deep learning-based fusion framework leveraging the GPS location information with the combination of the visual data.…”

Section: Introductionmentioning

confidence: 99%

On the Vision-Beam Aided Tracking for Wireless 5G-Beyond Networks Using Long Short-Term Memory with Soft Attention Mechanism

Sinani

Yılmaz

2022

EJRnD

View full text Add to dashboard Cite

The growth of 5G technology and the continuous success of deep learning for various computer vision tasks in healthcare, self-driving cars, visual recognition, and many other areas, brought new challenges in the field of wireless communication. Moreover, 5G-Beyond networks primarily rely on how to maintain line-of-sight (LOS) links between base stations and mobile users. As such, one of the main challenges in 5G-Beyond networks is how to proactively maintain the hand-over mechanism for mobile users before blockages prevent mobile users from communicating, so as to avoid the latency of searching the best beamforming for the best performance. Accordingly, vision-aided millimeter-wave (mmWave) beam and blockage prediction has opened the door for new research for proactive hand-off and resource allocation. The purpose of this paper is to study wireless beam tracking on mmWave bands using deep learning approach evaluated on the Vision-Wireless ViWi-BT dataset [1]. We present how to predict future beam sequences from previously observed beam sequences and images using a long short-term memory (LSTM) network as a base predictive method. As such, we utilize the soft attention mechanism to intelligently choose the most important features and thus we suggest replacing the softmax attention function with different periodic attention functions to eliminate the gradient vanishing problem.

show abstract

“…The proposed setting by Klautau et al [37] and Dias et al [38] comes closest to ours with GPS and LiDAR being used as the side information for LOS detection and also reducing the overhead in a vehicular setting. On the other hand, Muns et al [39] use GPS and camera images to speed up the beam selection with a focus on designing preprocessing step for images and fusion scheme.…”

Section: With Sensor Fusionmentioning

confidence: 99%

“…In [70] Klautau et al [37] and Dias et al [38] propose to reduce the sector search space using GPS and LiDAR sensors in vehicular settings. On the other hand, Muns et al [72] use GPS and camera images to speed up the beam selection. Nevertheless, none of this literature considers real-world experiments on live sensor data.…”

Section: Related Workmentioning

confidence: 99%

“…Multimodal methods design fusion architectures to combine information from multiple sensors [77,103] with reported accuracies of up to 56-88% and slightly higher inference time due to complicated fusion designs. Muns et al [72] use GPS and images to speed up the beam selection, while Salehi et al [104] extend the sensor suite to GPS, image and LiDAR. However, these designs are based on strong assumptions, requiring all vehicles to have the same three sensors on-board and do not offer a general solution to allow inputs from different combinations of sensors.…”

Section: Background and Related Work 621 Background On Conventional E...mentioning

confidence: 99%

See 1 more Smart Citation

Leveraging deep learning on multimodal sensor data for wireless communication: from mmWave beamforming to digital twins

Salehihikouei

View full text Add to dashboard Cite

show abstract

Deep Learning on Visual and Location Data for V2I mmWave Beamforming

Cited by 14 publications

References 21 publications

Simultaneous beam selection and users scheduling evaluation in a virtual world with reinforcement learning

Simultaneous beam selection and users scheduling evaluation in a virtual world with reinforcement learning

On the Vision-Beam Aided Tracking for Wireless 5G-Beyond Networks Using Long Short-Term Memory with Soft Attention Mechanism

Leveraging deep learning on multimodal sensor data for wireless communication: from mmWave beamforming to digital twins

Contact Info

Product

Resources

About