Millimeter Wave Base Stations with Cameras: Vision Aided Beam and Blockage Prediction

Alrabeiah, Muhammad; Hredzak, Andrew; Alkhateeb, Ahmed

doi:10.48550/arxiv.1911.06255

Cited by 8 publications

(16 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Again, despite its appeal, it still falls short in meeting the latency and reliability requirements as the sensory data are only expressive of stationary blockages. On a different note, the work in [19] explores a new dimension for blockage prediction in single user communication settings. It proposes a modified residual network [20] that uses visual data to predict stationary blockages.…”

Section: A Prior Workmentioning

confidence: 99%

“…In this paper and inspired by the recently proposed Vision-Aided Wireless Communication (VAWC) framework in [19] and [21], the link-blockage and user hand-off problems are addressed from a proactive perspective. Images and video sequences usually speak volumes about the environment they depict, and this is supported by the empirical evidence in [19]. As such, this work develops a deep neural network that learns proactive blockage prediction from sequences of jointly observed mmWave beams and video frames.…”

Section: B Contributionmentioning

confidence: 99%

“…This work aims to predict future link blockages using deep learning algorithms and a fusion of both vision and wireless data. As we progress from a single-user and stationary blockage [19] to a more realistic scenario with multiple moving objects and dynamic blockages, the task of future blockage prediction becomes far more challenging. A successful prediction of future link blockages in a realistic scene hinges, to a large extent, on the following two notions.…”

Section: A Blockage Prediction: Key Ideamentioning

confidence: 99%

See 2 more Smart Citations

Vision-Aided 6G Wireless Communications: Blockage Prediction and Proactive Handoff

Charan¹,

Alrabeiah²,

Alkhateeb³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

The sensitivity to blockages is a key challenge for the high-frequency (5G millimeter wave and 6G sub-terahertz) wireless networks. Since these networks mainly rely on line-of-sight (LOS) links, sudden link blockages highly threaten the reliability of the networks. Further, when the LOS link is blocked, the network typically needs to hand off the user to another LOS basestation, which may incur critical time latency, especially if a search over a large codebook of narrow beams is needed. A promising way to tackle the reliability and latency challenges lies in enabling proaction in wireless networks. Proaction basically allows the network to anticipate blockages, especially dynamic blockages, and initiate user hand-off beforehand. This paper presents a complete machine learning framework for enabling proaction in wireless networks relying on visual data captured, for example, by RGB cameras deployed at the base stations. In particular, the paper proposes a vision-aided wireless communication solution that utilizes bimodal machine learning to perform proactive blockage prediction and user hand-off. The bedrock of this solution is a deep learning algorithm that learns from visual and wireless data how to predict incoming blockages. The predictions of this algorithm are used by the wireless network to proactively initiate hand-off decisions and avoid any unnecessary latency. The algorithm is developed on a vision-wireless dataset generated using the ViWi data-generation framework. Experimental results on two basestations with different cameras indicate that the algorithm is capable of accurately detecting incoming blockages more than ∼ 90% of the time. Such blockage prediction ability is directly reflected in the accuracy of proactive hand-off, which also approaches 87%. This highlights a promising direction for enabling high reliability and low latency in future wireless networks.

show abstract

Section: A Prior Workmentioning

confidence: 99%

Section: B Contributionmentioning

confidence: 99%

Section: A Blockage Prediction: Key Ideamentioning

confidence: 99%

See 1 more Smart Citation

Vision-Aided 6G Wireless Communications: Blockage Prediction and Proactive Handoff

Charan¹,

Alrabeiah²,

Alkhateeb³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Interestingly, however, as the wireless systems move to higher frequency bands, the communication links between the transmitters and receivers become shorter with (visual) line-of-sight. This motivates leveraging visual data captured for example by RGB/depth cameras or Lidar to help overcome key challenges such as mmWave beam and blockage prediction-realizing what we call vision-aided wireless communications [1], [4].…”

Section: Introductionmentioning

confidence: 99%

“…With this motivation, we developed the Vision-Wireless dataset generation framework (ViWi) [1], which is a fremwork for generating co-existing and prametric datasets for wireless data (such as communication and radar channels) and visual data (such LiDAR as RGB/depth images). In the first ViWi release [1], we presented 4 ViWi scenarios/datasets that can enable a set of machine learning tasks/applications such as mmWave beam and blockage prediction as shown in [4]. The scenarios adopted in the initial ViWi release, however, were relatively simple compared to practical outdoor and indoor environments.…”

Section: Introductionmentioning

confidence: 99%

ViWi Vision-Aided mmWave Beam Tracking: Dataset, Task, and Baseline Solutions

Alrabeiah¹,

Booth²,

Hredzak³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Vision-aided wireless communication is motivated by the recent advances in deep learning and computer vision as well as the increasing dependence on line-of-sight links in millimeter wave (mmWave) and terahertz systems. By leveraging vision, this new research direction enables an interesting set of new capabilities such as vision-aided mmWave beam and blockage prediction, proactive hand-off, and resource allocation among others. These capabilities have the potential of reliably supporting highly-mobile applications such as vehicular/drone communications and wireless virtual/augmented reality in mmWave and terahertz systems. Investigating these interesting applications, however, requires the development of special dataset and machine learning tasks. Based on the Vision-Wireless (ViWi) dataset generation framework [1], this paper develops an advanced and realistic scenario/dataset that features multiple base stations, mobile users, and rich dynamics. Enabled by this dataset, the paper defines the vision-wireless mmWave beam tracking task (ViWi-BT) and proposes a baseline solution that can provide an initial benchmark for the future ViWi-BT algorithms.

show abstract

Blockage Prediction Using Wireless Signatures: Deep Learning Enables Real-World Demonstration

Alrabeiah

Chakrabarti

et al. 2022

IEEE Open J. Commun. Soc.

Self Cite

View full text Add to dashboard Cite

Overcoming the link blockage challenges is essential for enhancing the reliability and latency of millimeter wave (mmWave) and sub-terahertz (sub-THz) communication networks. Previous approaches relied mainly on either (i) multiple-connectivity, which under-utilizes the network resources, or on (ii) the use of out-of-band and non-RF sensors to predict link blockages, which is associated with increased cost and system complexity. In this paper, we propose a novel solution that relies only on in-band mmWave wireless measurements to proactively predict future dynamic line of sight (LOS) link blockages. The proposed solution utilizes deep neural networks and special patterns of received signal power, that we call pre-blockage wireless signatures to infer future blockages. Specifically, the developed machine learning models attempt to predict: (i) If a future blockage will occur? (ii) When will this blockage happen? (iii) What is the type of the blockage? And (iv) what is the direction of the moving blockage? To evaluate our proposed approach, we build a large-scale real-world dataset comprising nearly 0.5 million data points (mmWave measurements) for both indoor and outdoor blockage scenarios. The results, using this dataset, show that the proposed approach can successfully predict the occurrence of future dynamic blockages with more than 85% accuracy. Further, for the outdoor scenario with highly-mobile vehicular blockages, the proposed model can predict the exact time of the future blockage with less than 100 ms error for blockages happening within the future 600 ms. These results, among others, highlight the promising gains of the proposed proactive blockage prediction solution which could potentially enhance the reliability and latency of future wireless networks.INDEX TERMS Millimeter wave, machine learning, dynamic blockage prediction, wireless signatures.

show abstract

Millimeter Wave Base Stations with Cameras: Vision Aided Beam and Blockage Prediction

Cited by 8 publications

References 11 publications

Vision-Aided 6G Wireless Communications: Blockage Prediction and Proactive Handoff

Vision-Aided 6G Wireless Communications: Blockage Prediction and Proactive Handoff

ViWi Vision-Aided mmWave Beam Tracking: Dataset, Task, and Baseline Solutions

Blockage Prediction Using Wireless Signatures: Deep Learning Enables Real-World Demonstration

Contact Info

Product

Resources

About