ViWi: A Deep Learning Dataset Framework for Vision-Aided Wireless Communications

Alrabeiah, Muhammad; Hredzak, Andrew; Liu, Zhenhao; Alkhateeb, Ahmed

doi:10.1109/vtc2020-spring48590.2020.9128579

Cited by 83 publications

(85 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, the scientific community started exploring the possibility of bringing intelligence from CV systems to radio networks. In [17] the authors presented a framework for generating datasets with visual and radio information to facilitate research towards vision-aided wireless communication. The framework uses a game engine and a wireless propagation tool to recreate outdoor urban scenarios.…”

Section: A Related Workmentioning

confidence: 99%

Vision-Aided Radio: User Identity Match in Radio and Video Domains Using Machine Learning

et al. 2020

View full text Add to dashboard Cite

5G is designed to be an essential enabler and a leading infrastructure provider in the communication technology industry by supporting the demand for the growing data traffic and a variety of services with distinct requirements. The use of deep learning and computer vision tools has the means to increase the environmental awareness of the network with information from visual data. Information extracted via computer vision tools such as user position, movement direction, and speed can be promptly available for the network. However, the network must have a mechanism to match the identity of a user in both visual and radio systems. This mechanism is absent in the present literature. Therefore, we propose a framework to match the information from both visual and radio domains. This is an essential step to practical applications of computer vision tools in communications. We detail the proposed framework training and deployment phases for a presented setup. We carried out practical experiments using data collected in different types of environments. The work compares the use of Deep Neural Network and Random Forest classifiers and shows that the former performed better across all experiments, achieving classification accuracy greater than 99%.

show abstract

Section: A Related Workmentioning

confidence: 99%

Vision-Aided Radio: User Identity Match in Radio and Video Domains Using Machine Learning

et al. 2020

View full text Add to dashboard Cite

show abstract

“…a) Datasets: DL is data-hungry, so building datasets is an essential step. In [5], the authors proposed a parametric, systematic, scalable dataset framework called Vision-Wireless (ViWi). They utilized this framework to build the first-version dataset with four scenarios with different camera distributions (co-located and distributed) and views (blocked and direct).…”

Section: An Overview Of Related Workmentioning

confidence: 99%

“…Architecture of our proposed framework available bandwidth and ultra-high data-transmitting rate[5]-…”

mentioning

confidence: 99%

Applying Deep-Learning-Based Computer Vision to Wireless Communications: Methodologies,Opportunities, and Challenges

Tian¹,

Pan²,

Alouini³

2020

Preprint

View full text Add to dashboard Cite

<div>Deep learning (DL) has seen great success in the computer vision (CV) field, and related techniques have been used in security, healthcare, remote sensing, and many other fields. As a parallel development, visual data has become universal in daily life, easily generated by ubiquitous low-cost cameras. Therefore, exploring DL-based CV may yield useful information about objects, such as their number, locations, distribution, motion, etc. Intuitively, DL-based CV can also facilitate and improve the designs of wireless communications, especially in dynamic network scenarios. However, so far, such work is rare in the literature. The primary purpose of this article, then, is to introduce ideas about applying DL-based CV in wireless communications to bring some novel degrees of freedom to both theoretical research and engineering applications. To illustrate how DL-based CV can be applied in wireless communications, an example of using a DL-based CV with a millimeter-wave (mmWave) system is given to realize optimal mmWave multiple-input and multiple-output (MIMO) beamforming in mobile scenarios. In this example, we propose a framework to predict future beam indices from previously observed beam indices and images of street views using ResNet, 3-dimensional ResNext, and a long short-term memory network. The experimental results show that our frameworks achieve much higher accuracy than the baseline method, and that visual data can significantly improve the performance of the MIMO beamforming system. Finally, we discuss the opportunities and challenges of applying DL-based CV in wireless communications.</div>

show abstract

“…a) Datasets: DL is data-hungry, and building datasets is an essential step. In [5], the authors proposed a parametric, systematic, and scalable dataset framework called Vision-Wireless (ViWi). They utilized this framework to build the first-version dataset with four scenarios with different camera distributions (colocated and distributed) and views (blocked and direct).…”

Section: An Overview Of Related Workmentioning

confidence: 99%

“…MmWave communication is a promising technique in the 5th generation communication system, thanks to its very large available bandwidth and ultra-high data transmitting rate [5]- [7]. Beamforming should be implemented among a large antenna array to achieve the required high power gain and direction.…”

Section: A Problem Definitionmentioning

confidence: 99%

Applying Deep-Learning-Based Computer Vision to Wireless Communications: Methodologies,Opportunities, and Challenges

Tian¹,

Pan²,

Alouini³

2020

Preprint

View full text Add to dashboard Cite

<div>Deep learning (DL) has obtained great success in computer vision (CV) field, and the related techniques have been widely used in security, healthcare, remote sensing, etc. On the other hand, visual data is universal in our daily life, which is easily generated by prevailing but low-cost cameras. Therefore, DL-based CV can be explored to obtain and forecast some useful information about the objects, e.g., the number, locations, distribution, motion, etc. Intuitively, DL-based CV can facilitate and improve the designs of wireless communications, especially in dynamic network scenarios. However, so far, it is rare to see such kind of works in the existing literature. Then, the primary purpose of this article is to introduce ideas of applying DL-based CV in wireless communications to bring some novel degrees of freedom for both theoretical researches and engineering applications. To illustrate how DL-based CV can be applied in wireless communications, an example of using DL-based CV to millimeter wave (mmWave) system is given to realize optimal mmWave multiple-input and multiple-output (MIMO) beamforming in mobile scenarios. In this example, we proposed a framework to predict the future beam indices from the previously-observed beam indices and images of street views by using ResNet, 3-dimensional ResNext, and long short term memory network. Experimental results show that our frameworks can achieve much higher accuracy than the baseline method, and visual data can help significantly improve the performance of MIMO beamforming system. Finally, we discuss the opportunities and challenges of applying DL-based CV in wireless communications.</div>

show abstract

ViWi: A Deep Learning Dataset Framework for Vision-Aided Wireless Communications

Cited by 83 publications

References 13 publications

Vision-Aided Radio: User Identity Match in Radio and Video Domains Using Machine Learning

Vision-Aided Radio: User Identity Match in Radio and Video Domains Using Machine Learning

Applying Deep-Learning-Based Computer Vision to Wireless Communications: Methodologies,Opportunities, and Challenges

Applying Deep-Learning-Based Computer Vision to Wireless Communications: Methodologies,Opportunities, and Challenges

Contact Info

Product

Resources

About