Vision-Aided Radio: User Identity Match in Radio and Video Domains Using Machine Learning

Abstract

5G is designed to be an essential enabler and a leading infrastructure provider in the communication technology industry by supporting the demand for the growing data traffic and a variety of services with distinct requirements. The use of deep learning and computer vision tools has the means to increase the environmental awareness of the network with information from visual data. Information extracted via computer vision tools such as user position, movement direction, and speed can be promptly available for the network. However, the network must have a mechanism to match the identity of a user in both visual and radio systems. This mechanism is absent in the present literature. Therefore, we propose a framework to match the information from both visual and radio domains. This is an essential step to practical applications of computer vision tools in communications. We detail the proposed framework training and deployment phases for a presented setup. We carried out practical experiments using data collected in different types of environments. The work compares the use of Deep Neural Network and Random Forest classifiers and shows that the former performed better across all experiments, achieving classification accuracy greater than 99%.

Intelligent Massive MIMO Systems for Beyond 5G Networks: An Overview and Future Trends

“…This combines wireless data and vision data in order to overcome blockage, assist in the prediction of mMIMO channel subspace, enhance hand-over mechanism, enable context-aware communication, and provide proactive network management [329]- [331]. Examples of application of ML techniques in vision-aided wireless communication have been explored in [329]- [332] with potential benefits. However, the deployment of JCAS opens up new challenges such as the need for signal processing for the detection of the presence and shape of objects, interference, optimal overheads, and enhanced protocols for different radar re-quirements [326].…”

“…However, the deployment of JCAS opens up new challenges such as the need for signal processing for the detection of the presence and shape of objects, interference, optimal overheads, and enhanced protocols for different radar re-quirements [326]. In addition, the creation of an effective framework that captures scenario-dependent data and system configurations for vision-aided wireless communication is a promising area [329], [332].…”

Computer Vision Aided mmWave Beam Alignment in V2X Communications

“…Nevertheless, if BS is obliged to take images, then additional characteristic information of the MS is required by BS to identify the MS from all detected objects inside one image, i.e., to match the MS's radio signal with the corresponding visual information [17]. Therefore, in [13]- [16], the BS needs to rely on the previous beam sequences or the location fed back by MS to achieve the blockage/handover prediction or beam tracking/selection, which leads to inevitable communication overhead.…”

T109 Evaluation of the GEM® Premier™ 5000 with intelligent quality management 2 (IQM®2) at Beijing Tsinghua Changgung Hospital, Tsinghua University (Beijing, China)

“…Once the user has been matched to its visual image, the latter can be tracked continuously by the state-of-the-art object tracking techniques [16]- [17] without any communications overhead. To handle the user matching task, the authors in [18] design a classification DNN to distinguish user from all the environmental objects by the channel of the user and the detected object bounding boxes (BBoxes) in the image. Nevertheless, the algorithm of [18] can only handle two environmental objects, whereas the object number is random in realistic environment, and the channel is hard to obtain especially for the system with large scale antenna arrays.…”