Spherical Convolution Empowered Viewport Prediction in 360 Video Multicast with Limited FoV Feedback

Abstract

Field of view (FoV) prediction is critical in 360-degree video multicast, which is a key component of the emerging Virtual Reality (VR) and Augmented Reality (AR) applications. Most of the current prediction methods combining saliency detection and FoV information neither take into account that the distortion of projected 360-degree videos can invalidate the weight sharing of traditional convolutional networks, nor do they adequately consider the difficulty of obtaining complete multi-user FoV information, which degrades the prediction performance. This paper proposes a spherical convolution-empowered FoV prediction method, which is a multi-source prediction framework combining salient features extracted from 360-degree video with limited FoV feedback information. A spherical convolution neural network (CNN) is used instead of a traditional two-dimensional CNN to eliminate the problem of weight sharing failure caused by video projection distortion. Specifically, salient spatial-temporal features are extracted through a spherical convolution-based saliency detection model, after which the limited feedback FoV information is represented as a time-series model based on a spherical convolution-empowered gated recurrent unit network. Finally, the extracted salient video features are combined to predict future user FoVs. The experimental results show that the performance of the proposed method is better than other prediction methods.

Facial Emotion Recognition Using a Novel Fusion of Convolutional Neural Network and Local Binary Pattern in Crime Investigation

“…is is also consistent with the above research results [53]. Besides, paying attention to juveniles' psychological states, especially those involved in crimes, and giving appropriate psychological guidance and method guidance have a positive role in helping juveniles establish correct outlooks on life and values [54][55][56][57].…”

“…This is also consistent with the above research results [ 53 ]. Besides, paying attention to juveniles' psychological states, especially those involved in crimes, and giving appropriate psychological guidance and method guidance have a positive role in helping juveniles establish correct outlooks on life and values [ 54 – 57 ]. The hybrid CNN-CLBP model reported here achieves excellent performance in facial expression recognition and can precisely capture different facial expressions.…”

Construction of Emergency Procurement System and System Improvement Based on Convolutional Neural Network

“…From the perspective of international emergency equipment procurement regulations, it is divided into two types: general emergencies and major emergencies. In general emergencies, the procurement of emergency equipment is to avoid adverse consequences to purchase corresponding emergency treatment equipment and to reduce the emergency situations by reducing the consequences of the incident [41][42][43]. Emergency equipment procurement under major emergencies shall implement corresponding emergency equipment procurement in accordance with strict equipment standards to reduce or curb the occurrence and consequences of catastrophic event [44,45] disposal.…”

Coordinated Development of Smart City and Regional Industrial Economy under the Background of Internet of Things

“…e actual construction capacity of the project puts forward certain forward-looking development suggestions. e survey also uses the smart city entropy evaluation algorithm to process the economic data of the smart city to improve the accuracy of the evaluation [43,44]. e advantage of the method in this paper is that the efficiency and accuracy of data processing are improved through the entropy evaluation algorithm, thereby effectively solving the problem of excessive data volume in smart cities.…”