Real-Time Application for Generating Multiple Experiences from 360° Panoramic Video by Tracking Arbitrary Objects and Viewer’s Orientations

Abstract

We propose a novel authoring and viewing system for generating multiple experiences with a single 360° video and efficiently transferring these experiences to the user. An immersive video contains much more interesting information within the 360° environment than normal videos. There can be multiple interesting areas within a 360° frame at the same time. Due to the narrow field of view in virtual reality head-mounted displays, a user can only view a limited area of a 360° video. Hence, our system is aimed at generating multiple experiences based on interesting information in different regions of a 360° video and efficient transferring of these experiences to prospective users. The proposed system generates experiences by using two approaches: (1) Recording of the user’s experience when the user watches a panoramic video using a virtual reality head-mounted display, and (2) tracking of an arbitrary interesting object in a 360° video selected by the user. For tracking of an arbitrary interesting object, we have developed a pipeline around an existing simple object tracker to adapt it for 360° videos. This tracking algorithm was performed in real time on a CPU with high precision. Moreover, to the best of our knowledge, there is no such existing system that can generate a variety of different experiences from a single 360° video and enable the viewer to watch one 360° visual content from various interesting perspectives in immersive virtual reality. Furthermore, we have provided an adaptive focus assistance technique for efficient transferring of the generated experiences to other users in virtual reality. In this study, technical evaluation of the system along with a detailed user study has been performed to assess the system’s application. Findings from evaluation of the system showed that a single 360° multimedia content has the capability of generating multiple experiences and transfers among users. Moreover, sharing of the 360° experiences enabled viewers to watch multiple interesting contents with less effort.

Adaptive Playback Control: A Framework for Cinematic VR Creators to Embrace Viewer Interaction

“…They used this scale to measure learner enjoyment after watching a series of Massive Open Online Courses (MOOCs) in the form of 360-degree videos, similar to our study setup. A simple three-part evaluation form measured the general user experience towards the system, provided by Shah et al (2020) in their study evaluating a 360-degree video playback system. We removed one non-relevant item, then applied it to see if the participants find it easy to use the system, both the invisible interaction and the traditional point-and-activate method.…”

360° Virtual Reality Video Tours Generation Model for Hostelry and Tourism based on the Analysis of User Profiles and Case-Based Reasoning

“…The use of 360°videos is relevant from the use of VR technology to create immersive spaces in which the observer mimics, captures through the senses and feels what has been artificially created in the environment. According to [25], immersive videos show more interesting information within the 360°environment than normal videos and are more eyecatching, due to the greater interest of users in those visual contents, since there is a feeling of being present in the explored content; in addition, that information of interest to explore can be obtained, that is why the number of consumers in the entertainment industry using 360°videos is increasing day by day.…”

Predictive Maintenance of Norwegian Road Network Using Deep Learning Models

“…The fourth industrial revolution (Industry 4.0) strongly emphasizes the integration of physical and digital systems [ 6 , 7 ]. PdM-based road inspection is a novel and demanding concept to maintain the road’s infrastructure.…”