Modeling immersive media experiences by sensing impact on subjects

Abstract

As immersive technologies target to provide higher quality of multimedia experiences, it is important to understand the quality of experience (QoE) perceived by users from various multimedia rendering schemes, in order to design and optimize human-centric immersive multimedia systems. In this study, various QoE-related aspects, such as depth perception, sensation of reality, content preference, and perceived quality are being investigated and compared for presentation of 2D and 3D contents. Since the advantages of implicit over explicit QoE assessment have become essential, the way these QoE-related aspects influence brain and periphery is also investigated. In particular, two classification schemes using electroencephalography (EEG) and peripheral signals (electrocardiography and respiration) are carried out, to explore if it is possible to automatically recognize the QoE-related aspects under investigation. In addition, a decision-fusion scheme is applied to EEG and peripheral features, to explore the advantage of integrating information from the two modalities. The results reveal that the highest monomodal average informedness is achieved in the high beta EEG band (0.14% ± 0.09, p < 0.01), when recognizing sensation of reality. The highest and significantly non-random multimodal average informedness is achieved in when high beta EEG band is fused with peripheral features (0.17% ± 0.1, p < 0.01), for the case of sensation of reality. Finally, a temporal analysis is conducted to explore how the EEG correlates for the case of sensation of reality change over time. The * Refers to the corresponding author. The research leading to these results has been performed in the framework of Swiss National Foundation for Scientific Research (FN 200020-132673-1 and FN 200021-143696-1) results reveal that that the right cortex is more involved when sensation of reality is low, and the left one when sensation of reality is high, indicating that approach and withdrawal-related processes occur during sensation of reality.

Joint effects of depth-aiding augmentations and viewing positions on the quality of experience in augmented telepresence

“…task completion time, error rates). Reliance solely on subjective assessment, also known as explicit metrics, has been criticised by Kroupi et al in [27] and Hoßfeld et al in [28], highlighting the importance of including objective (i.e. implicit) metrics in QoE assessment.…”

“…implicit) metrics in QoE assessment. Implicit metrics include measurements of participant-task interaction as suggested in [26], as well as measurements of participant psychophysical state during the experiment, as discussed in [27]. The connection between psychophysical measurements and perceived QoE in VR has been brought up by Keighrey et al in [29].…”

Feasibility of Heart Rate and Respiratory Rate Estimation by Inertial Sensors Embedded in a Virtual Reality Headset

“…Thus, it is related to the quality of rendering of an immersive multimedia in terms of impact on users’ emotion, sense of presence, level of induced stress and degree of engagement [ 16 , 17 , 18 ], which can eventually be used as feedback to provide the patient with a more personalized media experience. The way the user perceive the VR environment can be assessed explicitly or implicitly [ 19 ]: the former refers to QoE evaluation via post-test questionnaires [ 20 ] or pre-defined rating scales [ 21 ], thus resulting in a subjective metric of the user immersion level or of the VR-induced stress. The latter is a bio-inspired approach based on the acquisition of physiological signals from the VR user, such as electroencephalography (EEG), heart rate (HR) and respiratory rate, facilitating real-time monitoring of QoE without subjective biases [ 19 ].…”

“…The way the user perceive the VR environment can be assessed explicitly or implicitly [ 19 ]: the former refers to QoE evaluation via post-test questionnaires [ 20 ] or pre-defined rating scales [ 21 ], thus resulting in a subjective metric of the user immersion level or of the VR-induced stress. The latter is a bio-inspired approach based on the acquisition of physiological signals from the VR user, such as electroencephalography (EEG), heart rate (HR) and respiratory rate, facilitating real-time monitoring of QoE without subjective biases [ 19 ]. In particular, there have been efforts to measure brain activity in order to understand QoE by using various types of EEG headsets [ 19 ], but their level of intrusiveness turned out to have a bad impact on the user’s QoE.…”

“…The latter is a bio-inspired approach based on the acquisition of physiological signals from the VR user, such as electroencephalography (EEG), heart rate (HR) and respiratory rate, facilitating real-time monitoring of QoE without subjective biases [ 19 ]. In particular, there have been efforts to measure brain activity in order to understand QoE by using various types of EEG headsets [ 19 ], but their level of intrusiveness turned out to have a bad impact on the user’s QoE. Other approaches to capture biometrics while using VR have utilized non-invasive wearable devices.…”