Multimodal Dataset for Assessment of Quality of Experience in Immersive Multimedia

Abstract

This paper presents a novel multimodal dataset for the analysis of Quality of Experience (QoE) in emerging immersive multimedia applications. In particular, the perceived Sense of Presence (SoP) induced by one-minute long video stimuli is explored with respect to content, quality, resolution, and sound reproduction and annotated with subjective scores. Furthermore, a complementary analysis of the recorded physiological signals, such as EEG, ECG, and respiration is carried out, aiming at an alternative evaluation of human experience while consuming immersive multimedia content. Results confirm the value of the introduced dataset and its consistency for the purposes of QoE assessment for immersive multimedia. More specifically, subjective ratings demonstrate that the created dataset enables distinction between low and high levels of immersiveness, which is also confirmed by a preliminary analysis of recorded physiological signals.

Psychophysiology-Based QoE Assessment: A Survey

“…The experiences assessed with multimodal approaches are broad and range from traditional applications, such as multimedia quality assessment [32], [47], through more advanced applications, such as assessing visual fatigue for 3D video [14], [80] and tone mapping perception for high dynamic range (HDR) video [35], to higher-level experiences, including immersiveness [76] , emotion [77], stress [79], and engagement [78]. Most studies do not abandon traditional self-reporting but rather include it as a well understood reference.…”

“…In decision fusion, feature vectors from each channel are used as inputs to independent classifiers, whose outputs are then combined. Very few works [35], [76], [78] jointly consider and fuse measurements from different modalities, and majority voting appears to be a common fusion strategy. We argue that significantly more investigation into fusion methods, especially biologically inspired ones, is needed to further advance multimodal approaches.…”

“…Besides some of the measures discussed in our survey, several other peripheral measures are included in these databases such as magnetoencephalogram (MEG), electrooculogram (EOG), electromyogram (EMG), respiratory measurements (RM), facial tracking (FT), and skin temperature (ST). Perrin et al [76], [101]. While these data sets are highly valuable for the research community, they provide only small pieces of the overall puzzle.…”

Implicit Analysis of Perceptual Multimedia Experience Based on Physiological Response: A Review

“…Table I compares the characteristics of the physiological channels in terms of relevance to perceptual factors, temporal resolution, spatial resolution, and portability. [38], [52], [62]- [64], QoE [47], [48], [50], fatigue [66] Respiration Being slowed down in relaxation and irregular with negative emotions × -Emotion [28], [36]- [38], [62]- [64], QoE [47], [48], [50]…”

Towards prediction of Sense of Presence in immersive audiovisual communications

“…This is achieved by building upon a past similar attempt by the authors of this paper [13], where the sense of presence in a variety of audiovisual experiences was measured as a function of the modality and the quality of audiovisual content, from lower quality and no audio, standard definition video, to higher quality surround audio and ultra high definition video. The SoP was evaluated both explicitly, by means of a questionnaire, as well as implicitly, by recording selected physiological signals such as EEG, ECG, and respiration.…”