Multimodal Engagement Classification for Affective Cinema

Abadi, Mojtaba Khomami; Staiano, Jacopo; Cappelletti, Alessandro; Zancanaro, Massimo; Sebe, Nicu

doi:10.1109/acii.2013.74

Cited by 16 publications

(10 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are numerous studies which focus on QoE assessment by using biological measurements. [23] shows a significant correlation between Electroencephalography (EEG)/Electrocardiography (ECG) and video quality levels, [24] presents that Electrodermal Activity (EDA) is effective to measure the perception to given visual fatigue, whereas [25] concludes that the same result was found from both EEG and EDA. Particularly, in the effort to directly measure the perceived video quality changes using EEG, the authors in [26] concluded that abrupt changes of video quality give rise to specific components in the EEG that can be detected in a single-trial basic.…”

Section: Related Workmentioning

confidence: 96%

Collaborative Approach Using Psychophysiology and Psychophysics for Optimal Threshold Determination in HAS Service QoE Management

Tan¹,

Kamioka²

2018

JCC

View full text Add to dashboard Cite

In HTTP adaptive streaming (HAS) services, Quality of Experience (QoE) management has increasingly played an important role in guaranteeing the expected video quality perceived by the users with the minimal network resource. To achieve this, QoE must be automatically monitored and accurately controlled when it falls to a specific threshold. However, how to determine the QoE threshold with a scientific basis has not been carefully studied yet. In this paper, a new method to determine a QoE threshold based on biological information is proposed for a QoE management in HAS services. More concretely, the QoE threshold is determined by modeling the impact of the stimulus on the biological information obtained from the users who are watching videos. As the biological information, Skin Conductance and Heart Rate associated with Autonomic Nervous System (ANS) have been investigated. The evaluation results demonstrate that the QoE management by using the determined threshold can save more than 4.855% of the bandwidth consumption per control ensuring as high video quality as by using the fair threshold.

show abstract

Section: Related Workmentioning

confidence: 96%

Collaborative Approach Using Psychophysiology and Psychophysics for Optimal Threshold Determination in HAS Service QoE Management

Tan¹,

Kamioka²

2018

JCC

View full text Add to dashboard Cite

show abstract

“…The experiences assessed with multimodal approaches are broad and range from traditional applications, such as multimedia quality assessment [32], [47], through more advanced applications, such as assessing visual fatigue for 3D video [14], [80] and tone mapping perception for high dynamic range (HDR) video [35], to higher-level experiences, including immersiveness [76] , emotion [77], stress [79], and engagement [78]. Most studies do not abandon traditional self-reporting but rather include it as a well understood reference.…”

Section: Multimodal Techniquesmentioning

confidence: 99%

“…In decision fusion, feature vectors from each channel are used as inputs to independent classifiers, whose outputs are then combined. Very few works [35], [76], [78] jointly consider and fuse measurements from different modalities, and majority voting appears to be a common fusion strategy. We argue that significantly more investigation into fusion methods, especially biologically inspired ones, is needed to further advance multimodal approaches.…”

Section: Multimodal Techniquesmentioning

confidence: 99%

“…Some widely adopted modalities, such as electroencephalography (EEG) and eye measurement (EM), typically report better agreement than other peripheral measures, such as electrocardiography (ECG) and respiratory measurement (RM) (see for instance findings in [32], [35], [77], [78], [80]). Not all studies are in agreement though, with [32] finding no significant correlation for electrodermal activity (EDA) but [80] finding EDA to be an effective measurement technique.…”

Section: Multimodal Techniquesmentioning

confidence: 99%

See 1 more Smart Citation

Psychophysiology-Based QoE Assessment: A Survey

Engelke

Darcy

Mulliken

et al. 2017

IEEE J. Sel. Top. Signal Process.

105

View full text Add to dashboard Cite

Abstract-We present a survey of psychophysiology-based assessment for Quality of Experience (QoE) in advanced multimedia technologies. We provide a classification of methods relevant to QoE and describe related psychological processes, experimental design considerations, and signal analysis techniques. We summarise multimodal techniques and discuss several important aspects of psychophysiology-based QoE assessment, including the synergies with psychophysical assessment and the need for standardised experimental design. This survey is not considered to be exhaustive but serves as a guideline for those interested to further explore this emerging field of research.

show abstract

“…In particular, in [12], 2D and 3D overall perceived quality is assessed using brain and peripheral signals, [11] concludes that perceived quality is related to emotional processes, whereas [10,20] analyze emotional experiences as opposed to SoP. The authors of [1] present a similar sensor-based assessment of SoP using galvanic skin response, EEG, and facial motion tracking. This paper presents a novel dataset that captures the differences in user experience during multimedia stimuli with various ILs.…”

Section: Introductionmentioning

confidence: 99%

Multimodal Dataset for Assessment of Quality of Experience in Immersive Multimedia

Perrin

Kroupi

et al. 2015

Proceedings of the 23rd ACM International Conference on Multimedia

View full text Add to dashboard Cite

This paper presents a novel multimodal dataset for the analysis of Quality of Experience (QoE) in emerging immersive multimedia applications. In particular, the perceived Sense of Presence (SoP) induced by one-minute long video stimuli is explored with respect to content, quality, resolution, and sound reproduction and annotated with subjective scores. Furthermore, a complementary analysis of the recorded physiological signals, such as EEG, ECG, and respiration is carried out, aiming at an alternative evaluation of human experience while consuming immersive multimedia content. Results confirm the value of the introduced dataset and its consistency for the purposes of QoE assessment for immersive multimedia. More specifically, subjective ratings demonstrate that the created dataset enables distinction between low and high levels of immersiveness, which is also confirmed by a preliminary analysis of recorded physiological signals.

show abstract

Multimodal Engagement Classification for Affective Cinema

Cited by 16 publications

References 24 publications

Collaborative Approach Using Psychophysiology and Psychophysics for Optimal Threshold Determination in HAS Service QoE Management

Collaborative Approach Using Psychophysiology and Psychophysics for Optimal Threshold Determination in HAS Service QoE Management

Psychophysiology-Based QoE Assessment: A Survey

Multimodal Dataset for Assessment of Quality of Experience in Immersive Multimedia

Contact Info

Product

Resources

About