Predicting subjective sensation of reality during multimedia consumption based on EEG and peripheral physiological signals

Kroupi, Eleni; Hanhart, Philippe; Lee, Jong Seok; Řeřábek, Martin; Ebrahimi, Touradj

doi:10.1109/icme.2014.6890239

Cited by 23 publications

(21 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [18], we demonstrated that classification can be also used to automatically recognize high from low sensation of reality, in a subject-dependent framework. In this paper, we aimed at predicting all evaluated aspects, i.e., content preference, depth perception, sensation of reality, and perceived overall quality, using subject-independent classification systems.…”

Section: Further Discussionmentioning

confidence: 99%

“…In this paper, we aimed at predicting all evaluated aspects, i.e., content preference, depth perception, sensation of reality, and perceived overall quality, using subject-independent classification systems. The current study also differs in the sense that we used an LDA classifier, whereas a SVM classifier was used in [18,19]. Moreover, in this paper, we also applied a decision-fusion scheme to EEG and peripheral features, to explore the advantage of integrating information from the two modalities.…”

Section: Further Discussionmentioning

confidence: 99%

“…The EEG high beta frequency band (21-29 Hz) seemed to be mainly responsible for this discrimination. Moreover, in [18] it was demonstrated that EEG-based classification can be also used to automatically recognize high from low sensation of reality, in a subject-dependent framework. Also, classification of sensation of reality from heart and respiration was possible, but less accurate than using EEG signals.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Modeling immersive media experiences by sensing impact on subjects

Kroupi

Hanhart

Lee

et al. 2015

Multimed Tools Appl

Self Cite

View full text Add to dashboard Cite

As immersive technologies target to provide higher quality of multimedia experiences, it is important to understand the quality of experience (QoE) perceived by users from various multimedia rendering schemes, in order to design and optimize human-centric immersive multimedia systems. In this study, various QoE-related aspects, such as depth perception, sensation of reality, content preference, and perceived quality are being investigated and compared for presentation of 2D and 3D contents. Since the advantages of implicit over explicit QoE assessment have become essential, the way these QoE-related aspects influence brain and periphery is also investigated. In particular, two classification schemes using electroencephalography (EEG) and peripheral signals (electrocardiography and respiration) are carried out, to explore if it is possible to automatically recognize the QoE-related aspects under investigation. In addition, a decision-fusion scheme is applied to EEG and peripheral features, to explore the advantage of integrating information from the two modalities. The results reveal that the highest monomodal average informedness is achieved in the high beta EEG band (0.14% ± 0.09, p < 0.01), when recognizing sensation of reality. The highest and significantly non-random multimodal average informedness is achieved in when high beta EEG band is fused with peripheral features (0.17% ± 0.1, p < 0.01), for the case of sensation of reality. Finally, a temporal analysis is conducted to explore how the EEG correlates for the case of sensation of reality change over time. The * Refers to the corresponding author. The research leading to these results has been performed in the framework of Swiss National Foundation for Scientific Research (FN 200020-132673-1 and FN 200021-143696-1) results reveal that that the right cortex is more involved when sensation of reality is low, and the left one when sensation of reality is high, indicating that approach and withdrawal-related processes occur during sensation of reality.

show abstract

Section: Further Discussionmentioning

confidence: 99%

Section: Further Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Modeling immersive media experiences by sensing impact on subjects

Kroupi

Hanhart

Lee

et al. 2015

Multimed Tools Appl

Self Cite

View full text Add to dashboard Cite

show abstract

“…Electroencephalography (EEG) and peripheral physiological signals including Electrocardiography (ECG) and respiration have been used in various studies to assess the physiological responses to stimuli. In particular, in [12], 2D and 3D overall perceived quality is assessed using brain and peripheral signals, [11] concludes that perceived quality is related to emotional processes, whereas [10,20] analyze emotional experiences as opposed to SoP. The authors of [1] present a similar sensor-based assessment of SoP using galvanic skin response, EEG, and facial motion tracking.…”

Section: Introductionmentioning

confidence: 99%

Multimodal Dataset for Assessment of Quality of Experience in Immersive Multimedia

Perrin

Kroupi

et al. 2015

Proceedings of the 23rd ACM International Conference on Multimedia

Self Cite

View full text Add to dashboard Cite

This paper presents a novel multimodal dataset for the analysis of Quality of Experience (QoE) in emerging immersive multimedia applications. In particular, the perceived Sense of Presence (SoP) induced by one-minute long video stimuli is explored with respect to content, quality, resolution, and sound reproduction and annotated with subjective scores. Furthermore, a complementary analysis of the recorded physiological signals, such as EEG, ECG, and respiration is carried out, aiming at an alternative evaluation of human experience while consuming immersive multimedia content. Results confirm the value of the introduced dataset and its consistency for the purposes of QoE assessment for immersive multimedia. More specifically, subjective ratings demonstrate that the created dataset enables distinction between low and high levels of immersiveness, which is also confirmed by a preliminary analysis of recorded physiological signals.

show abstract

“…In [14], to make the annotation more reliable, the proposed method integrates local pairwise comparison labels together to minimize a cost that corresponds to global inconsistency of ranking order. In [15], the authors construct subjective classification systems to predict sensation of reality from multimedia experiences based on EEG and peripheral physiological signals such as heart rate and respiration. In [16], the authors propose a machine learning based data fusion algorithm that can provide real time per frame training and decision based cooperative spectrum sensing.…”

Section: Related Workmentioning

confidence: 99%

Quality Assessment of Training Data with Uncertain Labels for Classification of Subjective Domains

Dai¹

2017

JCC

View full text Add to dashboard Cite

In order to improve the performance of classifiers in subjective domains, this paper defines a metric to measure the quality of the subjectively labelled training data (QoSTD) by means of K-means clustering. Then, the QoSTD is used as a weight of the predicted class scores to adjust the likelihoods of instances. Moreover, two measurements are defined to assess the performance of the classifiers trained by the subjective labelled data. The binary classifiers of Traditional Chinese Medicine (TCM) Zhengs are trained and retrained by the real-world data set, utilizing the support vector machine (SVM) and the discrimination analysis (DA) models, so as to verify the effectiveness of the proposed method. The experimental results show that the consistency of likelihoods of instances with the corresponding observations is increased notable for the classes, especially in the cases with the relatively low QoSTD training data set. The experimental results also indicate the solution how to eliminate the miss-labelled instances from the training data set to re-train the classifiers in the subjective domains.

show abstract

Predicting subjective sensation of reality during multimedia consumption based on EEG and peripheral physiological signals

Cited by 23 publications

References 11 publications

Modeling immersive media experiences by sensing impact on subjects

Modeling immersive media experiences by sensing impact on subjects

Multimodal Dataset for Assessment of Quality of Experience in Immersive Multimedia

Quality Assessment of Training Data with Uncertain Labels for Classification of Subjective Domains

Contact Info

Product

Resources

About