Psychophysiology-Based QoE Assessment: A Survey

Engelke, Ulrich; Darcy, Daniel P.; Mulliken, Grant H.; Bosse, Sebastian; Martini, Maria G.; Arndt, Sebastian; Voigt-Antons, Jan‐Niklas; Chan, Kit Yan; Ramzan, Naeem; Brunnström, Kjell

doi:10.1109/jstsp.2016.2609843

Cited by 103 publications

(64 citation statements)

References 97 publications

(107 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Blinks, winks, complete eye closure and partial eye opening or closure events are involved in certain eye gazes and saccades [18], and facial expressions of emotion [19]. Additionally, it was suggested that eye gaze and blink share common signalling pathways [4], which makes incorporation of eye closure and blink events important for a complete investigation of potential eye-based cues for affective computing. Eye-based cue estimation from video frames is now possible due to advances in computer vision models and tools [20].…”

Section: Introductionmentioning

confidence: 99%

Eye-based Continuous Affect Prediction

O'Dwyer

Murray

Flynn

2019

2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII)

View full text Add to dashboard Cite

Eye-based information channels include the pupils, gaze, saccades, fixational movements, and numerous forms of eye opening and closure. Pupil size variation indicates cognitive load and emotion, while a person's gaze direction is said to be congruent with the motivation to approach or avoid stimuli. The eyelids are involved in facial expressions that can encode basic emotions. Additionally, eye-based cues can have implications for human annotators of emotions or feelings. Despite these facts, the use of eye-based cues in affective computing is in its infancy, however, and this work is intended to start to address this. Eye-based feature sets, incorporating data from all of the aforementioned information channels, that can be estimated from video are proposed. Feature set refinement is provided by way of continuous arousal and valence learning and prediction experiments on the RECOLA validation set. The eye-based features are then combined with a speech feature set to provide confirmation of their usefulness and assess affect prediction performance compared with group-of-humans-level performance on the RECOLA test set. The core contribution of this paper, a refined eye-based feature set, is shown to provide benefits for affect prediction. It is hoped that this work stimulates further research into eye-based affective computing.

show abstract

Section: Introductionmentioning

confidence: 99%

Eye-based Continuous Affect Prediction

O'Dwyer

Murray

Flynn

2019

2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII)

View full text Add to dashboard Cite

show abstract

“…Eye-close frame count data, which is not used in [4], was found to be of benefit and is included in the proposed eye gaze feature set. Another difference between the feature set presented in this work and [4] is the use of eye average eye close frame count, standard deviation of eye close frame count, skewness of eye close frame count scan paths, based on [21], and inter quartile ranges of eye gaze coordinates. The inter quartile ranges of eye gaze coordinates were added to the proposed eye gaze feature set based on experimentation and performance optimization for the intended application of continuous affect prediction from audio and video.…”

Section: Eye Gaze Feature Setmentioning

confidence: 99%

Continuous affect prediction using eye gaze and speech

O'Dwyer¹,

Flynn²,

Murray³

2017

2017 IEEE International Conference on Bioinformatics and Biomedicine (BIBM)

View full text Add to dashboard Cite

Affective computing research traditionally focused on labeling a person's emotion as one of a discrete number of classes e.g. happy or sad. In recent times, more attention has been given to continuous affect prediction across dimensions in the emotional space, e.g. arousal and valence. Continuous affect prediction is the task of predicting a numerical value for different emotion dimensions. The application of continuous affect prediction is powerful in domains involving real-time audio-visual communications which could include remote or assistive technologies for psychological assessment of subjects. Modalities used for continuous affect prediction may include speech, facial expressions and physiological responses. As opposed to single modality analysis, the research community have combined multiple modalities to improve the accuracy of continuous affect prediction. In this context, this paper investigates a continuous affect prediction system using the novel combination of speech and eye gaze. A new eye gaze feature set is proposed. This novel approach uses open source software for real-time affect prediction in audio-visual communication environments. A unique advantage of the human-computer interface used here is that it does not require the subject to wear specialized and expensive eye-tracking headsets or intrusive devices. The results indicate that the combination of speech and eye gaze improves arousal prediction by 3.5% and valence prediction by 19.5% compared to using speech alone.

show abstract

“…The explicit methods are applied to qualitatively and quantitatively assess the viewing feelings of subjects, but they are time‐consuming, costly, and easily influenced by the experimental design and the variance among the participants . Also, sometimes, it is difficult for the subjects to convey perceived quality level through the particular rating scale accurately …”

Section: Introductionmentioning

confidence: 99%

“…16 Also, sometimes, it is difficult for the subjects to convey perceived quality level through the particular rating scale accurately. 17 The implicit methods usually record subject's brain activities or peripheral physiological activities during stimuli representation directly. The electroencephalography (EEG) is a kind of noninvasive technique to record electrical activities of the brain with a high temporal resolution.…”

Section: Introductionmentioning

confidence: 99%

Study of image quality using event‐related potentials and eye tracking measurement

et al. 2018

View full text Add to dashboard Cite

The image quality was investigated using electroencephalography and eye‐tracking measurements simultaneously. Different image qualities were obtained by changing σ values in the Gaussian kernel function. The experimental results illustrated that the measurable event‐related potentials and eye‐tracking parameters varied with the change of the image quality. The more blurred the image, the higher and earlier the P300 (event‐related potentials at latency about 300–550 ms) was elicited. Meanwhile, the visual fixation duration decreased and the detection rate increased with the increase of σ. The blurred image quality could be evaluated based on the synthetic results of the reciprocal of P300 latency and the visual fixation duration. The result showed a good correlation with the subjective rating scores, which proved the feasibility of utilizing electroencephalography and eye‐tracking technology to evaluate the blurred image quality.

show abstract

Psychophysiology-Based QoE Assessment: A Survey

Cited by 103 publications

References 97 publications

Eye-based Continuous Affect Prediction

Eye-based Continuous Affect Prediction

Continuous affect prediction using eye gaze and speech

Study of image quality using event‐related potentials and eye tracking measurement

Contact Info

Product

Resources

About