Implicit Affective Video Tagging Using Pupillary Response

Gui, Dongdong; Zhong, Sheng-hua; Ming, Zhong

doi:10.1007/978-3-319-73600-6_15

Cited by 7 publications

(8 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our method also has the highest F1-score among all 3 methods. It is worth noting that our method does not result in lower F1-score because of the sample imbalance reported by Gui et al [14], which shows CorrFeat can better generalize the distribution of data according to their correlations.…”

Section: Comparison With ML and Dl Methodsmentioning

confidence: 62%

“…Our method achieves the highest accuracy among other methods with only SC and PD. Gui et al [14] also obtain good result with only PD. However, they only validate their method on 23 subjects among all 27 in MAHNOB-HCI database.…”

Section: Comparison With ML and Dl Methodsmentioning

confidence: 89%

“…Most previous work [14,23,31] solve this problem by manually designing handcrafted features across different sensors and fusing them at decision level [29], which is timeconsuming and often leads to low accuracies. Other works [1,28,33] segment or pad the signals to let them have fixed lengths and train the data with neural networks.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

CorrFeat: Correlation-based Feature Extraction Algorithm using Skin Conductance and Pupil Diameter for Emotion Recognition

Zhang¹,

Ali

Wang³

et al. 2019

2019 International Conference on Multimodal Interaction

View full text Add to dashboard Cite

To recognize emotions using less obtrusive wearable sensors, we present a novel emotion recognition method that uses only pupil diameter (PD) and skin conductance (SC). Psychological studies show that these two signals are related to the attention level of humans exposed to visual stimuli. Based on this, we propose a feature extraction algorithm that extract correlation-based features for participants watching the same video clip. To boost performance given limited data, we implement a learning system without a deep architecture to classify arousal and valence. Our method outperforms not only state-of-art approaches, but also widely-used traditional and deep learning methods.

show abstract

Section: Comparison With ML and Dl Methodsmentioning

confidence: 62%

Section: Comparison With ML and Dl Methodsmentioning

confidence: 89%

See 1 more Smart Citation

CorrFeat: Correlation-based Feature Extraction Algorithm using Skin Conductance and Pupil Diameter for Emotion Recognition

Zhang¹,

Ali

Wang³

et al. 2019

2019 International Conference on Multimodal Interaction

View full text Add to dashboard Cite

show abstract

“…We chose the 12 videos according to 2D emotion annotations from the self-reports in MAHNOB dataset [ 97 ]. We use the videos in MAHNOB dataset because it is a widely used dataset [ 98 , 99 ] with emotion self-reports from more than 30 reviewers. We selected more videos compared with CASE because we aim to collect more samples for each emotion.…”

Section: Datasetsmentioning

confidence: 99%

CorrNet: Fine-Grained Emotion Recognition for Video Watching Using Wearable Physiological Sensors

Zhang

Ali

Wang³

et al. 2020

Sensors

View full text Add to dashboard Cite

Recognizing user emotions while they watch short-form videos anytime and anywhere is essential for facilitating video content customization and personalization. However, most works either classify a single emotion per video stimuli, or are restricted to static, desktop environments. To address this, we propose a correlation-based emotion recognition algorithm (CorrNet) to recognize the valence and arousal (V-A) of each instance (fine-grained segment of signals) using only wearable, physiological signals (e.g., electrodermal activity, heart rate). CorrNet takes advantage of features both inside each instance (intra-modality features) and between different instances for the same video stimuli (correlation-based features). We first test our approach on an indoor-desktop affect dataset (CASE), and thereafter on an outdoor-mobile affect dataset (MERCA) which we collected using a smart wristband and wearable eyetracker. Results show that for subject-independent binary classification (high-low), CorrNet yields promising recognition accuracies: 76.37% and 74.03% for V-A on CASE, and 70.29% and 68.15% for V-A on MERCA. Our findings show: (1) instance segment lengths between 1–4 s result in highest recognition accuracies (2) accuracies between laboratory-grade and wearable sensors are comparable, even under low sampling rates (≤64 Hz) (3) large amounts of neutral V-A labels, an artifact of continuous affect annotation, result in varied recognition performance.

show abstract

“…Wang et al [132] used EEG signals to construct a new EEG feature with the assistance of the relationship among video content by exploiting canonical correlation analysis (CCA). In [40], some viewer-related features are extracted from the whole pupil dilation ratio time-series without the differences among pupil diameter in human eyes, such as its average and derivation for global features as well as the four spectral power bands for local features.…”

Section: Viewer-related Featuresmentioning

confidence: 99%

Affective Computing for Large-scale Heterogeneous Multimedia Data

Zhao

Wang

Soleymani

et al. 2019

ACM Trans. Multimedia Comput. Commun. Appl.

View full text Add to dashboard Cite

The wide popularity of digital photography and social networks has generated a rapidly growing volume of multimedia data (i.e., image, music, and video), resulting in a great demand for managing, retrieving, and understanding these data. Affective computing (AC) of these data can help to understand human behaviors and enable wide applications. In this article, we survey the state-of-the-art AC technologies comprehensively for large-scale heterogeneous multimedia data. We begin this survey by introducing the typical emotion representation models from psychology that are widely employed in AC. We briefly describe the available datasets for evaluating AC algorithms. We then summarize and compare the representative methods on AC of different multimedia types, i.e., images, music, videos, and multimodal data, with the focus on both handcrafted features-based methods and deep learning methods. Finally, we discuss some challenges and future directions for multimedia affective computing.

show abstract

Implicit Affective Video Tagging Using Pupillary Response

Cited by 7 publications

References 27 publications

CorrFeat: Correlation-based Feature Extraction Algorithm using Skin Conductance and Pupil Diameter for Emotion Recognition

CorrFeat: Correlation-based Feature Extraction Algorithm using Skin Conductance and Pupil Diameter for Emotion Recognition

CorrNet: Fine-Grained Emotion Recognition for Video Watching Using Wearable Physiological Sensors

Affective Computing for Large-scale Heterogeneous Multimedia Data

Contact Info

Product

Resources

About