User-centric Affective Video Tagging from MEG and Peripheral Physiological Responses

Abadi, Mojtaba Khomami; Kia, Seyed Mostafa; Subramanian, Ramanathan; Avesani, Paolo; Sebe, Nicu

doi:10.1109/acii.2013.102

Cited by 20 publications

(19 citation statements)

References 22 publications

(43 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this section, we attempt to predict the gold standard (or groundtruth) dynamic V/A ratings for each clip from audio-visual features using MTL, and show why learning the audio visual feature- (1) Average energy of audio signal [14] and first pitch frequency Formants (4) Formants up to 4400Hz Time frequency (8) mean and std of: MSpectrum flux, Spectral centroid, Delta spectrum magnitude, Band energy ratio [14] Zero crossing rate (1) Average zero crossing rate of audio signal [14] Silence ratio (2) Mean and std of proportion of silence in a time window [5,14] Video features Description Brightness (6) Mean of: Lighting key, shadow proportion, visual details, grayness, median of Lightness for frames, mean of median saturation for frames Color Features (41) Color variance, 20-bin histograms for hue and lightness in HSV space VisualExcitement (1) Features as defined in [29] Motion (1) Mean inter-frame motion [15] emotion relationship simultaneously for the 12 movie scenes is more effective than learning scene-specific models. Fig.…”

Section: Experiments and Resultsmentioning

confidence: 99%

“…In a recent CS-based media tagging work, Soleymani et al [20] presented a dataset of 1000 songs for music emotion analysis, each annotated continuously over time by at least 10 users. Nevertheless, movies denote multimedia stimuli that best approximate the real world and movie clips have been found to be more effective for eliciting emotions in viewers as compared to music video clips in [1], and that is why we believe continuous emotion prediction with movie stimuli is important in the context of affective media representation and modeling.…”

Section: Crowdsourcing For Affective Media Taggingmentioning

confidence: 99%

“…We selected 12 video clips from [1] equally distributed among the four quadrants in the valence-arousal space. Table 1 presents characteristics of video clips from the different quadrants.…”

Section: Datasetmentioning

confidence: 99%

“…In this study, we asked crowd workers to continuously annotate 12 emotional movie scenes adopted from [1] via a web-based user interface-they were not allowed to access the scene content prior to the rating task. Our objective was to understand and model their emotional state over time, as they viewed the movie clips.…”

Section: Experimental Protocolmentioning

confidence: 99%

“…This joint learning procedure accounting for task relationships leads to more efficient models as compared to learning each task independently. For the purpose of learning the relationship between low-level audio-visual features and corresponding crowdworker VA annotations over time, we ask the following questions: (1) Given that emotion perception is highly subjective, and biases relating to crowdworker demographics may additionally exist, can we discover any patterns relating to their dynamic emotional perception? The exercise of seeking to acquire a gold standard annotation for each movie scene (or clip) via crowdsourcing is meaningful only if such patterns can be discovered.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

A Multi-task Learning Framework for Time-continuous Emotion Estimation from Crowd Annotations

Abadi

Abad

Subramanian

et al. 2014

Proceedings of the 2014 International ACM Workshop on Crowdsourcing for Multimedia

Self Cite

View full text Add to dashboard Cite

We propose Multi-task learning (MTL) for time-continuous or dynamic emotion (valence and arousal) estimation in movie scenes. Since compiling annotated training data for dynamic emotion prediction is tedious, we employ crowdsourcing for the same. Even though the crowdworkers come from various demographics, we demonstrate that MTL can effectively discover (1) consistent patterns in their dynamic emotion perception, and (2) the low-level audio and video features that contribute to their valence, arousal (VA) elicitation. Finally, we show that MTL-based regression models, which simultaneously learn the relationship between low-level audio-visual features and high-level VA ratings from a collection of movie scenes, can predict VA ratings for time-contiguous snippets from each scene more effectively than scene-specific models.

show abstract

Section: Experiments and Resultsmentioning

confidence: 99%

Section: Crowdsourcing For Affective Media Taggingmentioning

confidence: 99%

Section: Datasetmentioning

confidence: 99%

Section: Experimental Protocolmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

A Multi-task Learning Framework for Time-continuous Emotion Estimation from Crowd Annotations

Abadi

Abad

Subramanian

et al. 2014

Proceedings of the 2014 International ACM Workshop on Crowdsourcing for Multimedia

Self Cite

View full text Add to dashboard Cite

show abstract

Multimedia Implicit Tagging

Soleymani¹,

Pantić²

2017

Social Signal Processing

View full text Add to dashboard Cite

Social and behavioral signals carry invaluable information regarding how audiences perceive the multimedia content. Assessing the responses from the audience, we can generate tags, summaries and other forms of metadata for multimedia representation and indexing. Tags are a form of metadata which enables a retrieval system to find and re-find the content of interest (Larson et al. (2011)). Unlike classic tagging schemes where users direct input is needed, Implicit Human-Centered Tagging (IHCT) was proposed (Pantic and Vinciarelli (2009)) to generate tags without any specific input or effort from users. Translating the behavioral responses into tags results in "implicit" tags since there is no need for users' direct input as reactions to multimedia are displayed spontaneously (Soleymani and Pantic (2012)). User generated explicit tags are not always assigned with the intention of describing the content and might be given to promote the users themselves (Pantic and Vinciarelli (2009)). Implicit tags have the advantage of being detected for a certain goal relevant to a given application. For example, an online radio is interested in the mood of its songs can assess listeners emotions; a marketing company is interested in assessing in the success of its video advertisements. It is also worth mentioning that implicit tags can be a complementary a iBUG group, Imperial College London, UK. His work is supported by the European Research Council under the FP7 Marie Curie Intra-European Fellowship: Emotional continuous tagging using spontaneous behavior (EmoTag).

show abstract

A survey of affective brain computer interfaces: principles, state-of-the-art, and challenges

Mühl

Allison

Nijholt

et al. 2014

Brain-Computer Interfaces

234

159

View full text Add to dashboard Cite

Affective states, moods and emotions, are an integral part of human nature: they shape our thoughts, govern the behavior of the individual, and influence our interpersonal relationships. The last decades have seen a growing interest in the automatic detection of such states from voice, facial expression, and physiological signals, primarily with the goal of enhancing human-computer interaction with an affective component. With the advent of brain-computer interface research, the idea of affective brain-computer interfaces (aBCI), enabling affect detection from brain signals, arose. In this article, we set out to survey the field of neurophysiology-based affect detection. We outline possible applications of aBCI in a general taxonomy of brain-computer interface approaches and introduce the core concepts of affect and their neurophysiological fundamentals. We show that there is a growing body of literature that evidences the capabilities, but also the limitations and challenges of affect detection from neurophysiological activity

show abstract

User-centric Affective Video Tagging from MEG and Peripheral Physiological Responses

Cited by 20 publications

References 22 publications

A Multi-task Learning Framework for Time-continuous Emotion Estimation from Crowd Annotations

A Multi-task Learning Framework for Time-continuous Emotion Estimation from Crowd Annotations

Multimedia Implicit Tagging

A survey of affective brain computer interfaces: principles, state-of-the-art, and challenges

Contact Info

Product

Resources

About