Affective video content representation and modeling

Abstract

Abstract-This paper looks into a new direction in video content analysis -the representation and modeling of affective video content. The affective content of a given video clip can be defined as the intensity and type of feeling or emotion (both are referred to as affect) that are expected to arise in the user while watching that clip. The availability of methodologies for automatically extracting this type of video content will extend the current scope of possibilities for video indexing and retrieval. For instance, we will be able to search for the funniest or the most thrilling parts of a movie, or the most exciting events of a sport program. Furthermore, as the user may want to select a movie not only based on its genre, cast, director and story content, but also on its prevailing mood, the affective content analysis is also likely to contribute to enhancing the quality of personalizing the video delivery to the user. We propose in this paper a computational framework for affective video content representation and modeling. This framework is based on the dimensional approach to affect that is known from the field of psychophysiology. According to this approach, the affective video content can be represented as a set of points in the two-dimensional (2-D) emotion space that is characterized by the dimensions of arousal (intensity of affect) and valence (type of affect). We map the affective video content onto the 2-D emotion space by using the models that link the arousal and valence dimensions to low-level features extracted from video data. This results in the arousal and valence time curves that, either considered separately or combined into the so-called affect curve, are introduced as reliable representations of expected transitions from one feeling to another along a video, as perceived by a viewer.Index Terms-Affective video content analysis, video abstraction, video content modeling, video content representation, video highlights extraction.

Movie Analysis Based on Roles' Social Network

“…Some researches tackle with movie analysis from different perspectives. The so-called "affective content analysis" investigates human's perception drawn by audiovisual stimuli [5] [6]. However, the reported methods still rely on modeling lowlevel features and don't approach to "understand" movie content.…”

Structured Learning from Heterogeneous Behavior for Social Identity Linkage

“…According to studies on sentiment mining [31], [32], we can model the sentiment pattern using a two dimensional space (arousal-valence) [31] or roughly divide the emotion into several categories, e.g., happy/ fear /sad /neutral. It can be done by extracting the representative emotional key words in the textual content and learning a sentiment vocabulary.…”

Human-centered implicit tagging: Overview and perspectives

“…A summary of the recent relevant literature on this topic is given in Table I. There have been also studies using unnimodal or multimodal approaches for detecting the behavioral or emotional responses to multimedia [21], [23], [24], [20], [6]. There is currently a research trend towards estimating emotions from multimedia content automatically [13], [15], [14]. The emotion recognition has been also used in applications such as detecting topical relevance, or summarizing videos [11], [21], [9].…”

“…Parallel to this approach to indexing, an alternative has also emerged that also take affective aspects into account. Here, affect refers to the intensity and type of emotion that is evoked in a user while watching/listening to multimedia content [13]. Affective characteristics of multimedia are important features for describing multimedia content and can be presented by relevant emotional tags.…”