Affective Video Retrieval: Violence Detection in Hollywood Movies by Large-Scale Segmental Feature Extraction

Abstract

Without doubt general video and sound, as found in large multimedia archives, carry emotional information. Thus, audio and video retrieval by certain emotional categories or dimensions could play a central role for tomorrow's intelligent systems, enabling search for movies with a particular mood, computer aided scene and sound design in order to elicit certain emotions in the audience, etc. Yet, the lion's share of research in affective computing is exclusively focusing on signals conveyed by humans, such as affective speech. Uniting the fields of multimedia retrieval and affective computing is believed to lend to a multiplicity of interesting retrieval applications, and at the same time to benefit affective computing research, by moving its methodology “out of the lab” to real-world, diverse data. In this contribution, we address the problem of finding “disturbing” scenes in movies, a scenario that is highly relevant for computer-aided parental guidance. We apply large-scale segmental feature extraction combined with audio-visual classification to the particular task of detecting violence. Our system performs fully data-driven analysis including automatic segmentation. We evaluate the system in terms of mean average precision (MAP) on the official data set of the MediaEval 2012 evaluation campaign's Affect Task, which consists of 18 original Hollywood movies, achieving up to .398 MAP on unseen test data in full realism. An in-depth analysis of the worth of individual features with respect to the target class and the system errors is carried out and reveals the importance of peak-related audio feature extraction and low-level histogram-based video analysis.

Affective Video Content Analysis: A Multidisciplinary Insight

“…One year later, to classify video excerpts as violent or non-violent, Eyben et al fused by simple score averaging the predictions made by acoustic and visual linear kernel SVMs [75]. To capture temporal dynamics, numerous statistics are computed for the features over fixed size windows.…”

A Hybrid Latent Space Data Fusion Method for Multimodal Emotion Recognition

“…According to the previous studies [59], [62], [66], [68], [69], In addition to the color energy, for each n × m frame, the 16 histogram of hue in the HSV space is considered as color features. 4) Motion: It has been shown that there is a relationship between camera movement and emotions such as happiness, sadness, and fear [70].…”

VITEC: A Violence Detection Framework

“…Firstly, the systems work as a witness to violent events. Included in this approach are violence detection using surveillance video signals [16][17][18] and physiological signals measured from the subjects who were watching the violent scenes, e.g. EDA [19], EEG [20].…”

“…Secondly, the systems use signals measured from the victims directly, for example, their human body movement [21] and ECG signal [22]. Nam, Alghoniemy, and Tewfik pioneered a study on violent scene characterization based on flame and blood detection [16]. It opened a research pathway on violence detection using video.…”