A Sieving ANN for Emotion-Based Movie Clip Classification

Abstract

SaowalukC. WATANAPA•õa), Member, Bundit THIPAKORN•õ•õb), and Nipon CHAROENKITKARN•õc), Nonmembers SUMMARY Effective classification and analysis of semantic contents are very important for the content-based indexing and retrieval of video database. Our research attempts to classify movie clips into three groups of commonly elicited emotions, namely excitement, joy and sadness, based on a set of abstract-level semantic features extracted from the film sequence. In particular, these features consist of six visual and audio measures grounded on the artistic film theories. A unique sieving-structured neural network is proposed to be the classifying model due to its robustness. The performance of the proposed model is tested with 101 movie clips excerpted from 24 award-winning and well-known Hollywood feature films. The experimental result of 97.8% correct classification rate, measured against the collected human judges, indicates the great potential of using abstract-level semantic features as an engineered tool for the application of video-content retrieval/indexing.

Video Affective Content Analysis: A Survey of State-of-the-Art Methods

“…The most frequently used categories in the field of video affective content analysis are Ekman's six basic emotions [6], including happiness, sadness, anger, disgust, fear, and surprise [8], [9], [10], [11], [12], [13], [14], [15], [16], [17]. [36] In addition, other categories, such as amusement [18], [19], boredom [20], excitement [21], [22], or horror [23], [24], [25], [26], [20] are also used to describe affective content of videos for certain applications. Dimensional views of emotion have been advocated and applied by a several researchers.…”

“…Psychological research has shown that psycho-physiological characteristics like air intake, vocal muscle, intonation and pitch characteristics vary with emotions [55]. Based on such physiological studies, prosodic speech features are typically used to characterize the affective content of an utterance, since prosody captures the emotional state of the speaker through its non-lexical elements including the rhythm, stress, and intonation of speech [22]. Typical prosodic speech features include loudness, speech rate, pitch, inflection, rhythm, and voice quality.…”

“…Widely used music features include dynamics, timbre, harmony, and rhythm. In addition, many audio features are shared among different audio types including energy related features [19], [27], [38], [44], [24], [46], [2], [47], [10], [11], [15], [22], [23], [50], [31], ZCR [19], [27], [38], [11], [15], [49], [50], [31], MFCC [19], [27], [24], [11], [15], [23], [49], [50], spectral centroid [27], [24], [50], spectral rolloff [19], [27], [30], [11], spectral flux [19], [27], [30], [15], etc. These features are used for both emotional dimension prediction and emotional category classification.…”

“…Shot is an important film element that can control a video's tempo. Shot-related features [14], [11], [22], [48], [21], [49], [50] include shot duration, shot transition, and shot type transition. Shot duration represents the length of a shot.…”

A Two-Stage Classifier That Identifies Charge and Punishment under Criminal Law of Civil Law System

“…This type of ANN architecture has also been successfully applied in some other domains. For example, the environmental impact of ICT and e-business [15], the emotion-based classification of movie clips [16], and English pronunciation reasoning and protein prediction [17].…”