Automatic Social Network Construction from Movies Using Film-Editing Cues

Abstract

In this paper, we investigate the problem of automatically constructing characters' social network from movies. Unlike existing approaches that use co-appearance information to measure the relationship between two characters, we argue that a method that describes the characters' interaction, rather than the co-appearance, makes more sense. We propose a new scheme that quantifies the interaction of characters by the use of film-editing cues, based on which we construct the characters' social network. Experiments on real-world data validate the effectiveness of the proposed method. In addition, we show an application of discovering characters' social clusters enabled by the automatically constructed social network.

Extraction and Analysis of Fictional Character Networks

“…They experimentally find the community structure of the extracted networks to be relatively robust to face detection errors. A number of authors proceed automatically using off-the-shelf tools [148,177,190,252,273].…”

“…Face track clustering consists in identifying groups of face tracks (output from the occurrence detection step) corresponding to the same face, hence character. Certain authors use off-the-shelf tools [148,252,273], but others consider that these generic methods are not sufficiently efficient when applied to fiction works [267]. Their results can be improved by training them on a corpus of such works [268,269], but this requires additional work and resources.…”

“…The shot alternation (or shot/countershot) rule is particularly used during conversations: it specifies that consecutive shots alternatively show the involved characters. Yeh et al [273] leverage both of them to improve character detection in movies. Comics and animated films are apart, as in their cases, the medium itself is unlike anything related to real-life.…”

“…For co-reference resolution, the problem comes from longer sentences, more frequent use of pronouns and direct speech, more numerous and shorter co-reference chains [130]. Similarly to text, certain characteristics of fiction works make generic audiovisual processing tools inefficient [273]. For instance, movie directors use a variety of complex, possibly genre-related, editing techniques [147].…”

“…In audiovisual narratives, detecting character occurrences amounts to solving several distinct but related problems, depending on whether one focuses on the video or the audio stream. In videos, these are face detection and face tracking [273,274].…”