Computational Model of Film Editing for Interactive Storytelling

Abstract

International audienceGenerating interactive narratives as movies requires knowledge in cinematography (camera placement, framing, lighting) and film editing (cutting between cameras). We present a framework for generat- ing a well-edited movie from interactively generated scene contents and cameras. Our system computes a sequence of shots by simultaneously choosing which camera to use, when to cut in and out of the shot, and which camera to cut to.La génération de contenus narratifs interactifs sous forme d'animation 3D nécessite des connaissances en cinématographie (placement des caméras, cadrage, éclairage) et en montage. Nous proposons une approche permettant de générer automatiquement des films cadrés et montés selon les règles de l'art à partir d'animations 3D et de caméras virtuelles créés dynamiquement. Notre système calcule la séquence des plans du montage final et les instants de coupe entre les plans

GAZED– Gaze-guided Cinematic Editing of Wide-Angle Monocular Video Recordings

“…Another series of papers pose video editing as a discrete optimization problem, solved using dynamic programming [12,21,13,22]. The key idea is to define the importance of each shot based on the narrative, and solve for the best set of shots that maximize viewer engagement.…”

A Virtual Director Using Hidden Markov Models

“…A number of contributions actually rely on this hypothesis (see [ER07, LCL*10, LCCR11]). Authors [LCRB11] proposed this probabilistic representation where states encode shots and transitions are activated by events.…”

“…Computing the best sequence of frames consists in searching for the best frames, the best idioms and the best transitions between idioms by exploring and evaluating all the possibilities in the film tree, similar to a combinatorial branching search. By far the most general representation in cinematographic editing, inspiring many approaches [LCCR11, ER07], film‐trees actually require the provision of many parameters and weights to guide the search process, therefore hampering their use in practice.…”

“…moral positions of characters) rather than plain events adopted by a large number of techniques (e.g. a character is talking) [HCS96, ER07, LCL*10, LCCR11] a general representation of the editing process that enables learning and reproducing elements of style from real sequences, with variations in the pacing (rhythm of shots) and possibility to add constraints. a representation which is independent of the geometry and relies on a geometric solver to place cameras automatically.…”

The Prose Storyboard Language: A Tool for Annotating and Directing Movies

“…Other previous work in virtual cinematography [29,14,10,21,16,18] has been limited to simple shots with either a static camera or a single uniform camera movement. Our prose storyboard language is complementary to such previous work and can be used to define higher-level cinematic strategies, including arbitrarily complex combinations of camera and actor movements, for most existing virtual cinematography systems.…”