Multi-clip video editing from a single viewpoint

Abstract

We propose a framework for automatically generating multiple clips suitable for video editing by simulating pan-tilt-zoom camera movements within the frame of a single static camera. Assuming important actors and objects can be localized using computer vision techniques, our method requires only minimal user input to define the subject matter of each sub-clip. The composition of each sub-clip is automatically computed in a novel L1-norm optimization framework. Our approach encodes several common cinematographic practices into a single convex cost function minimization problem, resulting in aesthetically pleasing sub-clips which can easily be edited together using off-the-shelf multi-clip video editing software. We demonstrate our approach on five video sequences of a live theatre performance by generating multiple synchronized subclips for each sequence.

Zooming On All Actors: Automatic Focus+Context Split Screen Video Generation

“…Moreover, the conventional filtering based approaches only suppresses high frequency jitter and may not mimic the behaviour of a professional cameraman. Our work is closely related to recent L(1)-norm based camera stabilization/simulation methods [GKE11,GRG14]. The work in [GL08] showed that the trajectories of professional camera work can be modeled as distinct static, linear and parabolic segments and used this idea to stabilize noisy recordings.…”

“…Grundmann et al [GKE11] showed this can be implemented using a linear optimization framework. Gandhi et al [GRG14] extended this idea for virtual camera simulation and presented an application of multiclip editing from single viewpoint. These algorithms are focused on creating smooth camera movements, considering one trajectory at a time.…”

Video Interaction Tools

“…-Video Editing: Another related research field is video editing, where it is important to provide a well-structured content overview as well as efficient video navigation means. While some recent works can be found in the literature [Gandhi et al 2014], many works in this field, however, were published many years ago [Girgensohn et al 2000;Hua et al 2003;Casares et al 2002;Cattelan et al 2008]. It seems that more recent works focus on some special aspects of video editing that are more generous and can be used also for other purposes than video editing (e.g., for interactive search or for navigation in video).…”

Capturing and Indexing Rehearsals: The Design and Usage of a Digital Archive of Performing Arts

“…To remedy this problem, we implemented a "spatial zoom lens" to make it possible to focus on the main actors in any given scene. Based on audio and video analysis, we generated alternative framings using methods described elsewhere [5], [6]. For example, during a monologue, we automatically compute a medium-shot of the speaker, and during a dialogue, we compute a two-shot that contains the speaker and his addressee (see details in Figure 6).…”

“…lens [5], [6], which makes it possible for researchers and amateurs alike to use the archive as stock footage, which they can re-frame and re-edit according to their own interpretation (this is described in Section V).…”