CineScale: A dataset of cinematic shot scale in movies

Abstract

We provide a database containing shot scale annotations (i.e., the apparent distance of the camera from the subject of a filmed scene) for more than 792,000 image frames. Frames belong to 124 full movies from the entire filmographies by 6 important directors: Martin Scorsese, Jean-Luc Godard, Béla Tarr, Federico Fellini, Michelangelo Antonioni, and Ingmar Bergman. Each frame, extracted from videos at 1 frame per second, is annotated on the following scale categories: Extreme Close Up (ECU), Close Up (CU), Medium Close Up (MCU), Medium Shot (MS), Medium Long Shot (MLS), Long Shot (LS), Extreme Long Shot (ELS), Foreground Shot (FS), and Insert Shots (IS). Two independent coders annotated all frames from the 124 movies, whilst a third one checked their coding and made decisions in cases of disagreement. The CineScale database enables AI-driven interpretation of shot scale data and opens to a large set of research activities related to the automatic visual analysis of cinematic material, such as the automatic recognition of the director’s style, or the unfolding of the relationship between shot scale and the viewers’ emotional experience. To these purposes, we also provide the model and the code for building a Convolutional Neural Network (CNN) architecture for automated shot scale recognition. All this material is provided through the project website , where video frames can also be requested to authors, for research purposes under fair use.

The One Where They Reconstructed 3D Humans and Environments in TV Shows

“…In parallel, from our reconstructed humans we can extract reliable gaze information, which can outperform specialized gaze estimation [46]. Moreover, our results provide estimates of the camera-to-person distance, which is relevant for cinematography applications [50,51]. Finally, we illustrate the potential use in image editing applications, like object insertion or human deletion.…”

Historian: A Large-Scale Historical Film Dataset with Cinematographic Annotation

“…Based on video sequences gathered from TRECVID [6], RAI [17], Youtube or Weibo, they have published the datasets DeepSBD [15] and Clipshots [16]. A further dataset, called Cinescale [4] provides a fundamental base for exploring cinematographic shot types [3] in professional film recordings. It contains a massive number of frame-based labels (792k) that have been extracted from 124 art movies (made between 1949 and 2013).…”

“…Less research focus is put towards digitized original historical film [11,12,21]. Similar to [4], HistShotDS [21], contains frame-based shot type annotations of selected frames. However, this dataset lacks due to the limited number of available samples as well as the missing temporal shot information.…”

“…Automatic video analysis is an ongoing research field including different subdomains such as scene understanding [1,2] or cinematographic style analysis [3]. In order to deal with those topics, a number of datasets [4][5][6] have been published. They include mainly short sequences or single frames extracted from short video clips.…”

“…One shot is a continuously recorded situation characterized by specific cinematographic settings. One fundamental cinematographic setting is the Shot Type, which is a representation of the distance between the subject of interest and the camera lens [4]. Camera movements like pans, tilts, zooms or tracks [10] are another example of cinematographic properties used in professional filmmaking.…”