Moviescope: Large-scale Analysis of Movies using Multiple Modalities

Abstract

Film media is a rich form of artistic expression. Unlike photography, and short videos, movies contain a storyline that is deliberately complex and intricate in order to engage its audience. In this paper we present a large scale study comparing the effectiveness of visual, audio, text, and metadata-based features for predicting high-level information about movies such as their genre or estimated budget. We demonstrate the usefulness of content-based methods in this domain in contrast to human-based and metadatabased predictions in the era of deep learning. Additionally, we provide a comprehensive study of temporal feature aggregation methods for representing video and text and find that simple pooling operations are effective in this domain. We also show to what extent different modalities are complementary to each other. To this end, we also introduce Moviescope, a new large-scale dataset of 5,000 movies with corresponding movie trailers (video + audio), movie posters (images), movie plots (text), and metadata. * Indicates equal author contribution.† Work was conducted while affiliated with the University of Virginia.

Film Trailer Generation via Task Decomposition

“…To evaluate the trailers produced by our algorithm, we also collected a new held-out set of 41 movies. These movies were selected from the Moviescope dataset 5 [11], which contains official movie trailers. The held-out set does not contain any additional information, such as screenplays or TP annotations.…”

“…Accordingly, for the video-based network which operates over shots, we first aggregate shot probabilities to scene-level ones (see Sections 3. 3 and 4) and then compute the KL divergence between the aggregated probabilities and the teacher distribution (loss V in Equation ( 8), Section 3.3), in the same fashion as in Equation (11).…”

“…How Sentiment Connects with Trailers Empirical rules for trailer making 11 suggest that a trailer should start with shots of medium intensity to captivate the viewers, then decrease the sentiment intensity in order to deliver key information about the movie, and finally build up the tension until it reaches a climax.…”

“…During graph sparsification (i.e., selection of top-k neighbors), we consider different neighborhood options for the scene-and shot-based networks due to their different granularity and size. Following [42], we consider [1][2][3][4][5][6] neighbors for the scene network and we increase the neighborhood size to [6][7][8][9][10][11][12] for the shot network.…”

A Case Study of Deep Learning-Based Multi-Modal Methods for Labeling the Presence of Questionable Content in Movie Trailers

“…However, the aforementioned dataset only includes movie scripts and corresponding metadata but does not include movie trailers or related age-suitability tags (As we mentioned earlier, the MPAA rating scheme is different for movies and trailer). In (Cascante-Bonilla et al, 2019), authors introduced Moviescope, a dataset for movie genre classification. Similarly, it does not include MPAA age-suitability rating labels for movie trailers.…”