Video summarization based on the psychological unfolding of a drama

Abstract

SUMMARYThis paper proposes a method of composing digests of TV videos based on their psychological unfolding. Most of the studies of video summarization to date have been based on the video structure, such as cuts or objects in the video. It is generally considered, however, that the observer of a TV drama emphasizes instead the psychological unfolding of the video content (such as the climax). However, no method of summarizing a drama based on the psychological aspect of the video has been proposed. Track structures [such as cutting, lines spoken by the actors, BGM (background music), and sound effects] constitute physical components, and the dramatist skillfully manipulates the time structure based on his empirical knowledge in order to present a particular psychological impression. This paper proposes a method in which the psychologically important part is detected on the basis of empirical knowledge as a corresponding time-series pattern of the track structure, and a summarized video is generated. As an experiment, the proposed method was applied to a TV drama, and the time length was reduced to approximately 1/14. Subjective evaluation experiments show that the proposed method preserves the content of the main plot, with naturalness of extraction, thus indicating the usefulness of the proposed method.

Elvis

“…Verifying the effectiveness of video summaries, by measuring the extent to which video sub-segments overlap with a benchmark selection of video subsegments, is a recognised verification approach, and hence followed by a number of video summarisation studies [e.g. Babaguchi et al 2004;Moriyama and Sakauchi 2002;Rui et al 2000]. Figure 6 provides an overview of the process adopted to evaluate the ELVIS and RANDOM video sub-segment selections.…”