Scene Segmentation and Categorization Using NCuts

Abstract

For video summarization and retrieval, one of the important modules is to group temporal-spatial coherent shots into high-level semantic video clips namely scene segmentation. In this paper, we propose a novel scene segmentation and categorization approach using normalized graph cuts(NCuts). Starting from a set of shots, we first calculate shot similarity from shot key frames. Then by modeling scene segmentation as a graph partition problem where each node is a shot and the weight of edge represents the similarity between two shots, we employ NCuts to find the optimal scene segmentation and automatically decide the optimum scene number by Q function. To discover more useful information from scenes, we analyze the temporal layout patterns of shots, and automatically categorize scenes into two different types, i.e. parallel event scenes and serial event scenes. Extensive experiments are tested on movie, and TV series. The promising results demonstrate that the proposed NCuts based scene segmentation and categorization methods are effective in practice.

Constraint Score: A new filter method for feature selection with pairwise constraints

“…It's worthy to note that one should neither confuse the pairwise constraints mentioned in this paper with the pairwise similarity or dissimilarity value used in spectral graph based algorithms [6][18] [22][24], nor with some class pairwise methods [9]. In spectral graph based algorithms, one first computes the pairwise similarity or dissimilarity between samples to form the similarity or dissimilarity matrix, and then perform subsequent operations on it.…”

Static Summarization of Video Scenes Based on Minimal Spanning Tree

“…A heuristic approach based on the quasi-periodic pattern of the shots [3] is presented to detect the scenes in a video. Various Graph based schemes are also reported in [4,5].…”

Differential Edit Distance: A Metric for Scene Segmentation Evaluation

“…In [7] a visual bag-ofwords approach is proposed for decomposing the video into scenes, which for the purpose of evaluation are compared to the ground-truth using the Coverage and Overflow measures. The authors of [8] present a graph-based scene segmentation approach, which uses normalized cuts; evaluation is conducted with the help of the Precision-Recall measures. [9] proposes a probabilistic technique that aims to maximize the PrecisionRecall values of the estimated scene boundaries, by training a number of independent descriptors based on various modalities, with Precision-Recall again being used for its evaluation.…”