Automatic video temporal segmentation based on multiple features

Abstract

This paper investigates automatic video temporal segmentation techniques, also named shot boundary detection (SBD) techniques. Firstly, the existing SBD algorithms are reviewed in detail. Then, a new SBD algorithm is proposed aiming to obtain fast and accurate detection, and its performances are evaluated and compared with existing works. This algorithm computes the frame difference/similarity by such simple features as pixel difference and histogram difference, adopts motion-based difference to resist camera or object movements in the same shot and uses the flash detection to avoid false positives caused by light changes or flashes. The adopted features are computational efficient, and the combination of various features improve the detection accuracy. These properties make the algorithm suitable for real-time applications, such as broadcasted news segmentation.

Temporal segmentation tool for high-quality real-time video editing software

“…As evaluation measures, because they are very commonly used in the literature [26], we have used the Recall (R) and Precision (P) percentages,…”

“…On the other hand, methods based on motion analysis and those that combine several strategies are able to get better results, detecting most of the existing transitions and reducing the amount of false detections [26] . However, these methods are much more complex and considerably slower than previous ones [27].…”

Transition Detection Using Hilbert Transform and Texture Features

“…The experimental results of the existing methods like correlation based [6], Color layout descriptor [10], multiple features [8] and HSV histogram differences [9] are compared with our proposed method. The results for overall transitions in the video sequences are considered.…”

“…The main drawback of this method is the use of predefined threshold value for each stage of the detection process. As an improvement to the method [7], Lian in his work [8] has used multiple features (pixel wise differences, color histogram, motion) which detect transitions in serial manner using different threshold values. The number of thresholds used is comparatively less than the previous method [7].…”

Towards key-frame extraction methods for 3D video: a review

“…The decision methods used to find shot boundaries can be based on static thresholds (as in Fig. 2), adaptive thresholds (thresholds depend on the statistics of the visual features used), B-splines fittings [30], support vector machines (SVM) [31] and K-means clustering [11]. The detection accuracy of SBD methods is improved by combining several visual features [32].…”