A Novel Shot Detection Approach Based on ORB Fused With Structural Similarity

Abstract

Shots are the basic units for analyzing and retrieving video, and also the essential elements in creating video datasets. The traditional methods of shot detection exhibit unsatisfactory performance for being too sensitive to motion or too much time-consuming. This paper proposes an automatic shot detection method, by employing the fast feature descriptor of Oriented FAST and Rotated BRIEF (ORB) fused with Structural Similarity (SSIM). Firstly, ORB descriptor is used to preselect candidate segments with a high tolerance for rapidly extracting the features of twenty-frame intervals in video sequences. Then, the cut transition is detected by comparing ORB features, fused with SSIM, of consecutive frames in the candidate segment. Finally, the gradual transition is detected by determining the maximum amount of the continuous increasing/decreasing interframe differences in the candidate segment without cut transition. Experimental result indicates that the proposed method can achieve an F1-Score of 92.5% and five times of real-time speed with one CPU on 106049 test frames from the Open-video project, YouTube, and YOUKU. In addition, the proposed method can outperform the existing shot detection methods, including the rulebased and learning-based methods, by testing on the video sequences from the Open-video project and RAI dataset.

Tinjauan Literatur Sistematik tentang Structural Similarity Index Measure untuk Deteksi Anomali Gambar

“…Langkah-langkah yang dilakukan agar tujuan dari paper ini dapat tercapai adalah dengan meninjau literature yang ada. Dalam proses deteksi perbedaan gambar dan kualitas gambar tahapan pertama yang dilakukan adalah tahapan pengumpulan dataset, berbagai penelitian mengambil dataset dari berbagai sumber Open-video project, Youtube, YOUKU [4] . ORL(Olivetti Research Laboratory Cambridge), Caltecth dan Face96 [5] .…”

Video Shot Boundary Detection Based on Feature Fusion and Clustering Technique

“…In the application, feature detection and extraction usually can be divided into two directions based on processing compressed video domain and non-compressed video domain. The non-compressed domain method refers to the algorithms based on visual features, such as histogram [4] [9][10] [11] [12], pixel [13] [14] [15] [16], edge shape [17], motion [18] as well as orthogonal polynomial [19] [20] [21] [22]. While the compressed domain method refers to the algorithms based on compression coding, such as entropy coding including discrete cosine transform (DCT) and discrete Fourier transform (DFT) [12], macroblock coding [23], motion vector coding [24].…”

“…It reduces the computational complexity and improves the accuracy. Based on the point feature descriptors extracted by the ORB algorithm improved from SIFT and SURF, Liu et al [16] use the structural similarity index (SSIM) method to calculate the similarity of feature distance to estimate the similarity of frames, so as to achieve the detection of abrupt and gradual shot boundaries. Dhiman et al [12] implement the detection of abrupt and gradual shot boundary according to DCT feature matching and histogram feature matching respectively.…”

“…Especially, for the machine learning technology based on convolution neural network (CNN) which is popular in recent years, there are two significant limitations in shot boundary detection: running on expensive GPU and constructing the large-scale training dataset. In other words, the efficiency and accuracy of machine learning-based methods are largely determined by the performance of graphics hardware and the size of the training dataset [16]. In order to overcome these problems existing in current related algorithms, while without high performance of graphics hardware and large-scale training dataset, we propose a video shot boundary detection algorithm based on feature fusion and clustering technique (FFCT) in this paper.…”

“…The test for public standard datasets and comparison with relevant excellent algorithms can show whether the proposed algorithm is outstanding or not. For comparison algorithms of PSOGSA [4], ORB-SSIM [16], HLFPN-KM [26] and the proposed algorithm of FFCT, the experimental results of recall, precision and F1 for the public standard datasets of TRECVID 2001 and TRECVID 2007 are shown in TABLE 3. From the experimental results, for the dataset of TRECVID 2001, the proposed algorithm of FFCT and the comparison algorithms of PSOGSA, ORB-SSIM, HLFPN-KM may have some advantages in recall or precision.…”