Hybrid feature extraction and optimized deep convolutional neural network based video shot boundary detection

Abstract

Summary This paper presents a hybrid feature extraction and optimized deep learning model for the effective shot boundary detection (SBD). A cross‐guided bilateral filtering technique is used for pre‐processing. A hybrid fuzzy histogram with dual tree complex wavelet transform (FH‐DTCWT) is developed for the feature extraction to extract the visual features from the pre‐processed frames of the block. Candidate segment selection process is performed to identify the non‐boundary frames from the extracted features to improve the accuracy. Continuity matrix is created using the possible transition frames to verify that the frames are in sequential order without any gap. Classification of transition types are performed by the proposed optimized deep convolutional neural network. Optimized deep learning (DCNN‐RBESO) model is the combination of deep convolutional neural network (DCNN) and Rider bald eagle search optimization (RBESO) algorithm. To update the weight of DCNN model in learning, RBESO is utilized. MATLAB tool is used to develop the proposed SBD model. The experimental analysis is carried out for the proposed model using TREC Video Retrieval Evaluation (TRECVID) and VideoSeg datasets. The proposed model outperformed in the detection of shot boundaries in terms of precision, recall, and F1‐score.

Hybridized Neural Network Based Approaches Used for Video Shot Boundary Detection

“…It was built up of an input image M, a guided image G, and a filtered output image O. The position of identical pixels in each neighborhood is shown by the guide image G (12) . Whereas, guide picture G has been used in place of the input image.…”

“…The connection to robust estimation is misplaced as a result of this modification. This connection is reestablished to maintain the photometric weight P and produce G into a third weight T in cost J (12) . Proposed approach for SBD using DCNN with RBESO…”

“…Additionally, the filters get the filter coefficients in order to produce the DTCWT filter coefficients. The LL component, SD, and energy associated to sub bands may all be assessed at each step (12) . The DTCWT kth sub band's SD and energy calculations are expressed mathematically as follows:…”

“…Assuming that C is the input provided to the DCNN model and that the convolutional layer's output is, (C u x ) l,q : output from u th Conv centred as (l, q) θ 1 , θ 2 , and θ 3 : feature map created from each convolutional filter by scanning the input (12) . Additionally, the computation is made simpler and the negative values are removed using the activation function.…”

“…According to the current literature, several studies for the task of Video SBD have been reported. The issues like illumination effects and object/camera motion serve as a primary source of inspiration for contributing research in this field (12) . These are the challenges that encouraged to contribute the research in video shot boundary detection field.…”