Wipe Scene Change Detection in Object-Camera Motion Based on Linear Regression and an Inflated Spatial-Motion Neural Network

Abstract

To facilitate content-based video analysis, automatic scene change detection (SCD) with large-scale motion activity is an essential fundamental step for locating a transition from one video scene to another. With the exponential increase in digital media usage, SCD has become more challenging in processing large motion content with minimal information loss and maximum perseverance. Wipe SCD in object-camera motion is noticeable evidence of this issue. Wipe transitions, which are a type of gradual transition, have diverse motion pattern changes when influenced by object-camera motion (camera pan, large-object, and zoom-in/out), creating a velocity imbalance in the same frame. Furthermore, this motion imbalance leads to false detection. Due to the loss of motion information and longer processing time of existing frameworks, we propose a novel method of wipe scene change detection (WSCD) based on deep spatial-motion feature analysis. First, large input videos are segmented into shots using dimensionality reduction and adaptive threshold. Secondly, linear regression is used to compute slope angle changes in shots for candidate selection and wipe localization. Finally, only selected candidates are processed to extract features using a two-stream inflated 3D-convolutional neural network for RGB stream and optical flow velocity for motion stream network (I3DCNN) and then classified into wipe in-motion and no-motion clips. The experimental results are obtained by classifying wipe patterns using a detection reviewing and merging strategy on corresponding wipe frames. The average improvement in wipe scene change detection accuracy evaluated on the benchmark TRECVID dataset is 11.9%, demonstrating the efficacy of our proposed method.

Semantic Scene Object-Camera Motion Recognition for Scene Transition Detection Using Dense Spatial Frame Segments and Temporal Trajectory Analysis

“…In addition, the method did not investigate wipe transition patterns in object and camera movements. Addressing this issue, recently, Chakraborty et al [2] proposed spatial and optical flow motion vector features-based inflated 3D CNN network that classified 12 types of wipe transition patterns in high-motion activity frames. However, this method suffered a few misclassifications in the presence of multiple objects and camera motion flow in a single transition frame.…”

“…To capture the content's semantic features at spatial and temporal dimensions, existing video content analysis models perform shot or scene transition detection first and then process the non-transition frames, also referred to as normal frames. This approach is timeconsuming, prone to generating false alarms in the scene transition detection step due to the presence of object-camera motion, and often results in loss of semantic motion information [2]. In this article, object-camera motion refers to object movement, camera panning movement, and camera zoomin/out movement.…”

“…1. For a detailed definition of various wipe transition patterns, readers are referred to [2]. This technique enhances the presentation of the story-telling feature of content in various media platforms, including sports, news, movies, documentaries, animation, and more [6], [7].…”

“…Moreover, the detection of wipe scene transition patterns during complex object-camera motions was even less explored in the stateof-the-art methods [8], [9], [10]. Addressing this issue, the WSCD method was proposed based on PCA features and the I3DCNN network to classify various wipe scene transition patterns in the presence of object-camera motions [2]. The WSCD method achieved 94.90% precision and 95.30% recall rates, indicating that there is still room for improvement.…”

“…The classification of object and camera motion flow enhances the performance of scene transition detection. Hence, in this article, we aim to improve the existing wipe scene transition method proposed in [2] by introducing pixel-level semantic spatial features and temporal tracking of respective object and camera motion feature trajectories.…”