Temporal Video Segmentation Using Optical Flow Estimation

Abstract

Shot boundary detection is the process of segmenting a video into basic units known as shots by discovering transition frames between shots. Researches have been conducted to accurately detect the shot boundaries. However, the acceleration of the shot detection process with higher accuracy needs improvement. A new method was introduced in this paper to find out the boundaries of abrupt shots in the video with high accuracy and lower computational cost. The proposed method consists of two stages. First, projection features were used to distinguish non boundary transitions and candidate transitions that may contain abrupt boundary. Only candidate transitions were conserved for next stage. Thus, the speed of shot detection was improved by reducing the detection scope. In the second stage, the candidate segments were refined using motion feature derived from the optical flow to remove non boundary frames. The results manifest that the proposed method achieved excellent detection accuracy (0.98 according to F-Score) and effectively speeded up detection process. In addition, the comparative analysis results confirmed the superior performance of the proposed method versus other methods.

Temporal Action Segmentation in Sign Language System for Bahasa Indonesia (SIBI) Videos Using Optical Flow-Based Approach

“…Furthermore, using optical flow in sign language video segmentation addresses the challenge of co-articulation, where the previous sign influences one sign. The motion feature that uses Optical Flow can be used to detect shot boundaries [21]. In this research, shot boundaries are detected using the projection feature to obtain candidate boundary frames and the motion feature to remove non-boundary frames from the candidate frames provided by the projection feature.…”