Foveated shot detection for video segmentation

Abstract

We view scenes in the real world by moving our eyes three to four times each second and integrating information across subsequent fixations (foveation points). By taking advantage of this fact, in this paper we propose an original approach to partitioning of a video into shots based on a foveated representation of the video. More precisely, the shot-change detection method is related to the computation, at each time instant, of a consistency measure of the fixation sequences generated by an ideal observer looking at the video. The proposed scheme aims at detecting both abrupt and gradual transitions between shots using a single technique, rather than a set of dedicated methods. Results on videos of various content types are reported and validate the proposed approach

Gaze shifts as dynamical random sampling

“…[10], [11] , [12], [13], [14] or more traditional computer vision and robotics [20], [21]. Other applications of interest could involve scanpath simulation for quantitative evaluation and performance assessment [30].…”

Fuzzy color histogram-based video segmentation

“…Grana et al [3] propose a two-step iterative algorithm, unique for both cuts and gradual transitions detection, in the presence of fast object motion and camera operations. Boccignone et al [1] use a consistency measure of the fixation sequences generated by an ideal observer looking at the video for determining shot changes. A scene-break detection approach based on linear prediction model is proposed in [4].…”

“…The detection of shots, as in many video indexing and retrieval applications, is the first step of video analysis. A shot is defined as a series of related consecutive frames representing a continuous action in time and space taken by a single camera [1].…”

“…Various shot-boundary detection algorithms have been proposed [1][2][3][4][5][6] and compared [7][8][9][10]; however, to the best of our knowledge, no shot-boundary detection algorithm specialized for CBCD is found in the literature. Our aim is to propose an automatic shot-boundary detection algorithm for the videos on which various transformations are applied.…”

Video Classification and Shot Detection for Video Retrieval Applications

“…Human view the visual media by moving their eyes three to four times each second and incorporating information across foveation points. 22 From the observer's perspective 23 shots are detected based on a foveated representation of the video.…”