Motion-decision based spatiotemporal saliency for video sequences

Abstract

An adaptive spatiotemporal saliency algorithm for video attention detection using motion vector decision is proposed, motivated by the importance of motion information in video sequences for human visual system. This novel system can detect the saliency regions quickly by using only part of the classic saliency features in each iteration. Motion vectors calculated by block matching and optical flow are used to determine the decision condition. When significant motion contrast occurs (decision condition is satisfied), the saliency area is detected by motion and intensity features. Otherwise, when motion contrast is low, color and orientation features are added to form a more detailed saliency map. Experimental results show that the proposed algorithm can detect salient objects and actions in video sequences robustly and efficiently.

TV Commercial Detection Using Success Based Locally Weighted Kernel Combination

“…The Motion Distribution is a significant feature as many previous works have indicated that commercial shots mostly have high motion content as they try to convey maximum information in minimum possible time. This motivates us to compute dense optical flow (Horn-Schunk formulation) between consecutive frames and construct a distribution of flow magnitudes over the entire shot with 40 uniformly divided bins in range of [0, 40] [5], [30]. Often pixel intensities of regions suddenly change while the boundaries of the region do not move.…”

“…We have used existing features from the literature viz. shot length [29], scene motion distribution [5], [30], overlay text distribution [8], zero crossing rate [31], [6], short time energy (STE) [6], fundamental frequency, spectral centroid, flux and roll-off frequency [8] and MFCC Bag of Words [32]. We observed that, SVMs trained on a certain set of features fail to detect the commercial shots when ever the basic assumption involving those features are violated.…”