Compressed-Domain Correlates of Fixations in Video

Video saliency detection incorporating temporal information in compressed domain

“…Besides, it can be observed that motion information on backgrounds are often consistent, dominant in number and enclosed by the camera motion no matter whether the camera moves or not. In contrast, the motion information of moving objects in most of the cases are incompatible with the dominant camera motion [29,24].…”

“…Thus, how to precisely and effectively measure conspicuous maps in compressed domain becomes an imperative and tough problem. So far, only a limited number of algorithms have been proposed for video saliency detection in compressed domain [21][22][23][24]. In [21], Fang et al compiled saliency maps with adaptive entropy-based spatial and temporal uncertainty weighting.…”

“…Muthuswamy et al [23] used the DCT coefficients of luminance and chroma components to calculate spatial saliency maps, and motion vectors (MVs) were utilized to refine it with a component-wise multiplication. In [24], motion information was used to detect temporal saliency maps by entropy and l 0 norm of DCT coefficients was used to detect spatial saliency maps, and then the final video saliency maps were generated by both additive and multiplicative combination. From these compressed video saliency detection algorithms, it seems obvious that the performances of saliency detection in compressed domain are better than that of saliency detection in uncompressed domain.…”

“…From these compressed video saliency detection algorithms, it seems obvious that the performances of saliency detection in compressed domain are better than that of saliency detection in uncompressed domain. The major causes for this end are that compressed video saliency models are not a stand-alone entities, the features used in these models are the best choice from video encoder whose aim is to provide the most compact representation of video, such as motion tracking and conspicuous objects selecting [24].…”

“…Another important problem is the fusion of different saliency maps. Although the features used in [23,24] are obtained from several consecutive frames to reduce the influence of encoder, the fusion method, namely component-wise multiplication, cannot effectively reflect which saliency map is more important, resulting in inaccurate saliency detection, such as the incomplete contour of foreground objects and background noise. Therefore, to overcome these shortcomings, we propose a video saliency detection model in compressed domain illustrated in Fig.…”