Video salient object detection via spatiotemporal attention neural networks

Abstract

Recently, deep convolutional neural networks have been widely introduced into image salient object detection and achieve good performance in this community. However, as the complexity of video scenes, video salient object detection with deep learning models is still a challenge. The specific difficulties come from two aspects. First of all, the deep networks on image saliency detection cannot capture robust motion cues in video sequences. Secondly, as for the spatiotemporal fusing features, the existing methods simply exploit element-wise addition or concatenation, which not fully explores the contextual information and complementary correlation, thus they cannot produce more robust spatiotemporal features. To address these issues, we propose a two-stream based spatiotemporal attention neural network (STAN) for video salient object detection. We amply extract the motion information in terms of long short term memory (LSTM) network and 3D convolutional operation from optical flow-based prior and video sequences. Moreover, an attentive module is designed to integrate the different types of spatiotemporal feature maps by learning the corresponding weights. Meanwhile, in order to generate sufficient pixel-wise annotated video frames, we manually generate lots of coarse labels, which are well utilized to train a robust saliency prediction network. Experiments on the widely used challenging datasets (e.g., FBMS and DAVIS) prove that the proposed STAN has competitive performances among salient object detection methods.

Improved Deer Hunting Optimization Algorithm for Video- based Salient Object Detection

“…In 2019, Yi Tang et al [2], developed a 2-stream based STAN for video salient object detection. The motion information was sufficiently extracted regarding the LSTM network and 3D convolutional operation from video sequences and optical flow-based prior.…”

Visual Saliency Detection for Over-Temperature Regions in 3D Space via Dual-Source Images

“…In the field of target detection, deep learning is a commonly used technology. At present, in 2D target detection, many methods of optimizing the structure of deep convolutional neural networks improve the accuracy of target detection [ 14 , 15 , 16 ], such as fully convolutional networks (FCN), progressive fusion [ 17 ], multi-scale depth encoding [ 18 ], and data set balancing and smearing methods [ 19 , 20 ]. In mobile robot navigation, precise positioning of the target often requires obtaining spatial coordinates.…”

Convolutional neural network based object detection system for video surveillance application

“…16 However, the multi-object detection in images or videos is being the essential requirement of most of the real world problems. 7,[17][18][19][20] The construction of the useful detectors with modern deep learning models is more promising even when the "target objects are different, that is, different size, color, shape, and texture." Still, the object detection becomes complicated, when the target objects are small ("represented by a reduced number of pixels, similar size, shape, color, and texture").…”