Recurrent Attentional Networks for Saliency Detection

Kuen, Jason; Wang, Zhenhua; Wang, Gang

doi:10.1109/cvpr.2016.399

Cited by 209 publications

(112 citation statements)

References 43 publications

Supporting

Mentioning

112

Contrasting

Order By: Relevance

“…In addition to natural language processing, the attention mechanism has also been used in computer vision tasks. Examples include visual (and multi-modal) questionanswering tasks [5], [6], [48], image caption generation [4], [49], image classification [50], [51], [52], [53], action recognition [54], saliency detection [55], and so on. Attention mechanism can also be used as a long-term memory mechanism for NNs.…”

Section: Related Workmentioning

confidence: 99%

A^3: Accelerating Attention Mechanisms in Neural Networks with Approximation

Ham

Jung

Kim

et al. 2020

2020 IEEE International Symposium on High Performance Computer Architecture (HPCA)

108

View full text Add to dashboard Cite

With the increasing computational demands of neural networks, many hardware accelerators for the neural networks have been proposed. Such existing neural network accelerators often focus on popular neural network types such as convolutional neural networks (CNNs) and recurrent neural networks (RNNs); however, not much attention has been paid to attention mechanisms, an emerging neural network primitive that enables neural networks to retrieve most relevant information from a knowledge-base, external memory, or past states. The attention mechanism is widely adopted by many state-of-the-art neural networks for computer vision, natural language processing, and machine translation, and accounts for a large portion of total execution time. We observe today's practice of implementing this mechanism using matrix-vector multiplication is suboptimal as the attention mechanism is semantically a content-based search where a large portion of computations ends up not being used. Based on this observation, we design and architect A 3 , which accelerates attention mechanisms in neural networks with algorithmic approximation and hardware specialization. Our proposed accelerator achieves multiple orders of magnitude improvement in energy efficiency (performance/watt) as well as substantial speedup over the state-of-the-art conventional hardware.

show abstract

Section: Related Workmentioning

confidence: 99%

A^3: Accelerating Attention Mechanisms in Neural Networks with Approximation

Ham

Jung

Kim

et al. 2020

2020 IEEE International Symposium on High Performance Computer Architecture (HPCA)

108

View full text Add to dashboard Cite

show abstract

“…For example, Liu et al [48] adopted hierarchical recurrent CNNs to progressively refine the details of salient objects. In [49], a coarse saliency map was first generated by using the convolution-deconvolution networks. After that, it was refined by iteratively enhancing the results in various subregions.…”

Section: B Modelsmentioning

confidence: 99%

A Benchmark Dataset and Saliency-Guided Stacked Autoencoders for Video-Based Salient Object Detection

Xia

Chen

2018

IEEE Trans. on Image Process.

121

View full text Add to dashboard Cite

Abstract-Image-based salient object detection (SOD) has been extensively studied in the past decades. However, videobased SOD is much less explored since there lack large-scale video datasets within which salient objects are unambiguously defined and annotated. Toward this end, this paper proposes a video-based SOD dataset that consists of 200 videos (64 minutes). In constructing the dataset, we manually annotate all objects and regions over 7,650 uniformly sampled keyframes and collect the eye-tracking data of 23 subjects that freeview all videos. From the user data, we find salient objects in video can be defined as objects that consistently pop-out throughout the video, and objects with such attributes can be unambiguously annotated by combining manually annotated object/region masks with eye-tracking data of multiple subjects. To the best of our knowledge, it is currently the largest dataset for video-based salient object detection.Based on this dataset, this paper proposes an unsupervised baseline approach for video-based SOD by using saliencyguided stacked autoencoders. In the proposed approach, multiple spatiotemporal saliency cues are first extracted at pixel, superpixel and object levels. With these saliency cues, stacked autoencoders are unsupervisedly constructed which automatically infer a saliency score for each pixel by progressively encoding the high-dimensional saliency cues gathered from the pixel and its spatiotemporal neighbors. Experimental results show that the proposed unsupervised approach outperforms 30 state-of-the-art models on the proposed dataset, including 19 image-based & classic (unsupervised or non-deep learning), 6 image-based & deep learning, and 5 video-based & unsupervised. Moreover, benchmarking results show that the proposed dataset is very challenging and has the potential to boost the development of video-based SOD.

show abstract

“…Different from our proposed method, prediction of important objects is also studied by [8,9]. [8] uses recurrent attention and convolutional-deconvolutional network to tackle the salient object detection problem.…”

Section: Related Workmentioning

confidence: 99%

Context Aware Road-user Importance Estimation (iCARE)

Rahimpour

Martin

Tawari

et al. 2019

2019 IEEE Intelligent Vehicles Symposium (IV)

View full text Add to dashboard Cite

Road-users are a critical part of decision-making for both self-driving cars and driver assistance systems. Some road-users, however, are more important for decisionmaking than others because of their respective intentions, ego-vehicle's intention and their effects on each other. In this paper, we propose a novel architecture for road-user importance estimation which takes advantage of the local and global context of the scene. For local context, the model exploits the appearance of the road users (which captures orientation, intention, etc.) and their location relative to ego-vehicle. The global context in our model is defined based on the feature map of the convolutional layer of the module which predicts the future path of the ego-vehicle and contains rich global information of the scene (e.g., infrastructure, road lanes, etc.), as well as the ego-vehicle's intention information. Moreover, this paper introduces a new data set of real-world driving, concentrated around intersections and includes annotations of important road users. Systematic evaluations of our proposed method against several baselines show promising results.

show abstract

Recurrent Attentional Networks for Saliency Detection

Abstract: Abstract

Cited by 209 publications

References 43 publications

A^3: Accelerating Attention Mechanisms in Neural Networks with Approximation

A^3: Accelerating Attention Mechanisms in Neural Networks with Approximation

A Benchmark Dataset and Saliency-Guided Stacked Autoencoders for Video-Based Salient Object Detection

Context Aware Road-user Importance Estimation (iCARE)

Contact Info

Product

Resources

About