A Hierarchical Generative Model of Recurrent Object-Based Attention in the Visual Cortex

Reichert, David P.; Seriès, Peggy; Storkey, Amos

doi:10.1007/978-3-642-21735-7_3

Cited by 7 publications

(5 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Shapes. We use the simple Shapes dataset [24] to examine the basic properties of our system. It consists of 60,000 (train) + 10,000 (test) binary images of size 20x20.…”

Section: Experiments and Resultsmentioning

confidence: 99%

Tagger: Deep Unsupervised Perceptual Grouping

Greff¹,

Rasmus²,

Berglund³

et al. 2016

Preprint

View full text Add to dashboard Cite

We present a framework for efficient perceptual inference that explicitly reasons about the segmentation of its inputs and features. Rather than being trained for any specific segmentation, our framework learns the grouping process in an unsupervised manner or alongside any supervised task. We enable a neural network to group the representations of different objects in an iterative manner through a differentiable mechanism. We achieve very fast convergence by allowing the system to amortize the joint iterative inference of the groupings and their representations. In contrast to many other recently proposed methods for addressing multi-object scenes, our system does not assume the inputs to be images and can therefore directly handle other modalities. We evaluate our method on multi-digit classification of very cluttered images that require texture segmentation. Remarkably our method achieves improved classification performance over convolutional networks despite being fully connected, by making use of the grouping mechanism. Furthermore, we observe that our system greatly improves upon the semi-supervised result of a baseline Ladder network on our dataset. These results are evidence that grouping is a powerful tool that can help to improve sample efficiency.

show abstract

“…Shapes. We use the simple Shapes dataset [24] to examine the basic properties of our system. It consists of 60,000 (train) + 10,000 (test) binary images of size 20x20.…”

Section: Experiments and Resultsmentioning

confidence: 99%

Tagger: Deep Unsupervised Perceptual Grouping

Greff¹,

Rasmus²,

Berglund³

et al. 2016

Preprint

View full text Add to dashboard Cite

show abstract

“…Image caption is a technique that attempts to "translate" images into text using machine learning methods [2,3]. The research history of the image caption problem is only ten years old, but the image caption model has experienced many major changes.…”

Section: Image Captionmentioning

confidence: 99%

An Image Captioning Algorithm Based on Combination Attention Mechanism

et al. 2022

View full text Add to dashboard Cite

With the maturity of computer vision and natural language processing technology, we are becoming more ambitious in image captioning. In particular, we are more ambitious in generating longer, richer, and more accurate sentences as image descriptions. Most existing image caption models use an encoder—decoder structure, and most of the best-performing models incorporate attention mechanisms in the encoder—decoder structure. However, existing image captioning methods focus only on visual attention mechanism and not on keywords attention mechanism, thus leading to model-generated sentences that are not rich and accurate enough, and errors in visual feature extraction can directly lead to generated caption sentences that are incorrect. To fill this gap, we propose a combination attention module. This module comprises a visual attention module and a keyword attention module. The visual attention module helps in performing fast extractions of key local features, and the keyword attention module focuses on keywords that may appear in generated sentences. The results generated by the two modules can be corrected for each other. We embed the combination attention module into the framework of the Transformer, thus constructing a new image caption model CAT (Combination Attention Transformer) to generate more accurate and rich image caption sentences. Extensive experiments on the MSCOCO dataset demonstrate the effectiveness and superiority of our method over many state-of-the-art methods.

show abstract

“…Larochelle and Hinton [8] proposed using Boltzmann Machines that choose where to look next to find locations of the most informative intra-class objects, even if they are far away in the image. Reichert et al proposed a hierarchical model to show that certain aspects of attention can be modeled by Deep Boltzmann Machines [14]. Attention-based models were also proposed for generative models.…”

Section: Self-attention Mechanismmentioning

confidence: 99%

Self-Attention Capsule Networks for Object Classification

Hoogi,

Wilcox,

Gupta

et al. 2019

Preprint

View full text Add to dashboard Cite

We propose a novel architecture for object classification, called Self-Attention Capsule Networks (SACN). SACN is the first model that incorporates the Self-Attention mechanism as an integral layer within the Capsule Network (Cap-sNet). While the Self-Attention mechanism supplies a longrange dependencies, results in selecting the more dominant image regions to focus on, the CapsNet analyzes the relevant features and their spatial correlations inside these regions only. The features are extracted in the convolutional layer. Then, the Self-Attention layer learns to suppress irrelevant regions based on features analysis and highlights salient features useful for a specific task. The attention map is then fed into the CapsNet primary layer that is followed by a classification layer. The proposed SACN model was designed to solve two main limitations of the baseline Cap-sNet -analysis of complex data and significant computational load. In this work, we use a shallow CapsNet architecture and compensates for the absence of a deeper network by using the Self-Attention module to significantly improve the results. The proposed Self-Attention Cap-sNet architecture was extensively evaluated on six different datasets, mainly on three different medical sets, in addition to the natural MNIST, SVHN and CIFAR10. The model was able to classify images and their patches with diverse and complex backgrounds better than the baseline Cap-sNet. As a result, the proposed Self-Attention CapsNet significantly improved classification performance within and across different datasets and outperformed the baseline Cap-sNet, ResNet-18 and DenseNet-40 not only in classification accuracy but also in robustness.

show abstract

A Hierarchical Generative Model of Recurrent Object-Based Attention in the Visual Cortex

Cited by 7 publications

References 9 publications

Tagger: Deep Unsupervised Perceptual Grouping

Tagger: Deep Unsupervised Perceptual Grouping

An Image Captioning Algorithm Based on Combination Attention Mechanism

Self-Attention Capsule Networks for Object Classification

Contact Info

Product

Resources

About