Context-aware Attentional Pooling (CAP) for Fine-grained Visual Classification

Abstract

Deep convolutional neural networks (CNNs) have shown a strong ability in mining discriminative object pose and parts information for image recognition. For fine-grained recognition, context-aware rich feature representation of object/scene plays a key role since it exhibits a significant variance in the same subcategory and subtle variance among different subcategories. Finding the subtle variance that fully characterizes the object/scene is not straightforward. To address this, we propose a novel context-aware attentional pooling (CAP) that effectively captures subtle changes via sub-pixel gradients, and learns to attend informative integral regions and their importance in discriminating different subcategories without requiring the bounding-box and/or distinguishable part annotations. We also introduce a novel feature encoding by considering the intrinsic consistency between the informativeness of the integral regions and their spatial structures to capture the semantic correlation among them. Our approach is simple yet extremely effective and can be easily applied on top of a standard classification backbone network. We evaluate our approach using six state-of-the-art (SotA) backbone networks and eight benchmark datasets. Our method significantly outperforms the SotA approaches on six datasets and is very competitive with the remaining two.

SR-GNN: Spatial Relation-Aware Graph Neural Network for Fine-Grained Image Categorization

“…Performance-wise, CAP [7] is currently top of the leaderboard for many FGVC datasets. It uses attention to accumulate features from integral regions in a context-aware fashion.…”

“…The attention mechanism is proliferated to identify salient regions and/or subtle discriminatory features to attain superior performance [7]- [9], [28], [29]. A trilinear attention sampling in [29] learns features from hundreds of part proposals and then applies knowledge distillation to integrate them.…”

“…We use the lightweight Xception [40] backbone for extracting CNN features like CAP [7] and upsample it for region proposals as in [8] by exploring cells and blocks in the HOG computation. The region proposal generates R possible regions of different aspect ratios and areas.…”

“…Moreover, partbased methods limit both scalability and practicality of realworld FGVC applications. Thus, many recent methods have used image-level labels to guide their models in identifying the key object parts to discriminate the sub-categories by exploring attention mechanisms in the image space or feature space [7]- [10] to automatically mine discriminative features.…”

Self-Attentive Pooling for Efficient Deep Learning

“…RNNPool [35] uses recurrent neural networks (RNNs) to aggregate features of large 1D receptive fields across different dimensions. In order to extract context-aware rich features for fine-grained visual recognition, another recent work [3] called CAP proposed a novel attentive pooling that correlates between different regions of the convolutional feature map to help discriminate between subcategories and improve accuracy. In particular, CAP is applied late in the network (after all convolutional layers) and is not intended to reduce the model's memory footprint, in contrast to our work which applies pooling to down-sample the large activation maps early in the network.…”