Attentive Contexts for Object Detection

Abstract

Modern deep neural network based object detection methods typically classify candidate proposals using their interior features. However, global and local surrounding contexts that are believed to be valuable for object detection are not fully exploited by existing methods yet. In this work, we take a step towards understanding what is a robust practice to extract and utilize contextual information to facilitate object detection in practice. Specifically, we consider the following two questions: "how to identify useful global contextual information for detecting a certain object?" and "how to exploit local context surrounding a proposal for better inferring its contents?". We provide preliminary answers to these questions through developing a novel Attention to Context Convolution Neural Network (AC-CNN) based object detection model. AC-CNN effectively incorporates global and local contextual information into the region-based CNN (e.g. Fast RCNN) detection model and provides better object detection performance. It consists of one attentionbased global contextualized (AGC) sub-network and one multi-scale local contextualized (MLC) sub-network. To capture global context, the AGC sub-network recurrently generates an attention map for an input image to highlight useful global contextual locations, through multiple stacked Long Short-Term Memory (LSTM) layers. For capturing surrounding local context, the MLC sub-network exploits both the inside and outside contextual information of each specific proposal at multiple scales. The global and local context are then fused together for making the final decision for detection. Extensive experiments on PASCAL VOC 2007 and VOC 2012 well demonstrate the superiority of the proposed AC-CNN over well-established baselines. In particular, AC-CNN outperforms the popular Fast-RCNN by 2.0% and 2.2% on VOC 2007 and VOC 2012 in terms of mAP, respectively.

Improving object detection with region similarity learning

“…Under 07 + 12 train val, VGG16 has achieved up to 2.1% mAP improvement. Moreover, compared to other typical region-based detectors, such as AC-CNN [9], Yuting [15], MR-CNN [1], the proposed approach yields competitive performance as well. OHEM [12] is the state-of-the-art object detection approach, which has introduced online bootstrapping to the design of network structure based on the FastRCNN framework.…”

“…In FastRCNN [3], many hyperparameters are introduced for efficient learning, e.g., the thresholds to define foreground RoIs (regions of interest) and background RoIs, the sampling ratio of positive (foreground RoIs) and negative samples (background RoIs) in mini-batch stochastic gradient descent (SGD) optimization, etc. Li et al [9] proposed to use LSTM cells [10] to capture local context information of proposal boxes and global context information of entire images to strengthen the discrimination ability of RoI's feature. In [11], Li et al took advantages of multiple subnetworks' output to deal with large scale changes.…”

Graphonomy: Universal Human Parsing via Graph Transfer Learning

“…Liang et al [29] explicitly constructed a semantic neuron graph network by incorporating the semantic concept hierarchy. On the other hand, there are some sequential reasoning models for relationships [4,21]. In these works, a fixed graph is usually considered, while our Graphonomy makes further efforts from external knowledge embedding to graph representation transfer.…”

ASSD: Attentive single shot multibox detector

“…The common characteristic of these methods is that they only focus on single instance problems. For multi-object recognition, AC-CNN [15], LPA [13] and RelationNet [11] have been proposed to discover a global contextual guidance. AC-CNN examines the global context through the stacked Long Short-Term Memory (LSTM) units.…”