Weakly Supervised Localization Using Deep Feature Maps

Abstract

Object localization is an important computer vision problem with a variety of applications. The lack of large scale object-level annotations and the relative abundance of image-level labels makes a compelling case for weak supervision in the object localization task. Deep Convolutional Neural Networks are a class of state-of-the-art methods for the related problem of object recognition. In this paper, we describe a novel object localization algorithm which uses classification networks trained on only image labels. This weakly supervised method leverages local spatial and semantic patterns captured in the convolutional layers of classification networks. We propose an efficient beam search based approach to detect and localize multiple objects in images. The proposed method significantly outperforms the state-of-the-art in standard object localization data-sets with a 8 point increase in mAP scores.

WILDCAT: Weakly Supervised Learning of Deep ConvNets for Image Classification, Pointwise Localization and Segmentation

“…In spite of its simple and multipurpose architecture, our model outperforms by a large margin the complex cascaded architecture of ProNet [58]. It also outperforms the recent weakly supervised model [5] by 3.2 pt (resp. 4.2 pt) on VOC 2012 (resp.…”

“…Concerning the WSL localization task, [5] uses label co-occurrence information and a coarse-to-fine strategy based on deep feature maps to predict object locations. ProNet [58] uses a cascade of two networks: the first generates bounding boxes and the second classifies them.…”

“…For weakly supervised pointwise object detection, we extract the region (i.e. neuron in the feature map) with maximum score for each class and use it for point-wise localization, as it is done in [44,5]. For weakly supervised semantic segmentation we compute the final segmentation mask either by taking the class with maximum score at each spatial position independently or by applying a CRF for spatial prediction as is common practice [8,48].…”

Zigzag Learning for Weakly Supervised Object Detection

“…2. The first module estimates image difficulty automatically via a backbone network [18] trained with only image-level labels. The second module progressively adds samples to network training in an ascending order based on image difficulty.…”

“…Once we obtain the image difficulty, the remaining task is to mine object instances from the images. A natural way is to directly choose the top scored region as the target object, which is used for localization evaluation in [18]. However, since the whole network is trained with classification loss, which makes high scored regions tend to focus on object parts rather than the whole objects.…”