Zero-Shot Object Detection

Bansal, Ankan; Sikka, Karan; Sharma, Gaurav; Chellappa, Rama; Divakaran, Ajay

doi:10.1007/978-3-030-01246-5_24

Cited by 253 publications

(350 citation statements)

References 71 publications

Supporting

Mentioning

348

Contrasting

Order By: Relevance

“…Our empirical results indicate that such an enforcement of the proper grounding of all phrases via caption-conditioned image representations ( Figure 2) does indeed lead to a better phrase localization performance ( Table 3, 4). Moreover, we also show that the proposed discriminative representation allows us to achieve results that are comparable to the state-of-the-art on the downstream image-caption matching task on both COCO and Flickr30k datasets ( Table 2).…”

Section: Introductionsupporting

confidence: 52%

Align2Ground: Weakly Supervised Phrase Grounding Guided by Image-Caption Alignment

Datta

Sikka

Roy

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

Self Cite

View full text Add to dashboard Cite

We address the problem of grounding free-form textual phrases by using weak supervision from image-caption pairs. We propose a novel end-to-end model that uses caption-toimage retrieval as a "downstream" task to guide the process of phrase localization. Our method, as a first step, infers the latent correspondences between regions-of-interest (RoIs) and phrases in the caption and creates a discriminative image representation using these matched RoIs. In the subsequent step, this learned representation is aligned with the caption. Our key contribution lies in building this "captionconditioned" image encoding which tightly couples both the tasks and allows the weak supervision to effectively guide visual grounding. We provide extensive empirical and qualitative analysis to investigate the different components of our proposed model and compare it with competitive baselines. For phrase localization, we report improvements of 4.9% and 1.3% (absolute) over prior state-of-the-art on the VisualGenome and Flickr30k Entities datasets. We also report results that are at par with the state-of-the-art on the downstream caption-to-image retrieval task on COCO and Flickr30k datasets.Recent works [20,21] have shown evidence that operating under such a paradigm helps boost performance for imagecaption matching. Generally, these models consist of two stages: (1) a local matching module that infers the latent region-phrase correspondences to generate local matching information, and (2) a global matching module that uses this information to perform image-caption matching. This setup allows phrase grounding to act as an intermediate and a prerequisite task for image-caption matching. It is important to note that the primary objective of such works has been on image-caption matching and not phrase grounding.An artifact of training under such a paradigm is the amplification of correlations between selective regions and phrases."Young girl holding a kitten" by Gennadiy Kolodkin is licensed under CC BY-NC-ND 2.0.

show abstract

Section: Introductionsupporting

confidence: 52%

Align2Ground: Weakly Supervised Phrase Grounding Guided by Image-Caption Alignment

Datta

Sikka

Roy

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Corner-Net achieved a 42.1% AP on MS COCO, outperforming all previous one stage detectors; however, the average inference time is 8 Boxes of various sizes and aspect ratios that serve as object candidates. 9 The idea of using keypoints for object detection appeared previously in DeNet [269]. about 4FPS on a Titan X GPU, significantly slower than SSD [175] and YOLO [227].…”

Section: Unified (One Stage) Frameworkmentioning

confidence: 99%

Deep Learning for Generic Object Detection: A Survey

et al. 2019

View full text Add to dashboard Cite

Object detection, one of the most fundamental and challenging problems in computer vision, seeks to locate object instances from a large number of predefined categories in natural images. Deep learning techniques have emerged as a powerful strategy for learning feature representations directly from data and have led to remarkable breakthroughs in the field of generic object detection. Given this period of rapid evolution, the goal of this paper is to provide a comprehensive survey of the recent achievements in this field brought about by deep learning techniques. More than 300 research contributions are included in this survey, covering many aspects of generic object detection: detection frameworks, object feature representation, object proposal generation, context modeling, training strategies, and evaluation metrics. We finish the survey by identifying promising directions for future research.

show abstract

“…Co-Localization: COCO Dataset Creation and Faster-RCNN Training COCO dataset has 80 classes in total. We take the same 17 unseen classes which is used in zero-shot object detection paper [Ref1] and keep remaining 63 classes for training. The training set is constructed using the images in COCO 2017 train set which contain at least one object from the seen classes.…”

Section: Appendix Learning To Find Common Objects Across Few Image Comentioning

confidence: 99%

“…The COCO test set, is built by combining the unused images of the train set and images in COCO validation set which contain at least one object from the unseen classes. Similar to [Ref1], to avoid training the network to classify unseen objects as background, we remove objects from unseen classes from the training images using their ground-truth segmentation masks.…”

Section: Appendix Learning To Find Common Objects Across Few Image Comentioning

confidence: 99%

Learning to Find Common Objects Across Few Image Collections

Shaban

Rahimi

Bansal

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

Given a collection of bags where each bag is a set of images, our goal is to select one image from each bag such that the selected images are from the same object class. We model the selection as an energy minimization problem with unary and pairwise potential functions. Inspired by recent few-shot learning algorithms, we propose an approach to learn the potential functions directly from the data. Furthermore, we propose a fast greedy inference algorithm for energy minimization. We evaluate our approach on few-shot common object recognition as well as object co-localization tasks. Our experiments show that learning the pairwise and unary terms greatly improves the performance of the model over several well-known methods for these tasks. The proposed greedy optimization algorithm achieves performance comparable to state-of-the-art structured inference algorithms while being ∼10 times faster.

show abstract

Zero-Shot Object Detection

Cited by 253 publications

References 71 publications

Align2Ground: Weakly Supervised Phrase Grounding Guided by Image-Caption Alignment

Align2Ground: Weakly Supervised Phrase Grounding Guided by Image-Caption Alignment

Deep Learning for Generic Object Detection: A Survey

Learning to Find Common Objects Across Few Image Collections

Contact Info

Product

Resources

About