Synthesizing the Unseen for Zero-Shot Object Detection

Hayat, Nasir; Hayat, Munawar; Rahman, Shafin; Khan, Salman; Zamir, Syed Waqas; Khan, Fahad Shahbaz

doi:10.1007/978-3-030-69535-4_10

Cited by 31 publications

(48 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Zero-shot object detection. ZSD receives great research interest in recent years [4,7,16,22,31,32,[41][42][43]. Some researches focus on embedding function-based methods [4,7,22,31,32,42].…”

Section: Related Workmentioning

confidence: 99%

“…Once the model is trained, it can only recognize objects that appear in the training data, whereas other objects appearing in the test images but unseen during training would confuse the model dramatically, leading to avoidless faults in detection results. To address this problem, the task of zero-shot object detection (ZSD) [4,16,32,43] was raised in recent years. The goal is to enable the detection models to predict unseen objects which are without any available samples during training.…”

Section: Introductionmentioning

confidence: 99%

“…With the learned mapping functions, unseen object categories can be predicted by mapping their visual features into the semantic space and then performing the nearest neighbor search in the semantic space. However, due to that the mapping functions are learned all upon the seen categories provided by the training data, the models would get significantly biased towards the seen categories when dealing with the visual features in testing [16]. Recently, generative model-based methods [16,43] are presented as an alternative solution.…”

Section: Introductionmentioning

confidence: 99%

“…However, due to that the mapping functions are learned all upon the seen categories provided by the training data, the models would get significantly biased towards the seen categories when dealing with the visual features in testing [16]. Recently, generative model-based methods [16,43] are presented as an alternative solution. Usually, these methods utilize generative models to synthesize visual features from the provided semantic embeddings [2,29] corresponding to each object category.…”

Section: Introductionmentioning

confidence: 99%

“…Although some existing approaches have recognized the importance of intra-class diversity [16,41], without jointly considering the inter-class separability, these methods would either impose insufficient diversity to the synthesized visual features, leading to miss-classify the real unseen objects as image backgrounds (see Fig To overcome the feature synthesizing problems toward real-world detection scenarios, we build a novel zero-shot object detection framework as shown in Fig 2 . Specifically, we design two components for learning robust region features. To enable the model to synthesize diverse visual features, we propose an Intra-class Semantic Diverging (In-traSD) component which can diverge the semantic vector of a single class into a set of visual features.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Robust Region Feature Synthesizer for Zero-Shot Object Detection

Huang¹,

Han²,

Cheng³

et al. 2022

Preprint

View full text Add to dashboard Cite

Zero-shot object detection aims at incorporating class semantic vectors to realize the detection of (both seen and) unseen classes given an unconstrained test image. In this study, we reveal the core challenges in this research area: how to synthesize robust region features (for unseen objects) that are as intra-class diverse and inter-class separable as the real samples, so that strong unseen object detectors can be trained upon them. To address these challenges, we build a novel zero-shot object detection framework that contains an Intra-class Semantic Diverging component and an Inter-class Structure Preserving component. The former is used to realize the one-to-more mapping to obtain diverse visual features from each class semantic vector, preventing miss-classifying the real unseen objects as image backgrounds. While the latter is used to avoid the synthesized features too scattered to mix up the inter-class and foreground-background relationship. To demonstrate the effectiveness of the proposed approach, comprehensive experiments on PASCAL VOC, COCO, and DIOR datasets are conducted. Notably, our approach achieves the new state-of-the-art performance on PASCAL VOC and COCO and it is the first study to carry out zero-shot object detection in remote sensing imagery.

show abstract