Extending One-Stage Detection with Open-World Proposals

Abstract

In many applications, such as autonomous driving, hand manipulation, or robot navigation, object detection methods must be able to detect objects unseen in the training set. Open World Detection (OWD) seeks to tackle this problem by generalizing detection performance to seen and unseen class categories. Recent works have seen success in the generation of class-agnostic proposals, which we call Open-World Proposals (OWP), but this comes at the cost of a big drop on the classification task when both tasks are considered in the detection model. These works have investigated two-stage Region Proposal Networks (RPN) by taking advantage of objectness scoring cues; however, for its simplicity, run-time, and decoupling of localization and classification, we investigate OWP through the lens of fully convolutional one-stage detection network, such as FCOS [35]. We show that our architectural and sampling optimizations on FCOS can increase OWP performance by as much as 6% in recall on novel classes, marking the first proposal-free one-stage detection network to achieve comparable performance to RPN based two-stage networks. Furthermore, we show that the inherent, decoupled architecture of FCOS has benefits to retaining classification performance. While two-stage methods worsen by 6% in recall on novel classes, we show that FCOS only drops 2% when jointly optimizing for OWP and classification.

Prediction of Social Dynamic Agents and Long-Tailed Learning Challenges: A Survey

“…While the long-tailed open world framework has been recieving increasing attention within detection (e.g., Liu et al, 2019;Joseph et al, 2021;Saito et al, 2022;Konan et al, 2022), segmentation (e.g., , and tracking (e.g., Liu et al, 2022;Dave, 2021), open-world prediction is still unexplored territory. One method that may be directly applicable to action prediction tasks is that proposed in Liu et al (2019), which uses contrastive learning and a memory framework to improve performance on both common and uncommon classes, and recognize examples that don't fall into any of the given classes.…”