From Open Set to Closed Set: Counting Objects by Spatial Divide-and-Conquer

Abstract

Visual counting, a task that aims to estimate the number of objects from an image/video, is an open-set problem by nature, i.e., the number of population can vary in [0, +∞) in theory. However, collected data and labeled instances are limited in reality, which means that only a small closed set is observed. Existing methods typically model this task in a regression manner, while they are prone to suffer from an unseen scene with counts out of the scope of the closed set. In fact, counting has an interesting and exclusive property-spatially decomposable. A dense region can always be divided until sub-region counts are within the previously observed closed set. We therefore introduce the idea of spatial divide-and-conquer (S-DC) that transforms open-set counting into a closed-set problem. This idea is implemented by a novel Supervised Spatial Divide-and-Conquer Network (SS-DCNet). Thus, SS-DCNet can only learn from a closed set but generalize well to open-set scenarios via S-DC. SS-DCNet is also efficient. To avoid repeatedly computing sub-region convolutional features, S-DC is executed on the feature map instead of on the input image. We provide theoretical analyses as well as a controlled experiment on toy data, demonstrating why closed-set modeling makes sense. Extensive experiments show that SS-DCNet achieves the state-of-the-art performance on three crowd counting datasets (ShanghaiTech, UCF_CC_50 and UCF-QNRF), a vehicle counting dataset (TRANCOS) and a plant counting dataset (MTC), with a 7.7% relative improvement on the UCF-QNRF, 33.1% on the TRANCOS, and 26.4% on the MTC. SS-DCNet also reports the state-of-the-art cross-domain performance on crowd counting datasets. Particularly in the task from UCF-QNRF to ShanghaiTech Part_A, SS-DCNet even beats most existing models trained directly on the target domain. Code and models have been made available at: https://tinyurl.com/SS-DCNet.

Efficient Crowd Counting via Structured Knowledge Transfer

“…Idrees et al [21] 315 508 MCNN [76] 277 426 Encoder-Decoder [3] 270 478 CMTL [57] 252 514 SwitchCNN [48] 228 445 Resnet-101 [15] 190 277 CL [22] 132 191 TEDnet [24] 113 188 CAN [36] 107 183 S-DCNet [65] 104.40 176.10 DSSINet [31] 99 Inter-RT can fully distill the knowledge of the teacher networks. What's more, the proposed SKT is easily implemented and the distilled crowd counting models can be directly deployed on various edge devices.…”

Select to better learn: Fast and accurate deep learning using data selection from nonlinear manifolds

“…In open-set identification, test data of a classification problem may come from unknown classes other than the classes employed during training, and the goal is to identify such samples belong to open-set and not the known labeled classes [64]. Interested readers are referred to [65,66,67,68,69] to the state-of-the-art approaches for solving open-set problem.…”

HAGN: Hierarchical Attention Guided Network for Crowd Counting

“…Xu et al [9] proposed Learning to Scale Module (L2SM) to solve the density variation issue in crowd counting. Xiong et al [10] proposed Spatial Divide-and Conquer Network (S-DCNet), which has an excellent performance on counting data sets for multiple cross-categories. Liu et al [11] proposed Deep Structured Scale Integration Network (DSSINet), which could respond to crowd density changes through structured and multi-level feature learning and corresponding loss function optimization.…”