Top-Down Feedback for Crowd Counting Convolutional Neural Network

Abstract

Counting people in dense crowds is a demanding task even for humans. This is primarily due to the large variability in appearance of people. Often people are only seen as a bunch of blobs. Occlusions, pose variations and background clutter further compound the difficulty. In this scenario, identifying a person requires larger spatial context and semantics of the scene. But the current state-of-the-art CNN regressors for crowd counting are feedforward and use only limited spatial context to detect people. They look for local crowd patterns to regress the crowd density map, resulting in false predictions. Hence, we propose top-down feedback to correct the initial prediction of the CNN. Our architecture consists of a bottom-up CNN along with a separate top-down CNN to generate feedback. The bottom-up network, which regresses the crowd density map, has two columns of CNN with different receptive fields. Features from various layers of the bottom-up CNN are fed to the top-down network. The feedback, thus generated, is applied on the lower layers of the bottom-up network in the form of multiplicative gating. This masking weighs activations of the bottom-up network at spatial as well as feature levels to correct the density prediction. We evaluate the performance of our model on all major crowd datasets and show the effectiveness of top-down feedback.

Counting Objects in Images using DeepLearning: Methods and Current Challenges

“…Babu Sam et al [74] and Sindagi et al [75] explored a different strategy for feature fusion which used concepts known as top-bottom and bottom-top feature fusion. Bottom-top feature fusion fuses together features from early layers in the network with later layers.…”

Weakly-Supervised Crowd Counting Learns from Sorting Rather Than Locations

“…Several approaches employ multiple receptive fields to evaluate the instances with various scales. To obtain the multiple receptive fields, Zhang et al [46], Deb et al [7] and Sam et al [29] employ multi-column networks; several approaches [2,13,19] utilize inception blocks. Besides changing the convolution kernels, a deep network can also obtain various receptive fields from its different layers.…”

LMCNet: A Lightweight Multi-class Counting Network with Ghost Attention Mechanism and Focal-L2 Loss

“…According to the parameter amount, we can divide these models into two types: heavyweight and lightweight. Generally, the former has better counting performance, while the latter has higher computing efficiency Early models are lightweight, such as, CrowdNet [8], multi-column CNN (MCNN) [1], and top-down feedback CNN (TDF-CNN) [9]. CrowdNet consists of two parallel branches, one deep and one shallow, which is an early form of multi-column architecture [8].…”

“…Table 1 shows the specific information of these two experimental test environments. On the RSMCOC dataset, LMCNet is compared with three lightweight networks: MCNN [1], TDF-CNN [9], and C-CNN [16], and seven heavyweight networks: CP-CNN [12], SaCNN [13], ACSCP [14], CSRNet [10], SFCN [11], DSNet [15], and DSACA [5].…”