Crowd Counting and Density Estimation by Trellis Encoder-Decoder Networks

Abstract

Crowd counting has recently attracted increasing interest in computer vision but remains a challenging problem. In this paper, we propose a trellis encoder-decoder network (TEDnet) for crowd counting, which focuses on generating high-quality density estimation maps. The major contributions are four-fold. First, we develop a new trellis architecture that incorporates multiple decoding paths to hierarchically aggregate features at different encoding stages, which improves the representative capability of convolutional features for large variations in objects. Second, we employ dense skip connections interleaved across paths to facilitate sufficient multi-scale feature fusions, which also helps TEDnet to absorb the supervision information. Third, we propose a new combinatorial loss to enforce similarities in local coherence and spatial correlation between maps. By distributedly imposing this combinatorial loss on intermediate outputs, TEDnet can improve the back-propagation process and alleviate the gradient vanishing problem. Finally, on four widely-used benchmarks, our TEDnet achieves the best overall performance in terms of both density map quality and counting accuracy, with an improvement up to 14% in MAE metric. These results validate the effectiveness of TEDnet for crowd counting.

Multi-Level Bottom-Top and Top-Bottom Feature Fusion for Crowd Counting

“…Most recently, several methods have focused on incorporating additional cues such as segmentation and semantic priors [61,75], attention [31,54,58], perspective [50], context information respectively [33], multiple-views [70] and multi-scale features [20] into the network. Wang et al [63] introduced a new synthetic dataset and proposed a SSIM based CycleGAN [78] to adapt the synthetic datasets to real world dataset.…”

Pushing the Frontiers of Unconstrained Crowd Counting: New Dataset and Benchmark Method

“…Recent approaches like [22,47,48,51,62] have aimed at incorporating various forms of related information like attention [22], semantic priors [51], segmentation [62], inverse attention [48], and hierarchical attention [47] respectively into the network. Other techniques such as [12,23,40,60] leverage features from different layers of the network using different techniques like trellis style encoder decoder [12], explicitly considering perspective [40], context information [23], and multiple views [60]. Crowd Datasets.…”

“…Method MAE MSE Idrees et al [10] 315.0 508.0 Zhang et al [59] 277.0 426.0 CMTL et al [43] 252.0 514.0 Switching-CNN [38] 228.0 445.0 Idrees et al [11] 132.0 191.0 Jian et al [12] 113.0 188.0 CG-DRCN (proposed) 112.2 176.3 column network (MCNN) [61], cascaded multi-task learning for crowd counting (CMTL) [43], Switching-CNN [38], CSR-Net [20] and SANet [4] 2 . Furthermore, we also evaluate the proposed method (CG-DRCN) and demonstrate its effectiveness over the other methods.…”

“…ShanghaiTech: The proposed network is trained on the train splits using the same strategy as discussed in Section 3.4. Table 5 shows the results of the proposed method on ShanghaiTech as compared with several recent approaches ( [38], [44], [2], [41], [24], [20], [33] , [4], [39] and [12]). It can be observed that the proposed method outperforms all existing methods on Part A of the dataset, while achieving comparable performance on Part B.…”

“…Results on the UCF-QNRF [11] dataset as compared with recent methods ( [10], [61], [43]) are shown in Table 6. The proposed method is compared against different approaches: [10], [61], [43], [38], [11] and [12]. It can be observed that the proposed method outperforms other methods by a considerable margin.…”