Crowd counting using a self‐attention multi‐scale cascaded network

Abstract

Recent developments of crowd analysis and behaviour prediction have attracted much attention. Crowd counting, as the essential and challenging task in crowd analysis, is riddled with many issues, such as large scale variations, serious occlusion, and so on. In this study, a self-attention-based multi-scale cascaded network called SAMC-Net to estimate density map for crowd counting, especially for high congested scene, is proposed. The proposed SAMC-Net consists of two components: a classification sub-network for density estimation and an end-to-end multi-scale convolution neural network for crowd counting. In order to reduce the negative effect of multi-scale issue on crowd counting task, the main network is designed as a multi-scale structure similar to U-Net. In order to enhance the crowd feature representation, this study proposes a selfattention-based crowd feature extraction way and uses it in the proposed SAMC-Net. Extensive experiments demonstrate the feasibility, effectiveness and robustness of the proposed SAMC-Net.

Object counting method based on dual attention network

“…We compare the proposed Count‐DANet with several crowd counting methods, and the detailed results are shown in Table 2. It can be observed from Table 2, Count‐DANet has an 8.7% improvement of MAE with Li et al [16] on the Mall dataset. The estimated density maps by Count‐DANet and the ground truth of several samples on Mall datasets are shown in Fig.…”

“…Shanghaitech_A contains 482 images, where 300 images are used for training, and the remaining images are for testing. We compare our method with several typical crowd counting methods on Shanghaitech_A dataset: Marsden et al [29], Zhang et al [12], Sindagi and Patel [18], Sam et al [13], Li et al [16], SaCNN [14], Li et al [19], and Zhang et al [15], the results of which are shown in Table 3. As is shown in Table 3, the proposed method gets a 0.3% improvement of MAE with Li et al [19] and achieves the lowest 85.9 of MAE among them, except the method proposed in [15].…”

“…Similarly, Zhang et al [15] also developed a cascaded network based on feature fusion for object counting. Based on the cascaded architecture in [15], Li et al [16] designed a multi‐scale structure similar to U‐Net to cope with the scale‐variant issue. Simultaneously, though some researchers attempted to add the context information from the images to the CNN architecture to boosting the counting accuracy, these methods cannot solve the multi‐scale problem.…”

MSR‐FAN: Multi‐scale residual feature‐aware network for crowd counting

“…The mall dataset is similar to UCSD, which contains single scene and a small number of pedestrians. On this dataset, this work gets the best result compared with [7, 8, 42, 43, 44]. It shows that MSR‐FAN has good generalization ability for general datasets.…”

SCFFNet: Spatial Context Feature Fusion Network for Understanding the Highly Congested Scenes

“…Even compared with the recently proposed MGANet [57], the MAE and MSE indicators of SCFFNet also decreased by 31.8% and 27.1%. is means that in extremely congested crowd scenes, the multiscale context feature fusion module is applied to model multiscale contextual features, an attention mechanism is introduced to process the extracted features, and attention [49] 467.0 498.5 MCNN [12] 377.6 509.1 FCN [53] 338.6 424.5 Switch-CNN [47] 318.1 439.2 CP-CNN [23] 295.8 320.9 DR-ResNet [52] 307.4 421.6 CSRNet [5] 266.1 397.5 SANet [22] 258.4 334.9 SAMC-net [54] 250.8 375.1 SCAR [43] 259.0 374.0 Multiscale-CNN [28] 264.9 382.1 LGCCNN [55] 336.5 510.2 SCLNet [24] 258.9 326.2 Khan and Basalamah [58] 229.4 325.6 ED-CNN [56] 271. information is obtained from the channel and spatial dimensions, which effectively solves the problems of complex background, scale variation, and occlusion in congested scenes. Figure 10 shows the visualization of some samples of the UCF_CC_50 dataset, from which we can see that SCFFNet performs well in crowd scenarios with different levels of congestion.…”