Geometric and Physical Constraints for Drone-Based Head Plane Crowd Density Estimation

Abstract

State-of-the-art methods for counting people in crowded scenes rely on deep networks to estimate crowd density in the image plane. While useful for this purpose, this imageplane density has no immediate physical meaning because it is subject to perspective distortion. This is a concern in sequences acquired by drones because the viewpoint changes often. This distortion is usually handled implicitly by either learning scaleinvariant features or estimating density in patches of different sizes, neither of which accounts for the fact that scale changes must be consistent over the whole scene.In this paper, we explicitly model the scale changes and reason in terms of people per square-meter. We show that feeding the perspective model to the network allows us to enforce global scale consistency and that this model can be obtained on the fly from the drone sensors. In addition, it also enables us to enforce physically-inspired temporal consistency constraints that do not have to be learned. This yields an algorithm that outperforms state-of-the-art methods in inferring crowd density from a moving drone camera especially when perspective effects are strong.

A Geometric Approach to Obtain a Bird's Eye View From an Image

“…We have presented a complete pipeline for removing perspective distortion from an image, and obtaining the bird's eye view from a monocular image automatically. Our method can be used as plug and play to help other networks which suffer from multiple-scales due to perspective distortion such as vehicle tracking [28], crowd counting [24,25] or penguin counting [4] etc. Our method is fast, robust and can be used in real-time on videos to generate a bird's eye view of the scene.…”

“…It can be used as a preprocessing step for many other computer vision tasks like object detection [19,29] and tracking [10], and has applications in video surveillance and traffic control. For example, in crowd counting, where perspective distortion affects the crowd density in the image, the crowd density can instead be predicted in the world [24]. * The author is now at Latent Logic, Oxford Figure 1: An overview of our method for obtaining the bird's eye view of a scene from a single perspective image.…”

Misplaced Trust: Measuring the Interference of Machine Learning in Human Decision-Making

“…Crowd Comparison Application: We used an implementation [31] of the multi-column convolutional neural network (MCNN) architecture trained using part A of the ShanghaiTechDataset [40]. The subtasks consisted of 4 sets of crowd images that looked distinct and were sourced from various datasets: (1) UCF-QNRF, consisting of images of extremely large crowds scraped from the internet [14], (2) Venice, consisting of images from cameras around a plaza in Venice, Italy [22,23], (3) Shanghai A-test, a subset of the ShanghaiTech-Dataset consisting of images of crowds scraped from the internet (which differs significantly from UCF-QNRF in crowd density), and (4) Shanghai B, a subset of the ShanghaiTechDataset consisting of images from cameras around Shanghai streets. Although MCNN outputs a predicted count of people in an image, our subtasks are binary classification problems asking which of two images has more people.…”

Convolutional-Neural Network-Based Image Crowd Counting: Review, Categorization, Analysis, and Performance Evaluation

“…The negative sampling and data-driven approach is missing in all the listed methods. Liu et al [82] proposed a geometric-aware crowd-density-estimation technique. An explicit model was proposed to deal with perspective distortion effects.…”