Multi-modal target detection for autonomous wide area search and surveillance

Abstract

Generalised wide are search and surveillance is a common-place tasking for multi-sensory equipped autonomous systems. Here we present on a key supporting topic to this task -the automatic interpretation, fusion and detected target reporting from multi-modal sensor information received from multiple autonomous platforms deployed for wide-area environment search. We detail the realization of a real-time methodology for the automated detection of people and vehicles using combined visible-band (EO), thermal-band (IR) and radar sensing from a deployed network of multiple autonomous platforms (ground and aerial). This facilities real-time target detection, reported with varying levels of confidence, using information from both multiple sensors and multiple sensor platforms to provide environment-wide situational awareness. A range of automatic classification approaches are proposed, driven by underlying machine learning techniques, that facilitate the automatic detection of either target type with cross-modal target confirmation. Extended results are presented that show both the detection of people and vehicles under varying conditions in both isolated rural and cluttered urban environments with minimal false positive detection. Performance evaluation is presented at an episodic level with individual classifiers optimized for maximal each object of interest (vehicle/person) detection over a given search path/pattern of the environment, across all sensors and modalities, rather than on a per sensor sample basis. Episodic target detection, evaluated over a number of wide-area environment search and reporting tasks, generally exceeds 90%+ for the targets considered here.

Posture estimation for improved photogrammetric localization of pedestrians in monocular infrared imagery

“…Furthermore, Figure 6 shows an extended comparison of the resulting reported target position tracks as a planar view of the {Y /Z} tracked position (for the scenario of Figure 5). These are reported relative to the camera position as per [1,7]. This is illustrated using both standard photogrammetric position localization as per prior work [1] (Figure 6 A) and posture estimation via regression for localization correction as per Section 2.2.2 (Figure 6 B).…”

“…Prior work explicitly dealing with thermal-band (IR) imagery within an automated surveillance context is presently largely focused upon pedestrian detection [3,5,7,22,23] and tracking [24,25]. More recently extended studies have investigated the fundamentals of both background scene modeling [26,27] and feature point descriptors [28] that commonly form the basis of many such techniques [3,5].…”

“…A key advantage of such thermal-band infrared (IR) imagery for pedestrian localization is both androbust detection of human shape signatures within the scene [5][6][7] and robust localization of their scene bounds in pixel-space (e.g. Figure 1).…”

“…As such, the principles of photogrammetry can be used to recover 3D pedestrian position within the scene based on a known camera projection model and an assumption that variance in human height is in fact quite small (statistically supported by [8,9]). In [1] we experimentally investigated the accuracy of classical photogrammetry, within the context of current target detection and classification techniques [5][6][7], as a means of recovering the true 3D position of pedestrian targets within the scene. A real-time approach for the detection, classification and localization of pedestrian targets via thermal-band (infrared) sensing was presented with supporting statistical evidence underpinning the key photogrammetric assumptions.…”

A comparison of features for regression-based driver head pose estimation under varying illumination conditions

“…Face detection is initially performed using an ensemble of trained Haar-cascade classifiers [22] (a multi-orientation cascade of cascades [23,24,25]). Within the driver context, this ensemble of cascades is biased towards frontal profile detection (higher a priori probability) followed by subsequent side profile (left/right) detection (as illustrated in Fig.…”