Pedestrian Detection at Day/Night Time with Visible and FIR Cameras: A Comparison

Abstract

Despite all the significant advances in pedestrian detection brought by computer vision for driving assistance, it is still a challenging problem. One reason is the extremely varying lighting conditions under which such a detector should operate, namely day and nighttime. Recent research has shown that the combination of visible and non-visible imaging modalities may increase detection accuracy, where the infrared spectrum plays a critical role. The goal of this paper is to assess the accuracy gain of different pedestrian models (holistic, part-based, patch-based) when training with images in the far infrared spectrum. Specifically, we want to compare detection accuracy on test images recorded at day and nighttime if trained (and tested) using (a) plain color images; (b) just infrared images; and (c) both of them. In order to obtain results for the last item, we propose an early fusion approach to combine features from both modalities. We base the evaluation on a new dataset that we have built for this purpose as well as on the publicly available KAIST multispectral dataset.

Unsupervised Domain Adaptation for Multispectral Pedestrian Detection

“…In order to conduct our experiments on multimodal domain adaptation, we utilize the KAIST [11] and CVC-14 [8] multispectral pedestrian benchmarks as the source and target domain datasets respectively.…”

“…For a fair comparison with the current state-of-the-art multispectral pedestrian detector designed by Cao et al [4], we downscale the images to the resolution of 320 × 240 through bilinear interpolation during training and testing phase on target domain. The annotations of CVC-14 test set under the reasonable setting (pedestrians larger than 50 pixels [8]) are used to evaluate detection performance.…”

“…Detection results of the current state-of-the-art multispectral pedestrian detector well-trained using visible and thermal image pairs from KAIST [11] dataset following the method presented by Cao et al [4]. (a) Results on the KAIST dataset; (b) Results on the CVC-14 [8] dataset. Please note that the visible images in KAIST dataset are transferred from RGB to gray level in order to decrease domain differences between these two datasets.…”

“…For example, as shown in Fig. 1, the current state-of-the-art multispectral pedestrian detector [4] well-trained in the KAIST [11] dataset can't generate reliable detection results on the images from CVC-14 [8] dataset. This is because multispectral pedestrian detectors well-trained on major open dataset tend to overfit the training data, which is usually biased to specific environments [36].…”

Box-level segmentation supervised deep neural networks for accurate and real-time multispectral pedestrian detection

“…visible and infrared), which can supply complementary information about the targets of interest, are considering to build more robust pedestrian detectors under various illumination conditions. In the past few years, multispectral pedestrian detection solutions are developed by many research works to achieve more accurate and Email addresses: caoyp@zju.edu.cn (Yanpeng Cao), 11725001@zju.edu.cn (Dayan Guan), 3160105381@zju.edu.cn (Yulun Wu), yangjx@zju.edu.cn (Jiangxin Yang), sdcaoyl@zju.edu.cn (Yanlong Cao), michael.yang@utwente.nl (Michael Ying Yang) stable pedestrian detection results for around-the-clock application [30,28,53,41,20,14].…”