Visible-Thermal Pedestrian Detection via Unsupervised Transfer Learning

Abstract

Recently, pedestrian detection using visible-thermal pairs plays a key role in around-the-clock applications, such as public surveillance and autonomous driving. However, the performance of a well-trained pedestrian detector may drop significantly when it is applied to a new scenario. Normally, to achieve a good performance on the new scenario, manual annotation of the dataset is necessary, while it is costly and unscalable. In this work, an unsupervised transfer learning framework is proposed for visible-thermal pedestrian detection tasks. Given well-trained detectors from a source dataset, the proposed framework utilizes an iterative process to generate and fuse training labels automatically, with the help of two auxiliary single-modality detectors (visible and thermal). To achieve label fusion, the knowledge of daytime and nighttime is adopted to assign priorities to labels according to their illumination, which improves the quality of generated training labels. After each iteration, the existing detectors are updated using new training labels. Experimental results demonstrate that the proposed method obtains state-of-the-art performance without any manual training labels on the target dataset.

An Unsupervised Transfer Learning Framework for Visible-Thermal Pedestrian Detection

“…All the detectors used in this paper are implemented based on Faster R-CNN [25]. We utilize the Feature-Map Fusion method [17] concatenating feature maps of the thermal and visible branches in Faster R-CNN with a backbone of VGG16 [60] to form a multispectral pedestrian detector, which follows the successful design of Halfway Fusion [10]. After the concatenation operation, a convolutional layer called Network-in-Network (NIN) with 1 × 1 kernel is attached to reduce the dimension as well as to fit into the standard Faster R-CNN architecture.…”

“…As far as we know, only limited initial work [15,16] exists on unsupervised transfer learning in the area of visible-thermal pedestrian detection. Inspired by their idea of using pseudo training labels, we proposed a basic unsupervised transfer learning framework in our prior paper [17] to adapt pedestrian detectors to new scenarios, where the pseudo labels are generated to update the parameters of a detector.…”