Pedestrian recognition through different cross-modality deep learning methods

Abstract

Abstract-A wide variety of approaches have been proposed for pedestrian detection in the last decay and it still remains an open challenge due to its outstanding importance in the field of automotive. In recent years, deep learning classification methods, in particular convolutional neural networks, combined with multi-modality images applied on different fusion schemes have achieved great performances in computer vision tasks. For the pedestrian recognition task, the late-fusion scheme outperforms the early and intermediate integration of modalities. In this paper, we focus on improving and optimizing the late-fusion scheme for pedestrian classification on the Daimler stereo vision data set. We propose different training methods based on Cross-Modality deep learning of Convolutional Neural Networks (CNNs): (1) a correlated model, (2) an incremental model and, (3) a particular cross-modality model, where each CNN is trained on one modality, but tested on a different one. The experiments show that the incremental cross-modality deep learning of CNNs achieves the best performances. It improves the classification performances not only for each modality classifier, but also for the multi-modality late-fusion scheme. The particular cross-modality model is a promising idea for automated annotation of modality images with a classifier trained on a different modality and/or for cross-dataset training.

Fusion of Thermal and Depth Image to Improve Human Segmentation for a Mobile Robot

“…Fusion is a process of fusing the relevant modalities from several sources into a single image, where the results will be complete and more informative than any of the single image ( [18], [19], [20], [21], [22]). These techniques can enhance the quality and increase the reliability of these data.…”

Development and Experimental Evaluation of Machine-Learning Techniques for an Intelligent Hairy Scalp Detection System

“…Machine learning-based techniques are widely discussed, studied and applied for image classification, image recognition, and object detection in many fields [3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21]. The related application cases of machine learning-based image detection and classification are introduced as follows.…”

“…For traffic applications [3][4][5][6][7][8]19], Lousier and Abdelkrim [3] proposed a bag of features (Bove)-based machine learning framework for image classification, and this assessed the performance of training models using different image classification algorithms on the Caltech 101 images [4]. These authors also adopted the proposed BoF-based machine-learning framework to identify stop sign images for applying the trained classifier in a robotic system.…”

“…The error rate of their proposed CNN-LSTM-based model for multiple ego-motion classification was 0.0417. Pop et al [8] proposed different cross-modality CNN-based deep-learning training approaches for pedestrian recognition, and they consisted of a correlated model, an incremental model, and a particular cross-modality model.…”