Using Deep Learning in Infrared Images to Enable Human Gesture Recognition for Autonomous Vehicles

Abstract

The realization of a novel human gesture recognition algorithm is essential to enable the effective collision avoidance of autonomous vehicles. Compared to visible spectrum cameras, the use of infrared imaging can enable more robust human gesture recognition in a complex environment. However, gesture recognition in infrared images has not been extensively investigated. In this work, we propose a model to detect human gestures, based on the improved YOLO-V3 network involving a saliency map as the second input channel to enhance the reuse of features and improve the network performance. Three DenseNet blocks are added before the residual components in the YOLO-V3 network to enhance the convolutional feature propagation. The saliency maps are obtained by multiscale superpixel segmentation, superpixel block clustering and cellular automata saliency detection. The obtained five scale saliency maps are fused using a Bayesian based fusion algorithm, and the final saliency image is generated. The infrared images composed of 4 main gesture classes are collected, each of which contains several approximated gestures in morphological terms. The training and testing datasets are generated, including original and augmented infrared images with a resolution of 640 × 480. The experimental results show that the proposed approach can enable real time human gesture detection for autonomous vehicles, with an average detection accuracy of 86.2%. INDEX TERMS Human gesture recognition, autonomous vehicles, deep learning approach, infrared images, saliency maps.

A Spatio-Temporal Multilayer Perceptron for Gesture Recognition

“…traffic control officers and pedestrians [23], [24]. Further studies address other aspects of the problem, such as the lack of public datasets on traffic control gestures [5], relying on robust motion capture sensors [25], driver behavior prediction [26], and pedestrian intention prediction [27], [28], [29]. These works shed light on the importance of an accurate understanding of the human body language for automated driving.…”

A robust multiclass 3D object recognition based on modern YOLO deep learning algorithms

“…The following parameters were used to estimate the efficiency of the detection model. 29 The precision (P) and recall (R) can be classified as follows:…”

Maneuvering Digital Watermarking In Face Recognition

“…digital images hold thousands of secret and important data, be it personal or medical images. Accordingly, many works have been carried out to process the images and discover the information they contain, whether medical images [5][6][7][8][9][10][11], face recognition [12,13], or vehicles [14,15]. Therefore, these images need a way to protect them from the information they contain.…”

“…The Natural Preserve Transform (NPT) of S pm is gotten as: Ap = ѱ (α) S pm ѱ (α) (13) This step uniformly distributes the watermark within the host image in the background. For making the watermarking logo invisible, the last rows, r, are replaced with A p, along with the original image's last row.…”