Convolutional Neural Network-Based Classification of Driver’s Emotion during Aggressive and Smooth Driving Using Multi-Modal Camera Sensors

Abstract

Because aggressive driving often causes large-scale loss of life and property, techniques for advance detection of adverse driver emotional states have become important for the prevention of aggressive driving behaviors. Previous studies have primarily focused on systems for detecting aggressive driver emotion via smart-phone accelerometers and gyro-sensors, or they focused on methods of detecting physiological signals using electroencephalography (EEG) or electrocardiogram (ECG) sensors. Because EEG and ECG sensors cause discomfort to drivers and can be detached from the driver’s body, it becomes difficult to focus on bio-signals to determine their emotional state. Gyro-sensors and accelerometers depend on the performance of GPS receivers and cannot be used in areas where GPS signals are blocked. Moreover, if driving on a mountain road with many quick turns, a driver’s emotional state can easily be misrecognized as that of an aggressive driver. To resolve these problems, we propose a convolutional neural network (CNN)-based method of detecting emotion to identify aggressive driving using input images of the driver’s face, obtained using near-infrared (NIR) light and thermal camera sensors. In this research, we conducted an experiment using our own database, which provides a high classification accuracy for detecting driver emotion leading to either aggressive or smooth (i.e., relaxed) driving. Our proposed method demonstrates better performance than existing methods.

Deep CNN: A Machine Learning Approach for Driver Drowsiness Detection Based on Eye State

“…Feature extraction is one type of dimensionality reduction where useful parts of an image represented as a feature vector. In this paper features from the eye region images are extracted using a Convolutional Neural Network (CNN) [22][23][24].…”

Predicting Heat Meters’ Failures With Selected Machine Learning Models

“…Thanks to the increase of the computational power, we are no longer so much limited by the number of layers of hidden networks when designing a network -hence the popularity of the DNNs (deep neural networks). The application of specialist filters and image data preprocessing enabled CNNs (convolutional neural networks), which work well in handwriting recognition or image classification ( [35], [36] The results are a bit better than in case of SVM, but also here predicting failure is rather poor.…”

Deep Learning-Based Drivers Emotion Classification System in Time Series Data for Remote Applications

“…Visual cues of drivers can be used for finding emotional state, fatigue, or abnormal behavior. Visual behavior based methods can be classified into multi cameras-based [25][26][27][28][29] and single camera-based methods [30][31][32][33][34][35][36][37]. Multi cameras-based methods were also been used by researchers for classifying driver's behaviors while using visual cues.…”

“…Two PERCLOS cameras were used by them for detecting drowsiness in truck drivers. They performed in-vehicle experiment while using illuminated eye detection and PERCLOS measurement [25].…”