Driver behavior detection and classification using deep convolutional neural networks

Shahverdy, Mohammad; Fathy, Mahmood; Berangi, Reza; Sabokrou, Mohammad

doi:10.1016/j.eswa.2020.113240

Cited by 203 publications

(96 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Recently, deep learning has made a major advances in recognising driver activities from images/videos [18], [14], [6], [17]. Driver's activity recognition can be seen as a subset of the traditional human activity recognition problem.…”

Section: Related Workmentioning

confidence: 99%

“…cues of body-objects interactions) plays a vital role [12], [13]. To improve the recognition accuracy, recently researchers have developed deep models focusing on spatio-temporal structures [14], [12], [15], [16], [17]. The main drawbacks in such approaches are: 1) mainly for solving video classification problems (complete observation) whereas our focus is on monitoring driver's on-going activity from partial observation so that the vehicle should be able to anticipate a distraction activity at the beginning.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Deep CNN, Body Pose, and Body-Object Interaction Features for Drivers’ Activity Monitoring

Behera

Wharton

Keidel

et al. 2022

IEEE Trans. Intell. Transport. Syst.

View full text Add to dashboard Cite

Automatic recognition and prediction of in-vehicle human activities has a significant impact on the next generation of driver assistance and intelligent autonomous vehicles. In this paper, we present a novel single image driver action recognition algorithm inspired by human perception that often focuses selectively on parts of the images to acquire information at specific places which are distinct to a given task. Unlike existing approaches, we argue that human activity is a combination of pose and semantic contextual cues. In detail, we model this by considering the configuration of body joints, their interaction with objects being represented as a pairwise relation to capture the structural information. Our body-pose and bodyobject interaction representation is built to be semantically rich and meaningful, and is highly discriminative even though it is coupled with a basic linear SVM classifier. We also propose a Multi-stream Deep Fusion Network (MDFN) for combining highlevel semantics with CNN features. Our experimental results demonstrate that the proposed approach significantly improves the drivers' action recognition accuracy on two exacting datasets. Index Terms-Transfer learning, intelligent vehicles, in-vehicle activity monitoring, deep learning, body pose and contextual descriptor, neural network-based fusion.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Deep CNN, Body Pose, and Body-Object Interaction Features for Drivers’ Activity Monitoring

Behera

Wharton

Keidel

et al. 2022

IEEE Trans. Intell. Transport. Syst.

View full text Add to dashboard Cite

show abstract

“…The overall prediction accuracy for this method reached 89.62%. Streiffer et al [24] proposed a deep learning-based method for acquiring distracted driving data and constructed an analysis system called DarNet, which achieved a classification accuracy of 87.02% on their collected dataset.…”

Section: Inceptionv3 Without a Classification Layermentioning

confidence: 99%

HCF: A Hybrid CNN Framework for Behavior Detection of Distracted Drivers

et al. 2020

View full text Add to dashboard Cite

Distracted driving causes a large number of traffic accident fatalities and is becoming an increasingly important issue in recent research on traffic safety. Gesture patterns are less distinguishable in vehicles due to in-vehicle physical constraints and body occlusions from the drivers. However, by capitalizing on modern camera technology, convolutional neural network (CNN) can be used for visual analysis. In this paper, we present a hybrid CNN framework (HCF) to detect the behaviors of distracted drivers by using deep learning to process image features. To improve the accuracy of the driving activity detection system, we first apply a cooperative pretrained model that combines ResNet50, Inception V3 and Xception to extract driver behavior features based on transfer learning. Second, because the features extracted by pretrained models are independent, we concatenate the extracted features to obtain comprehensive information. Finally, we train the fully connected layers of the HCF to filter out anomalies and hand movements associated with nondistracted driving. We apply an improved dropout algorithm to prevent the proposed HCF from overfitting to the training data. During the evaluation, we apply the class activation mapping (CAM) technique to highlight the feature area involving ten tested classes of typical distracted driving behaviors. The experimental results show that the proposed HCF achieves the classification accuracy of 96.74% when detecting distracted driving behaviors, demonstrating that it can potentially help drivers maintain safe driving habits.

show abstract

“…Cheng et al in 2018 presented an ANN trained by means features extracted by acceleration, pedal angle, and speed signals gathered from a car simulator to classify drivers into aggressive, normal, and calm [17]. Recently Shahverdy et al used a convolutional neural network to classify driving styles as: (i) normal; (ii) aggressive; (iii) distracted; (iv) drowsy; or (v) drunk starting from in-vehicle sensors [18]. Similarly, Zhang et al implemented a convolutional neural network which makes use of in-vehicle sensor data to distinguish the driver's own style [19].…”

Section: Related Workmentioning

confidence: 99%

Improving Machine Learning Identification of Unsafe Driver Behavior by Means of Sensor Fusion

Lattanzi¹,

Castellucci²,

Freschi³

2020

Applied Sciences

View full text Add to dashboard Cite

Most road accidents occur due to human fatigue, inattention, or drowsiness. Recently, machine learning technology has been successfully applied to identifying driving styles and recognizing unsafe behaviors starting from in-vehicle sensors signals such as vehicle and engine speed, throttle position, and engine load. In this work, we investigated the fusion of different external sensors, such as a gyroscope and a magnetometer, with in-vehicle sensors, to increase machine learning identification of unsafe driver behavior. Starting from those signals, we computed a set of features capable to accurately describe the behavior of the driver. A support vector machine and an artificial neural network were then trained and tested using several features calculated over more than 200 km of travel. The ground truth used to evaluate classification performances was obtained by means of an objective methodology based on the relationship between speed, and lateral and longitudinal acceleration of the vehicle. The classification results showed an average accuracy of about 88% using the SVM classifier and of about 90% using the neural network demonstrating the potential capability of the proposed methodology to identify unsafe driver behaviors.

show abstract

Driver behavior detection and classification using deep convolutional neural networks

Cited by 203 publications

References 35 publications

Deep CNN, Body Pose, and Body-Object Interaction Features for Drivers’ Activity Monitoring

Deep CNN, Body Pose, and Body-Object Interaction Features for Drivers’ Activity Monitoring

HCF: A Hybrid CNN Framework for Behavior Detection of Distracted Drivers

Improving Machine Learning Identification of Unsafe Driver Behavior by Means of Sensor Fusion

Contact Info

Product

Resources

About