Distracted driver detection by combining in-vehicle and image data using deep learning

Omerustaoglu, Furkan; Şakar, C. Okan; Kar, Gorkem

doi:10.1016/j.asoc.2020.106657

Cited by 58 publications

(33 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Those studies train the deep learning network on the set of the physical exercise data and predict the type of activity during normal and fatigue driving conditions. The training results are obtained by application of various machine learning methods by CNN deep learning neural networks [15,24] for the data obtained by multimodal channels (acceleration and heart activity). Despite the tendency to learn from the training data, the loss is very high for most combinations of parameters, and the abrupt decrease of the loss for two of these combinations is just an illustration of over-training, but not the mark of the very reliable model.…”

Section: Resultsmentioning

confidence: 99%

“…In reference [24], the authors determine distracted driving through the vehicle's driver image and multisensor to define multimodal features. The convolutional neural network (CNN) models are created by transfer learning and then use the recurrent neural network and long short-term memory (RNN-LSTM) model for predicting the Hypo-V stage.…”

Section: Literature Reviewmentioning

confidence: 99%

See 1 more Smart Citation

Hypo-Driver: A Multiview Driver Fatigue and Distraction Level Detection System

Riquelme¹,

Olivares²,

Mu駉z³

et al. 2022

Computers, Materials &Amp; Continua

View full text Add to dashboard Cite

Traffic accidents are caused by driver fatigue or distraction in many cases. To prevent accidents, several low-cost hypovigilance (hypo-V) systems were developed in the past based on a multimodal-hybrid (physiological and behavioral) feature set. Similarly in this paper, real-time driver inattention and fatigue (Hypo-Driver) detection system is proposed through multi-view cameras and biosignal sensors to extract hybrid features. The considered features are derived from non-intrusive sensors that are related to the changes in driving behavior and visual facial expressions. To get enhanced visual facial features in uncontrolled environment, three cameras are deployed on multiview points (0 • , 45 • , and 90 • ) of the drivers. To develop a Hypo-Driver system, the physiological signals (electroencephalography (EEG), electrocardiography (ECG), electro-myography (sEMG), and electrooculography (EOG)) and behavioral information (PERCLOS70-80-90%, mouth aspect ratio (MAR), eye aspect ratio (EAR), blinking frequency (BF), head-titled ratio (HT-R)) are collected and pre-processed, then followed by feature selection and fusion techniques. The driver behaviors are classified into five stages such as normal, fatigue, visual inattention, cognitive inattention, and drowsy. This improved hypo-Driver system utilized trained behavioral features by a convolutional neural network (CNNs), recurrent neural network and long short-term memory (RNN-LSTM) model is used to extract physiological features. After fusion of these features, the Hypo-Driver system is classified hypo-V into five stages based on trained layers and dropout-layer in the deep-residual neural network (DRNN) model. To test the performance of a hypo-Driver system, data from 20 drivers are acquired. The results of Hypo-Driver compared to state-of-theart methods are presented. Compared to the state-of-the-art Hypo-V system, on average, the Hypo-Driver system achieved a detection accuracy (AC) of 96.5%. The obtained results indicate that the Hypo-Driver system based on multimodal and multiview features outperforms other state-of-the-art driver Hypo-V systems by handling many anomalies.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Literature Reviewmentioning

confidence: 99%

Hypo-Driver: A Multiview Driver Fatigue and Distraction Level Detection System

Riquelme¹,

Olivares²,

Mu駉z³

et al. 2022

Computers, Materials &Amp; Continua

View full text Add to dashboard Cite

show abstract

“…Andrew Liu proposed that driver control behavior can be predicted through vehicle movement behavior [ 10 ]. Omerustaoglu et al combined in-car data and image data to study distracted driving behaviors through deep learning [ 11 ]. Jeong et al used the data collected by the built-in 3-axis gyroscope of the vehicle to identify two driving behaviors by support vector machine [ 12 ].…”

Section: Related Workmentioning

confidence: 99%

A hybrid neural network for driving behavior risk prediction based on distracted driving behavior data

Meng

Wang

et al. 2022

PLoS ONE

View full text Add to dashboard Cite

Distracted driving behavior is one of the main factors of road accidents. Accurately predicting the risk of driving behavior is of great significance to the active safety of road transportation. The large amount of information collected by the sensors installed on the vehicle can be identified by the algorithm to obtain the distracted driving behavior data, which can be used to predict the driving behavior risk of the vehicle and the area. In this paper, a new neural network named Driving Behavior Risk Prediction Neural Network (DBRPNN) is developed for prediction based on the distracted driving behavior data. The network consists of three modules: the Feature Processing Module, the Memory Module, and the Prediction Module. In this process, attribute data (time in a day, daily driving time, and daily driving mileage) that can reflect external factors and driver statuses, are added to the network to increase the accuracy of the model. We predicted the driving behavior risk of different objects (Vehicle and Area). For the applicability improvement of the model, we further classify the distracted driving behavior categories, and DBRPNN can provide more accurate risk prediction. The results show that compared with traditional models (Classification and Regression Tree, Support Vector Machines, Recurrent Neural Network, and Long Short-Term Memory), DBRPNN has better prediction performance. The method proposed in this paper has been fully verified and may be transplanted into active safety early warning system for more accurate and flexible application.

show abstract

“…In Huang (2011) , the real-time recognition of vehicle Z-curve driving state based on image processing technology was proposed, which would automatically warn and provide feedback to the driver when the relevant image monitoring metric exceeded the preset threshold. Omerustaoglu et al (2020) studied the driver’s distracted driving behavior by combining in-vehicle and image data using deep learning. Based on the theory of support vector machine (SVM), Jeong et al (2013) recognized two kinds of driving behaviors, namely lane-changing and Z-curve driving using the data collected by the built-in 3-axis gyroscope of vehicle.…”

Section: Related Workmentioning

confidence: 99%

Prediction of Dangerous Driving Behavior Based on Vehicle Motion State and Passenger Feeling Using Cloud Model and Elman Neural Network

et al. 2021

View full text Add to dashboard Cite

Dangerous driving behavior is the leading factor of road traffic accidents; therefore, how to predict dangerous driving behavior quickly, accurately, and robustly has been an active research topic of traffic safety management in the past decades. Previous works are focused on learning the driving characteristic of drivers or depended on different sensors to estimate vehicle state. In this paper, we propose a new method for dangerous driving behavior prediction by using a hybrid model consisting of cloud model and Elman neural network (CM-ENN) based on vehicle motion state estimation and passenger’s subjective feeling scores, which is more intuitive in perceiving potential dangerous driving behaviors. To verify the effectiveness of the proposed method, we have developed a data acquisition system of driving motion states and apply it to real traffic scenarios in ShenZhen city of China. Experimental results demonstrate that the new method is more accurate and robust than classical methods based on common neural network.

show abstract

Distracted driver detection by combining in-vehicle and image data using deep learning

Cited by 58 publications

References 36 publications

Hypo-Driver: A Multiview Driver Fatigue and Distraction Level Detection System

Hypo-Driver: A Multiview Driver Fatigue and Distraction Level Detection System

A hybrid neural network for driving behavior risk prediction based on distracted driving behavior data

Prediction of Dangerous Driving Behavior Based on Vehicle Motion State and Passenger Feeling Using Cloud Model and Elman Neural Network

Contact Info

Product

Resources

About