Hypo-Driver: A Multiview Driver Fatigue and Distraction Level Detection System

Abstract

Traffic accidents are caused by driver fatigue or distraction in many cases. To prevent accidents, several low-cost hypovigilance (hypo-V) systems were developed in the past based on a multimodal-hybrid (physiological and behavioral) feature set. Similarly in this paper, real-time driver inattention and fatigue (Hypo-Driver) detection system is proposed through multi-view cameras and biosignal sensors to extract hybrid features. The considered features are derived from non-intrusive sensors that are related to the changes in driving behavior and visual facial expressions. To get enhanced visual facial features in uncontrolled environment, three cameras are deployed on multiview points (0 • , 45 • , and 90 • ) of the drivers. To develop a Hypo-Driver system, the physiological signals (electroencephalography (EEG), electrocardiography (ECG), electro-myography (sEMG), and electrooculography (EOG)) and behavioral information (PERCLOS70-80-90%, mouth aspect ratio (MAR), eye aspect ratio (EAR), blinking frequency (BF), head-titled ratio (HT-R)) are collected and pre-processed, then followed by feature selection and fusion techniques. The driver behaviors are classified into five stages such as normal, fatigue, visual inattention, cognitive inattention, and drowsy. This improved hypo-Driver system utilized trained behavioral features by a convolutional neural network (CNNs), recurrent neural network and long short-term memory (RNN-LSTM) model is used to extract physiological features. After fusion of these features, the Hypo-Driver system is classified hypo-V into five stages based on trained layers and dropout-layer in the deep-residual neural network (DRNN) model. To test the performance of a hypo-Driver system, data from 20 drivers are acquired. The results of Hypo-Driver compared to state-of-theart methods are presented. Compared to the state-of-the-art Hypo-V system, on average, the Hypo-Driver system achieved a detection accuracy (AC) of 96.5%. The obtained results indicate that the Hypo-Driver system based on multimodal and multiview features outperforms other state-of-the-art driver Hypo-V systems by handling many anomalies.

Research on the Training and Management of Industrializing Workers in Prefabricated Building with Machine Vision and Human Behaviour Modelling Based on Industry 4.0 Era

“…By comparing the facial expressions of a great number of normal people with those of psychopaths, a theoretical model for emotional decoding of human behaviour and facial expressions was developed [ 26 ]. Riquelme et al collected a large amount of facial expression data of drivers and extracted feature codes to develop a driver fatigue monitoring system, which detects the driver's mouth aspect ratio, blinking frequency, and head tilt angle to determine whether the driver is in fatigue and alerts control staff in time, so as to avoid traffic accidents [ 27 ]. In a study on classroom teaching, Schneider et al found that facial expressions are expressions of mental activity and that different facial expressions have a greater impact on learning efficiency.…”

A multi‐scale feature fusion convolutional neural network for facial expression recognition

“…For example, in medical applications (Fu et al, 2023; Jiang et al, 2020; Liu et al, 2019), psychologists determine a patient's condition by constantly observing changes in the patient's facial expressions to suggest a treatment plan. In the application of fatigue driving (Abbas et al, 2022; Xiao et al, 2022), if the in‐vehicle detection system detects signs of fatigue driving in the driver's facial expressions, it will emit an alarm to remind the driver. In sales applications (Ijjina et al, 2020), customers' facial expressions are essential data for computers to determine whether they need a human sales assistant.…”