Fatigue Working Detection Based on Facial Multifeature Fusion

Yi, Ying; Zhang, Hu; Zhang, Wei; Yuan, Yahua; Li, Changping

doi:10.1109/jsen.2023.3239029

Cited by 5 publications

(3 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This enables the system to differentiate the driver's condition in scenarios characterized by long-term temporal dependencies, such as speaking while yawning or blinking while closing their eyes. Yi [8] proposed a fatigue detection system that integrates multiple facial features. By employing video processing, adjusting image intensity, and performing histogram equalization, essential facial characteristics were extracted.…”

Section: Multi-feature Detection Methodsmentioning

confidence: 99%

DA-ViViT: Fatigue detection framework using joint and facial keypoint features with dynamic distributed attention video vision transformer

Deng,

Yang,

Guo

et al. 2024

Preprint

View full text Add to dashboard Cite

Fatigue driving stands as a primary contributor to severe traffic accidents. Despite notable advancements in fatigue monitoring techniques, current methods frequently overlook the utilization of shifts in drivers' body postures induced by fatigue. These postural changes can offer valuable indicators of drivers' levels of alertness and degree of fatigue. Therefore, a real-time system with the ability to process body information is needed. This study presents a novel framework that integrates facial fatigue characteristics with alterations in drivers' upper body postures to comprehensively monitor fatigue across multiple dimensions. The neural network architecture at the core of the proposed approach utilizes a video vision transformer framework, with the self-attention mechanism optimized to accommodate the magnitude of fatigue-induced posture changes. This optimization ensures high accuracy while simultaneously reducing computational expenses. This gives the approach the potential for embedded deployment. According to requirements, the standard DMD dataset captured in real-world driving scenarios has been selected for validating the proposed method. In terms of accuracy, compared to detection methods that solely rely on facial and upper body poses, the multi-feature fatigue detection approach has achieved an improvement of 3.4% and 11.9%, reaching 96.24%. Additionally, it has been observed that the adoption of the Dynamic Distributed Attention-based backbone has led to a significant increase in inference speed of over 30%.

show abstract

Section: Multi-feature Detection Methodsmentioning

confidence: 99%

DA-ViViT: Fatigue detection framework using joint and facial keypoint features with dynamic distributed attention video vision transformer

Deng,

Yang,

Guo

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…Normally, the average range of head movement for adults in yaw is [−79.8 • , 75. Combined with the P80 standard, which has the largest correlation coefficient with the objective fatigue degree in the PERCLOS standard [18], this paper sets the Euler angle |Pitch| ≥ 20 • for the head down or the Euler angle |Roll| ≥ 15.4 • for the head tilt as the fatigue features of the head.…”

Section: Head Fatigue Test Based On Hpe Algorithmmentioning

confidence: 99%

A Multi-Feature Fusion and Situation Awareness-Based Method for Fatigue Driving Level Determination

2023

View full text Add to dashboard Cite

The detection and evaluation of fatigue levels in drivers play a crucial role in reducing traffic accidents and improving the overall quality of life. However, existing studies in this domain often focus on fatigue detection alone, with limited research on fatigue level evaluation. These limitations include the use of single evaluation methods and relatively low accuracy rates. To address these issues, this paper introduces an innovative approach for determining fatigue driving levels. We employ the Dlib library and fatigue state detection algorithms to develop a novel method specifically designed to assess fatigue levels. Unlike conventional approaches, our method adopts a multi-feature fusion strategy, integrating fatigue features from the eyes, mouth, and head pose. By combining these features, we achieve a more precise evaluation of the driver’s fatigue state level. Additionally, we propose a comprehensive evaluation method based on the analytic hierarchy process (AHP) and fuzzy comprehensive evaluation, combined with situational prediction. This approach effectively evaluates the fatigue state level of drivers at specific moments or stages and provides accurate predictions. Furthermore, we optimize the gated recurrent unit (GRU) network using an enhanced marine predator algorithm (MAP), which results in significant improvements in predicting fatigue levels during situational prediction. Experimental results demonstrate a classification accuracy of 92% across various scenarios while maintaining real-time performance. In summary, this paper introduces a novel approach for determining fatigue driving levels through multi-feature fusion. We also incorporate AHP-fuzzy comprehensive evaluation and situational prediction techniques, enhancing the accuracy and reliability of fatigue level evaluation. This research holds both theoretical and practical significance in the field of fatigue driving.

show abstract

“…Yang et al [12] proposed the KPE-YOLOv5 model, which improves the feature extraction capability for small objects by incorporating the scSE attention module and adding a small-object detection layer. In face detection of personnel, Ying et al [13] proposed a fatigue detection algorithm based on facial multifeature fusion. The video processing involved marking gray image frames and performing histogram equalization using the Dlib toolkit.…”

Section: Introductionmentioning

confidence: 99%

Driver Attention Detection Based on Improved YOLOv5

2023

View full text Add to dashboard Cite

In response to negative impacts such as personal and property safety hazards caused by drivers being distracted while driving on the road, this article proposes a driver’s attention state-detection method based on the improved You Only Look Once version five (YOLOv5). Both fatigue and distracted behavior can cause a driver’s attention to be diverted during the driving process. Firstly, key facial points of the driver are located, and the aspect ratio of the eyes and mouth is calculated. Through the examination of relevant information and repeated experimental verification, threshold values for the aspect ratio of the eyes and mouth under fatigue conditions, corresponding to closed eyes and yawning, are established. By calculating the aspect ratio of the driver’s eyes and mouth, it is possible to accurately detect whether the driver is in a state of fatigue. Secondly, distracted abnormal behavior is detected using an improved YOLOv5 model. The backbone network feature extraction element is modified by adding specific modules to obtain different receptive fields through multiple convolution operations on the input feature map, thereby enhancing the feature extraction ability of the network. The introduction of Swin Transformer modules in the feature fusion network replaces the Bottleneck modules in the C3 module, reducing the computational complexity of the model while increasing its receptive field. Additionally, the network connection in the feature fusion element has been modified to enhance its ability to fuse information from feature maps of different sizes. Three datasets were created of distracting behaviors commonly observed during driving: smoking, drinking water, and using a mobile phone. These datasets were used to train and test the model. After testing, the mAP (mean average precision) has improved by 2.4% compared to the model before improvement. Finally, through comparison and ablation experiments, the feasibility of this method has been verified, which can effectively detect fatigue and distracted abnormal behavior.

show abstract

Fatigue Working Detection Based on Facial Multifeature Fusion

Cited by 5 publications

References 25 publications

DA-ViViT: Fatigue detection framework using joint and facial keypoint features with dynamic distributed attention video vision transformer

DA-ViViT: Fatigue detection framework using joint and facial keypoint features with dynamic distributed attention video vision transformer

A Multi-Feature Fusion and Situation Awareness-Based Method for Fatigue Driving Level Determination

Driver Attention Detection Based on Improved YOLOv5

Contact Info

Product

Resources

About