A review of driver fatigue detection and its advances on the use of RGB-D camera and deep learning

Fan, Liu; Chen, Delong; Zhou, Jun; Xu, Feng

doi:10.1016/j.engappai.2022.105399

Cited by 26 publications

(3 citation statements)

References 98 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this paper, the K-means clustering algorithm is used to cluster image annotation files in the face region data set, and the clustering objects are the width and height of prior boxes [9] . The SSD network selects six convolutional layers as feature extraction layers to output the feature map, and each output feature map is assigned a detection box.…”

Section: Prior Box Improvementmentioning

confidence: 99%

Improved SSD for crew fatigue driving detection algorithm

Liu,

Zhang,

Xiao

et al. 2024

Seventh International Conference on Traffic Engineering and Transportation System (ICTETS 2023)

View full text Add to dashboard Cite

To solve the problems of low recognition accuracy and slow detection of crew fatigue driving behavior in the cockpit of ships during the process of sailing in and out of the port, the SSD model was studied. By replacing its backbone network and improving the prior frame generation mechanism, the MV-SSD model is proposed. Replace the backbone network VGG16 in the original SSD model with MobileNetV3, reducing the network parameters of the backbone network. Using the K-means algorithm to cluster the real detection boxes in the face area dataset, the prior box allocation mechanism of the SSD model was redesigned, reducing the number of prior box generation by nearly half, and the ERT algorithm in the Dlib library is combined to locate the face key points, and finally, the PERCLOS criterion is used to determine whether the driver is fatigued. Experimental results show that the average accuracy (mAP value) of the MV-SSD model is 7.15% higher than that of the original SSD model, and the detection speed (FPS value) is increased by 98frames/s, which is more suitable for the detection of the crew face area, and the average accuracy of the constructed fatigue detection algorithm for fatigue features is more than 94%.

show abstract

Section: Prior Box Improvementmentioning

confidence: 99%

Improved SSD for crew fatigue driving detection algorithm

Liu,

Zhang,

Xiao

et al. 2024

Seventh International Conference on Traffic Engineering and Transportation System (ICTETS 2023)

View full text Add to dashboard Cite

show abstract

“…The results show that the accuracy of the system reaches 85%. Liu [ 29 ] focused on RGB-D cameras and deep learning generative adversarial networks and utilized multi-channel schemes to improve fatigue detection performance. Research indicates that fatigue features extracted with convolutional neural networks outperform traditional manual fatigue features.…”

Section: Introductionmentioning

confidence: 99%

Remote Photoplethysmography and Motion Tracking Convolutional Neural Network with Bidirectional Long Short-Term Memory: Non-Invasive Fatigue Detection Method Based on Multi-Modal Fusion

Kong,

Xie,

Niu

et al. 2024

Sensors

View full text Add to dashboard Cite

Existing vision-based fatigue detection methods commonly utilize RGB cameras to extract facial and physiological features for monitoring driver fatigue. These features often include single indicators such as eyelid movement, yawning frequency, and heart rate. However, the accuracy of RGB cameras can be affected by factors like varying lighting conditions and motion. To address these challenges, we propose a non-invasive method for multi-modal fusion fatigue detection called RPPMT-CNN-BiLSTM. This method incorporates a feature extraction enhancement module based on the improved Pan–Tompkins algorithm and 1D-MTCNN. This enhances the accuracy of heart rate signal extraction and eyelid features. Furthermore, we use one-dimensional neural networks to construct two models based on heart rate and PERCLOS values, forming a fatigue detection model. To enhance the robustness and accuracy of fatigue detection, the trained model data results are input into the BiLSTM network. This generates a time-fitting relationship between the data extracted from the CNN, allowing for effective dynamic modeling and achieving multi-modal fusion fatigue detection. Numerous experiments validate the effectiveness of the proposed method, achieving an accuracy of 98.2% on the self-made MDAD (Multi-Modal Driver Alertness Dataset). This underscores the feasibility of the algorithm. In comparison with traditional methods, our approach demonstrates higher accuracy and positively contributes to maintaining traffic safety, thereby advancing the field of smart transportation.

show abstract

“…To process this data, specific deep learning algorithms (e.g. Fuzzy algorithms) are often used together with ANN (artificial neural networks - (Liu et al, 2022).…”

Section: Introductionmentioning

confidence: 99%

Driver alertness monitoring system in the context of safety increasing and sustainable energy use

Scurt

Beles²,

Vesselényi³

et al. 2023

CogSust

View full text Add to dashboard Cite

Road transport is an important factor in carbon dioxide emissions. These emissions can be reduced by improving propulsion sources and traffic flow (avoiding traffic jams due to accidents). This article presents a system for monitoring and warning the driver to prevent a possible accident involving material damage, injury, or loss of life. The system performs video monitoring of the driver in order to determine his state (tired or attentive). By reducing traffic incidents and traffic jams, the energy consumed will not be wasted; thus, more sustainable transport energy use can be achieved.

show abstract

A review of driver fatigue detection and its advances on the use of RGB-D camera and deep learning

Cited by 26 publications

References 98 publications

Improved SSD for crew fatigue driving detection algorithm

Improved SSD for crew fatigue driving detection algorithm

Remote Photoplethysmography and Motion Tracking Convolutional Neural Network with Bidirectional Long Short-Term Memory: Non-Invasive Fatigue Detection Method Based on Multi-Modal Fusion

Driver alertness monitoring system in the context of safety increasing and sustainable energy use

Contact Info

Product

Resources

About