Real-Time Driver Drowsiness Detection using Facial Action Units

Vijay, M.; Vinayak, Nandagopal Netrakanti; Nunna, Maanvi; Natarajan, S.

doi:10.1109/icpr48806.2021.9412288

Cited by 8 publications

(5 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the domain of driver state analysis systems, research primarily revolves around the development of a single integrated system, in contrast to fundamental technological research such as object recognition or image Visual RGB Huang et al 37 MTCNN, LSTM NTHU-DDD 38 Vijay et al 39 CNN, XGBoost 5, employ a diverse range of sensors including RGB, Depth, IR, NIR, EEG, and Motion capture for data acquisition. In this paper, to facilitate the actual commercial deployment of DMS, we custom-designed a camera tailored for driver state analysis.…”

Section: Comparative Experiments With Existing Driver Monitoring Systemsmentioning

confidence: 99%

Real-time driver monitoring system with facial landmark-based eye closure detection and head pose recognition

Kim,

Park,

Kim

et al. 2023

Sci Rep

View full text Add to dashboard Cite

This paper introduces a real-time Driver Monitoring System (DMS) designed to monitor driver behavior while driving, employing facial landmark estimation-based behavior recognition. The system utilizes an infrared (IR) camera to capture and analyze video data. Through facial landmark estimation, crucial information about the driver’s head posture and eye area is extracted from the detected facial region, obtained via face detection. The proposed method consists of two distinct modules, each focused on recognizing specific behaviors. The first module employs head pose analysis to detect instances of inattention. By monitoring the driver’s head movements along the horizontal and vertical axes, this module assesses the driver’s attention level. The second module implements an eye-closure recognition filter to identify instances of drowsiness. Depending on the continuity of eye closures, the system categorizes them as either occasional drowsiness or sustained drowsiness. The advantages of the proposed method lie in its efficiency and real-time capabilities, as it solely relies on IR camera video for computation and analysis. To assess its performance, the system underwent evaluation using IR-Datasets, demonstrating its effectiveness in monitoring and recognizing driver behavior accurately. The presented real-time Driver Monitoring System with facial landmark-based behavior recognition offers a practical and robust approach to enhance driver safety and alertness during their journeys.

show abstract

Section: Comparative Experiments With Existing Driver Monitoring Systemsmentioning

confidence: 99%

Real-time driver monitoring system with facial landmark-based eye closure detection and head pose recognition

Kim,

Park,

Kim

et al. 2023

Sci Rep

View full text Add to dashboard Cite

show abstract

“…There are numerous studies related to driver fatigue detection, however, research related to subtle facial muscle changes as an early warning indicator for fatigue are less explored. Facial Action Coding System (FACS) has become a standard for facial expression identification [13], however, there is a paucity of literature on identifying human weariness. Facial AUs related to yawn (AU25,26,27) and eye blinking (AU45) are usually considered [13], on the other hand other possible AUs are barely explored.…”

Section: Literature Reviewmentioning

confidence: 99%

“…Facial Action Coding System (FACS) has become a standard for facial expression identification [13], however, there is a paucity of literature on identifying human weariness. Facial AUs related to yawn (AU25,26,27) and eye blinking (AU45) are usually considered [13], on the other hand other possible AUs are barely explored. Depiction accuracy of facial muscular changes based AUs information is limited in 2D data, since early fatigue changes are inconspicuous, hence 3D data offers a better representation of these minuscule facial variations.…”

Section: Literature Reviewmentioning

confidence: 99%

An Adaptive Snake Based Shadow Segmentation for Robust Driver Fatigue Detection: A 3D Facial Feature Based Photometric Stereo Perspective

Sikander,

Anwar,

Husnain

et al. 2023

IEEE Access

View full text Add to dashboard Cite

Fatigue detection has many applications, one of them is intelligent transportation for accident prevention. It is challenging to detect driver fatigue adopting photometric stereo and 3D fatigue-related facial action unit identification. During 3D reconstruction shadows are encountered, causing reconstruction error, therefore, for accurate fatigue detection shadow removal is crucial. This study presents a novel snake based approach for shadow handling in photometric stereo. Seven light sources were employed and region visibility of the light source for each region of interest (ROI) was deduced. Surface normal estimation enhancement was achieved via snake model based shadow labeling. Discrepancies measured among shadowed regions and ROIs were recorded. Light sources having high intersection with the shadow map were excluded from the reconstruction process at pixel level. This results in an automatic driver fatigue related action unit detection employing uncalibrated Lambertian surface under shadow conditions. The developed method was tested and compared with other established driver fatigue detection methods and higher performance in terms of accuracy ( i.e. 97.85%) was achieved. The developed method proves to be well suited for driver fatigue detection under uncalibrated conditions.

show abstract

“…1) Input and Feature Extraction: Most algorithms rely on driver-facing video cameras to detect drowsiness. Nearinfrared imaging (NIR) cameras are often used (along [118], [120], [128] or combined with visible imaging color cameras [130]) due to their versatility for day and night conditions and robustness to changes in illumination and low light conditions.…”

Section: Drowsiness Detectionmentioning

confidence: 99%

“…2) Classifiers: Rule-based approaches and thresholding are computationally the simplest methods for detecting drowsiness [132], [140], [141], [148], however, they are susceptible to differences between drivers and variations of signal due to changes in illumination and vibration of the vehicle. Learning methods such as SVM [135], [137], [145], boosting [128], or HMMs [124], are more robust in practice.…”

Section: Drowsiness Detectionmentioning

confidence: 99%

Attention for Vision-Based Assistive and Automated Driving: A Review of Algorithms and Datasets

Kotseruba

Tsotsos

2022

IEEE Trans. Intell. Transport. Syst.

View full text Add to dashboard Cite

Driving safety has been a concern since the first cars appeared on the streets. Driver inattention has been singled out as a major cause of accidents early on. This is hardly surprising, as drivers routinely perform other tasks in addition to controlling the vehicle. Decades of research into what causes lapses or misdirection of drivers' attention resulted in improvements in road safety through better design of infrastructure, driver training programs, in-vehicle interfaces, and, more recently, the development of driving assistance systems (ADAS) and driving automation. This review focuses on the methods for modeling and detecting spatio-temporal aspects of drivers' attention, i.e. where and when they look, for the two latter categories of applications.We start with a brief theoretical background on human visual attention, methods for recording and measuring attention in the driving context, types of driver inattention, and factors causing it. We then discuss machine learning approaches for 1) modeling gaze for assistive and self-driving applications and 2) detecting gaze for driver monitoring. Following the overview of state-of-the-art models, we provide an extensive list of publicly available datasets that feature recordings of drivers' gaze and other attention-related annotations. We conclude with a general overview of the remaining challenges, such as data availability and quality, evaluation methods, and the limited scope of attention modeling, and outline steps toward rectifying some of these issues. Categorized and annotated lists of the reviewed models and datasets are available at https://github.com/ykotseruba/attention_and_driving

show abstract

Real-Time Driver Drowsiness Detection using Facial Action Units

Cited by 8 publications

References 13 publications

Real-time driver monitoring system with facial landmark-based eye closure detection and head pose recognition

Real-time driver monitoring system with facial landmark-based eye closure detection and head pose recognition

An Adaptive Snake Based Shadow Segmentation for Robust Driver Fatigue Detection: A 3D Facial Feature Based Photometric Stereo Perspective

Attention for Vision-Based Assistive and Automated Driving: A Review of Algorithms and Datasets

Contact Info

Product

Resources

About