Real-time and Robust Driver Yawning Detection with Deep Neural Networks

Xie, Yongquan; Kexun, Chen; Murphey, Yi Lu

doi:10.1109/ssci.2018.8628881

Cited by 15 publications

(7 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The CNN extracts spatial features from individual frames, and the LSTM network analyzes the temporal features of driver actions between the adjacent frames. For example, some authors attempted to employ this CNN-LSTM network to build a DDD system depending only on the analysis of yawning features [19][20][21]. Most of them used a public dataset named YawDD.…”

Section: Spatiotemporal-based Systemsmentioning

confidence: 99%

“…One of the main differences between these studies is the type of inputs to the models. Xie et al [19] use the whole frames as inputs, Zhang et al [20] use frame edges, and Fei et al [21] focus on the extracted mouth regions from the drivers' faces. Other researchers utilized the CNN-LSTM network to learn the spatiotemporal feature of drivers' eyes.…”

Section: Spatiotemporal-based Systemsmentioning

confidence: 99%

“…The different style of drowsiness actions for each driver is the reason behind the complexity of DDD systems. For that, building a system that depends on only yawning detection [13,[19][20][21]23] or eye state recognition [15][16][17]22] cannot be used as an effective real-life DDD system.…”

Section: Nthu-dddmentioning

confidence: 99%

See 2 more Smart Citations

A Lightweight Driver Drowsiness Detection System Using 3DCNN With LSTM

Alameen¹,

Alhothali²

2023

Computer Systems Science and Engineering

View full text Add to dashboard Cite

Today, fatalities, physical injuries, and significant economic losses occur due to car accidents. Among the leading causes of car accidents is drowsiness behind the wheel, which can affect any driver. Drowsiness and sleepiness often have associated indicators that researchers can use to identify and promptly warn drowsy drivers to avoid potential accidents. This paper proposes a spatiotemporal model for monitoring drowsiness visual indicators from videos. This model depends on integrating a 3D convolutional neural network (3D-CNN) and long short-term memory (LSTM). The 3DCNN-LSTM can analyze long sequences by applying the 3D-CNN to extract spatiotemporal features within adjacent frames. The learned features are then used as the input of the LSTM component for modeling high-level temporal features. In addition, we investigate how the training of the proposed model can be affected by changing the position of the batch normalization (BN) layers in the 3D-CNN units. The BN layer is examined in two different placement settings: before the non-linear activation function and after the non-linear activation function. The study was conducted on two publicly available drowsy drivers datasets named 3MDAD and YawDD. 3MDAD is mainly composed of two synchronized datasets recorded from the frontal and side views of the drivers. We show that the position of the BN layers increases the convergence speed and reduces overfitting on one dataset but not the other. As a result, the model achieves a test detection accuracy of 96%, 93%, and 90% on YawDD, Side-3MDAD, and Front-3MDAD, respectively.

show abstract

Section: Spatiotemporal-based Systemsmentioning

confidence: 99%

Section: Spatiotemporal-based Systemsmentioning

confidence: 99%

See 1 more Smart Citation

A Lightweight Driver Drowsiness Detection System Using 3DCNN With LSTM

Alameen¹,

Alhothali²

2023

Computer Systems Science and Engineering

View full text Add to dashboard Cite

show abstract

“…Mehta et al [26] have created a mobile app that can detect facial landmarks and then compute the Eye Aspect Ratio (EAR) and Eye Closure Ratio (ECR) to predict driver drowsiness with an accuracy of 84% based on machine learning models. Smart glass has been created by a start-up Ellcie-Healthy [30] that incorporates a somnolence monitoring technology by providing blink detection, Eye recording, and control of vital signs.…”

Section: Literatue Reviewmentioning

confidence: 99%

Real-time Driver Drowsiness Detection using Deep Learning

Dipu¹,

Hossain²,

Arafat³

et al. 2021

IJACSA

View full text Add to dashboard Cite

Every year thousands of lives pass away worldwide due to vehicle accidents, and the main reason behind this is the drowsiness in drivers. A drowsiness detection system will help to reduce this accident and save many lives around the world. To defend this problem, we propose a methodology based on Convolutional Neural Networks (CNN) that illustrates drowsiness detection as a task to detect an object. It will detect and localize whether the eyes are open or close based on the realtime video stream of drivers. The MobileNet CNN Architecture with Single Shot Multibox Detector is the technology used for this object detection task. A separate algorithm is used based on the output given by the SSD_MobileNet_v1 architecture. A dataset that consists of around 4500 images was labeled with the object's face yawn, no-yawn, open eye, and closed eye to train the SSD_MobileNet_v1 Network. Around 600 randomly selected images are used to test the trained model using the PASCAL VOC metric. The proposed approach is to ensure better accuracy and computational efficiency. It is also affordable as it can process incoming video streams in real-time and does not need any expensive hardware support. There only needs a standalone camera to be implemented using cheap devices in cars using Raspberry Pi 3 or other IP cameras.

show abstract

“…This study was conducted on the NTHU [15] dataset. Another study conducted by Xie et al [16] used transfer learning and sequential learning from yawning video clips to detect yawning on the YawDD and NTHU-DDD database. This system was able to have higher precision and was robust to changes in the position and angle of the face to the camera.…”

Section: Literature Reviewmentioning

confidence: 99%

Driver Drowsiness Detection Model Using Convolutional Neural Networks Techniques for Android Application

Jabbar¹,

Shinoy²,

Kharbeche³

et al. 2020

Preprint

View full text Add to dashboard Cite

A sleepy driver is arguably much more dangerous on the road than the one who is speeding as he is a victim of microsleeps. Automotive researchers and manufacturers are trying to curb this problem with several technological solutions that will avert such a crisis. This article focuses on the detection of such micro sleep and drowsiness using neural networkbased methodologies. Our previous work in this field involved using machine learning with multi-layer perceptron to detect the same. In this paper, accuracy was increased by utilizing facial landmarks which are detected by the camera and that is passed to a Convolutional Neural Network (CNN) to classify drowsiness. The achievement with this work is the capability to provide a lightweight alternative to heavier classification models with more than 88% for the category without glasses, more than 85% for the category night without glasses. On average, more than 83% of accuracy was achieved in all categories. Moreover, as for model size, complexity and storage, there is a marked reduction in the new proposed model in comparison to the benchmark model where the maximum size is 75 KB. The proposed CNN based model can be used to build a real-time driver drowsiness detection system for embedded systems and Android devices with high accuracy and ease of use.

show abstract

Real-time and Robust Driver Yawning Detection with Deep Neural Networks

Cited by 15 publications

References 19 publications

A Lightweight Driver Drowsiness Detection System Using 3DCNN With LSTM

A Lightweight Driver Drowsiness Detection System Using 3DCNN With LSTM

Real-time Driver Drowsiness Detection using Deep Learning

Driver Drowsiness Detection Model Using Convolutional Neural Networks Techniques for Android Application

Contact Info

Product

Resources

About