A CNN-LSTM-based Deep Learning Approach for Driver Drowsiness Prediction

Abstract

The development of neural networks and machine learning techniques has recently been the cornerstone for many applications of artificial intelligence. These applications are now found in practically all aspects of our daily life. Predicting drowsiness is one of the most particularly valuable of artificial intelligence for reducing the rate of traffic accidents. According to earlier studies, drowsy driving is at responsible for 25 to 50% of all traffic accidents, which account for 1,200 deaths and 76,000 injuries annually. The goal of this research is to diminish car accidents caused by drowsy drivers. This research tests a number of popular deep learning-based models and presents a novel deep learning-based model for predicting driver drowsiness using a combination of convolutional neural networks (CNN) and Long-Short-Term Memory (LSTM) to achieve results that are superior to those of state-of-the-art methods. Utilizing convolutional layers, CNN has excellent feature extraction abilities, whereas LSTM can learn sequential dependencies. The National Tsing Hua University (NTHU) driver drowsiness dataset is used to test the model and compare it to several other current models as well as state-of-the-art models. The proposed model outperformed state-of-the-art models, with results up to 98.30% for training accuracy and 97.31% for validation accuracy.

A CNN-Based Approach for Driver Drowsiness Detection by Real-Time Eye State Identification

“…The methods used by [23,24] present another approach combining CNN and LSTM; this combination involves the use of convolutional layers of the CNN for feature extraction from the input data to then pass to the LSTM and make sequence prediction. The authors use this method by observing that drowsiness symptoms occur in small time sequences in the state of the eyes; it is a very robust technique.…”

“…The EAR determines the eye status by blink threshold values, which makes it dependent on those values when characterizing the size of the drivers' eyes; moreover, the thresholds are predefined for most drivers, while those thresholds vary for each driver. [20] Dlib Face 97% Rajkar et al [21] Haar Cascade Eyes 96.82% Hashemi et al [22] Haar Cascade/Dlib Eyes 98.15% Alameen and Alhothali [23] 3D-CNN+LSTM Face 96% Gomaa et al [24] CNN+LSTM Face 97.31% Singh et al [25] Dlib Eyes 80% This research proposes an efficient method for the correction and extraction of the region of interest of the eyes to be evaluated for drowsiness detection by means of transfer learning using deep learning with three CNNs (InceptionV3, VGG16 and ResNet50V2). In addition, a visual analysis is presented for each CNN, which other authors do not take into account or do not provide.…”

“…Gomaa et al [24] proposed several architectures combined with CNN and LSTM, the CNNs were based on ResNet, VGG-16, GoogleNet and MobileNet. The authors also propose their own architecture called "CNN-LSTM", training and testing the architectures on the "NTHU" dataset.…”

Drowsiness detection in real-time via convolutional neural networks and transfer learning

“…Gomaa, R. Mahmoud and. A. Sarhan [7] use a novel deep learning-based model for predicting driver drowsiness by combining CNN and long short-term memory (LSTM) networks, they achieved superior results compared to state-of-the-art methods, accurately predicting driver drowsiness from video footage captured during driving.…”

Efficient Eye State Detection for Driver Fatigue Monitoring Using Optimized YOLOv7-Tiny

“…Facial landmark analysis has also received attention, with technologies like MTCNN (Multi-task Cascaded Convolutional Neural Network) for face detection and classification systems achieving high accuracies in identifying drowsiness [15,16]. The exploration of 3D-CNN (Three-dimensional Convolutional Neural Network) integrated with LSTM (Long Short-term Memory) and other CNN architectures for facial expression analysis has yielded high accuracy rates on various datasets [17][18][19][20].…”