Driver Drowsiness Detection Using Condition-Adaptive Representation Learning Framework

Yu, Jongmin; Park, Sangwoo; Lee, Sangwook; Jeon, Moongu

doi:10.1109/tits.2018.2883823

Cited by 92 publications

(46 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To fully model the problem of the complex correlation between two modal features in the process of computer vision tasks, Hong et al [26] observed that the correlation between related features can be captured using the element-wise multiplication interaction between the feature maps. Therefore, Yu et al [27] fused the spatio-temporal feature representation obtained from 3D-DCNN with the related scene annotation by element-wise multiplication interaction, and added a set of conditional adaptive representation to effectively distinguish driver drowsiness.…”

Section: Related Workmentioning

confidence: 99%

Driver Drowsiness Estimation Based on Factorized Bilinear Feature Fusion and a Long-Short-Term Recurrent Convolutional Network

2020

View full text Add to dashboard Cite

The effective detection of driver drowsiness is an important measure to prevent traffic accidents. Most existing drowsiness detection methods only use a single facial feature to identify fatigue status, ignoring the complex correlation between fatigue features and the time information of fatigue features, and this reduces the recognition accuracy. To solve these problems, we propose a driver sleepiness estimation model based on factorized bilinear feature fusion and a long- short-term recurrent convolutional network to detect driver drowsiness efficiently and accurately. The proposed framework includes three models: fatigue feature extraction, fatigue feature fusion, and driver drowsiness detection. First, we used a convolutional neural network (CNN) to effectively extract the deep representation of eye and mouth-related fatigue features from the face area detected in each video frame. Then, based on the factorized bilinear feature fusion model, we performed a nonlinear fusion of the deep feature representations of the eyes and mouth. Finally, we input a series of fused frame-level features into a long-short-term memory (LSTM) unit to obtain the time information of the features and used the softmax classifier to detect sleepiness. The proposed framework was evaluated with the National Tsing Hua University drowsy driver detection (NTHU-DDD) video dataset. The experimental results showed that this method had better stability and robustness compared with other methods.

show abstract

Section: Related Workmentioning

confidence: 99%

Driver Drowsiness Estimation Based on Factorized Bilinear Feature Fusion and a Long-Short-Term Recurrent Convolutional Network

2020

View full text Add to dashboard Cite

show abstract

“…To learn discriminative features which cover both the complexity and the diversity of human actions, we attempted a joint learning method for motion and appearance based on the convolutional neural network. Fusing of multiple data is a commonly used approach in visual recognition studies for scene segmentation [56], [57], object detection [58], and event detection [59]- [61]. In recent action recognition studies [50]- [53], several have reported that using multiple information can provide better action recognition performance than using a single data type only.…”

Section: A Joint Spatio-temporal Representation Extractionmentioning

confidence: 99%

“…α, C andā are the extracted joint ST-representation, the number of action classes in the training dataset and the given label corresponding to an input video, respectively. This combining loss function is a commonly used approach in various visual recognition studies [61], [68], [69]. According to [68], the center loss, which simultaneously learns the center for the deep features of each class and penalizes the distances between the deep features and their corresponding class centers, showed better performance than conventional loss functions such as softmax loss.…”

Section: Outputmentioning

confidence: 99%

Spatio-Temporal Representation Matching-Based Open-Set Action Recognition by Joint Learning of Motion and Appearance

Yoon

Jeon

2019

IEEE Access

Self Cite

View full text Add to dashboard Cite

In this paper, we propose the spatio-temporal representation matching (STRM) for video-based action recognition under the open-set condition. Open-set action recognition is a more challenging problem than closed-set action recognition since samples of the untrained action class need to be recognized and most of the conventional frameworks are likely to give a false prediction. To handle the untrained action classes, we propose STRM, which involves jointly learning both motion and appearance. STRM extracts spatio-temporal representations from video clips through a joint learning pipeline with both motion and appearance information. Then, STRM computes the similarities between the ST-representations to find the one with highest similarity. We set the experimental protocol for open-set action recognition and carried out experiments on UCF101 and HMDB51 to evaluate STRM. We first investigated the effects of different hyperparameter settings on STRM, and then compared its performance with existing state-of-the-art methods. The experimental results showed that the proposed method not only outperformed existing methods under the open-set condition, but also provided comparable performance to the state-of-the-art methods under the closed-set condition.

show abstract

“…This method showed a classification accuracy of 92.33%. Likewise [12], introduces an adaptive conditional representation learning system for driver drowsiness detection based on a 3D-CNN. The proposed system consists of four steps (spatio-temporal representation, data preprocessing, features combination and somnolence detection).…”

Section: Introductionmentioning

confidence: 99%

EEG-Based Hypo-vigilance Detection Using Convolutional Neural Network

Boudaya

Bouaziz

Chaabene

et al. 2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Hypo-vigilance detection is becoming an important active research areas in the biomedical signal processing field. For this purpose, electroencephalogram (EEG) is one of the most common modalities in drowsiness and awakeness detection. In this context, we propose a new EEG classification method for detecting fatigue state. Our method makes use of a and awakeness detection. In this context, we propose a new EEG classification method for detecting fatigue state. Our method makes use of a Convolutional Neural Network (CNN) architecture. We define an experimental protocol using the Emotiv EPOC+ headset. After that, we evaluate our proposed method on a recorded and annotated dataset. The reported results demonstrate high detection accuracy (93%) and indicate that the proposed method is an efficient alternative for hypo-vigilance detection as compared with other methods.

show abstract

Driver Drowsiness Detection Using Condition-Adaptive Representation Learning Framework

Cited by 92 publications

References 53 publications

Driver Drowsiness Estimation Based on Factorized Bilinear Feature Fusion and a Long-Short-Term Recurrent Convolutional Network

Driver Drowsiness Estimation Based on Factorized Bilinear Feature Fusion and a Long-Short-Term Recurrent Convolutional Network

Spatio-Temporal Representation Matching-Based Open-Set Action Recognition by Joint Learning of Motion and Appearance

EEG-Based Hypo-vigilance Detection Using Convolutional Neural Network

Contact Info

Product

Resources

About