Driver sleepiness detection from EEG and EOG signals using GAN and LSTM networks

Jiao, Yingying; Deng, Yini; Luo, Yaping; Lu, Bao-Liang

doi:10.1016/j.neucom.2019.05.108

Cited by 110 publications

(51 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Their research was based on only 16 participants. The second one also used LSTM, but for prediction of the underlying alpha phenomena that is the base for determining drowsiness level [ 175 ]. The other three papers used CNN as a classification method.…”

Section: Discussionmentioning

confidence: 99%

“…Acquiring the data is often a problem when it comes to EEG-based drowsiness detection. Authors of research studies that use deep learning approaches often employ generative adversarial networks for the augmentation of the dataset [ 175 ]. This process often leads to an improved performance of the model.…”

Section: Discussionmentioning

confidence: 99%

“…Yingying et al [ 175 ] used a Long Short-Term Memory (LSTM) network to classify sleepiness in two classes and their final classification accuracy achieved was 98%. Their ground truth labels for classification were based on the alpha-blocking phenomenon and the alpha wave attenuation-disappearance phenomenon.…”

Section: Driver Drowsiness Detection Systemsmentioning

confidence: 99%

See 2 more Smart Citations

A Review of EEG Signal Features and Their Application in Driver Drowsiness Detection Systems

Stančin¹,

Cifrek²,

Jović³

2021

Sensors

126

View full text Add to dashboard Cite

Detecting drowsiness in drivers, especially multi-level drowsiness, is a difficult problem that is often approached using neurophysiological signals as the basis for building a reliable system. In this context, electroencephalogram (EEG) signals are the most important source of data to achieve successful detection. In this paper, we first review EEG signal features used in the literature for a variety of tasks, then we focus on reviewing the applications of EEG features and deep learning approaches in driver drowsiness detection, and finally we discuss the open challenges and opportunities in improving driver drowsiness detection based on EEG. We show that the number of studies on driver drowsiness detection systems has increased in recent years and that future systems need to consider the wide variety of EEG signal features and deep learning approaches to increase the accuracy of detection.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

A Review of EEG Signal Features and Their Application in Driver Drowsiness Detection Systems

Stančin¹,

Cifrek²,

Jović³

2021

Sensors

126

View full text Add to dashboard Cite

show abstract

“…GAN is also used by Luo and Lu for EEG data augmentation [ 152 ]. You et al [ 153 ] and Jiao et al [ 154 ] utilized GAN-based model for detecting seizure using EEG signal and Driver sleepiness using EEG and Electrooculography (EOG) signals, respectively. Singh et al proposed a new GAN framework for denoising ECG [ 155 ].…”

Section: Deep Learning and Biological Datamentioning

confidence: 99%

Deep Learning in Mining Biological Data

et al. 2021

View full text Add to dashboard Cite

Recent technological advancements in data acquisition tools allowed life scientists to acquire multimodal data from different biological application domains. Categorized in three broad types (i.e. images, signals, and sequences), these data are huge in amount and complex in nature. Mining such enormous amount of data for pattern recognition is a big challenge and requires sophisticated data-intensive machine learning techniques. Artificial neural network-based learning systems are well known for their pattern recognition capabilities, and lately their deep architectures—known as deep learning (DL)—have been successfully applied to solve many complex pattern recognition problems. To investigate how DL—especially its different architectures—has contributed and been utilized in the mining of biological data pertaining to those three types, a meta-analysis has been performed and the resulting resources have been critically analysed. Focusing on the use of DL to analyse patterns in data from diverse biological domains, this work investigates different DL architectures’ applications to these data. This is followed by an exploration of available open access data sources pertaining to the three data types along with popular open-source DL tools applicable to these data. Also, comparative investigations of these tools from qualitative, quantitative, and benchmarking perspectives are provided. Finally, some open research challenges in using DL to mine biological data are outlined and a number of possible future perspectives are put forward.

show abstract

“…GAN, AE ve ESA gibi güncel yapay öğrenme tekniklerinin birlikte kullanıldığı bu mimarilerle basit görsellere bakıldığında üretimin yapılabildiği, ancak karmaşık görsellerde üretim kalitesinin oldukça düştüğü görülmektedir. EEG sinyallerinin sınıflandırılması [9], EEG sinyallerinden yeni EEG sinyalleri üretme [10], [11] ve EEG/EOG sinyallerinden sürücünün uyku hali tespiti [12] gibi çalışmalar EEG-GAN'lar yapılan diğer çalışmalardır. EEG-GAN'lar, sadece veri artırma ile sınırlı kalmamakla birlikte, bozuk sinyallerin onarımı gibi yeni uygulama alanları yaratmaktadır [13].…”

Section: Introductionunclassified

Generating the image viewed from EEG signals

Çelik¹,

Talu²

2021

Pamukkale J Eng Sci

View full text Add to dashboard Cite

EEG sinyalleri kullanılarak engelliler için kontrol edilebilir tekerlekli sandalyelerin üretildiği veya yapılması düşünülen aktivitenin tahmin edildiği çalışmalara literatürde sıklıkla rastlanmaktadır. Genel olarak bu çalışmalarda elektroensefalografi (EEG) sinyalinin önceden belirlenen sınıflara aktarımı gerçekleştirilir. Bu çalışmalar EEG sinyalinin sınıflandırmasından ibarettir. Ancak son yıllarda yapay öğrenme alanında yaşanan gelişmelerle sınıflandırmadan öteye gidildiği, EEG sinyalinden bakılan görselin üretilebildiği görülmektedir. Klasik çekişmeli üretici ağlar (Generative adversarial networks-GAN) ve otomatik kodlayıcı (Auto encoder-AE) yaklaşımlarının kullanıldığı sınırlı sayıdaki bu çalışmalar incelendiğinde, EEG sinyallerinden kabaca görsellerin üretilebildiği görülmektedir. Bu çalışmanın özgün yönü, görsel üretim kabiliyetini arttıracak matematiksel yaklaşımlar içermesidir. Klasik GAN mimarileri üretilen görüntülerin çeşitliliğini sağlayabilmek için rastgele vektör girişini kullanırlar. Bu yaklaşım ile EEG sinyalinden üretilen görsellerin düşük kalitede olduğu gözlemlenmiştir. Önerilen yöntemde giriş iki kısım (kodlanmış EEG ve rastgelelik) olarak düşünülmüştür. EEG'nin kodlanması için değişken oto kodlayıcı (Variational auto encoder-VAE) ve fourier dönüşümü (FD) kullanılırken, rastgelelik için iki farklı yaklaşım önerilmiştir. Bu özgün GAN kullanımı, EEG sinyallerinden daha kaliteli görsel üretilmesini sağlamıştır. Bu kalitenin sayısal olarak anlaşılabilmesi için önceden eğitilmiş evrişimsel sinir ağları (ESA) kullanılmıştır. Yapılan deneysel çalışmalar neticesinde, klasik GAN ile EEG'den üretilen görsellerin başarım seviyesi %93 civarındayken, önerilen yaklaşımda bu seviyenin %95-%100 aralığına çıktığı görülmektedir.In the literature, it is encountered a vast amount of studies related to the production of controllable wheelchairs for people with disabilities or the prediction of activity thought to be performed. In general, the electroencephalography (EEG) signal is transferred to predetermined classes in these studies. These studies consist of the classification of the EEG signal. However, it has been observed that in the recent years, with the developments in the field of artificial learning, the classification has gone beyond, It can be seen that the visual viewed from the EEG signal can be produced. When the limited number of studies using classical generative adversarial networks (GAN) and autu encoder (AE) approaches are examined, it is seen that visuals from EEG signals can be produced roughly. The original aspect of this study is that it includes mathematical approaches to increase the visual production capability. Classical GAN architectures use random vector input to provide a variety of images produced. With this approach, it is observed that the visuals produced from the EEG signal are of low quality. In the proposed method, the input is considered as two parts (coded EEG and randomness). Variable auto encoder (VAE) and fourier transform (FT) are used to encode the EEG, while two differ...

show abstract

Driver sleepiness detection from EEG and EOG signals using GAN and LSTM networks

Cited by 110 publications

References 34 publications

A Review of EEG Signal Features and Their Application in Driver Drowsiness Detection Systems

A Review of EEG Signal Features and Their Application in Driver Drowsiness Detection Systems

Deep Learning in Mining Biological Data

Generating the image viewed from EEG signals

Contact Info

Product

Resources

About