Deep Learning-Based Gaze Detection System for Automobile Drivers Using a NIR Camera Sensor

Naqvi, Rizwan Ali; Arsalan, Muhammad; Batchuluun, Ganbayar; Yoon, Hyo Sik; Park, Kang Ryoung

doi:10.3390/s18020456

Cited by 128 publications

(49 citation statements)

References 56 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The device or apparatus used to track gaze by analyzing eye movements is called a gaze tracker. A gaze tracker performs two main tasks simultaneously: localization of the eye position in the video or images, and tracking its motion to determine the gaze direction [15,16]. A generic representation of such techniques is shown in Figure 1.…”

Section: Background and Motivationmentioning

confidence: 99%

Gaze and Eye Tracking: Techniques and Applications in ADAS

Khan

Lee

2019

Sensors

View full text Add to dashboard Cite

Tracking drivers’ eyes and gazes is a topic of great interest in the research of advanced driving assistance systems (ADAS). It is especially a matter of serious discussion among the road safety researchers’ community, as visual distraction is considered among the major causes of road accidents. In this paper, techniques for eye and gaze tracking are first comprehensively reviewed while discussing their major categories. The advantages and limitations of each category are explained with respect to their requirements and practical uses. In another section of the paper, the applications of eyes and gaze tracking systems in ADAS are discussed. The process of acquisition of driver’s eyes and gaze data and the algorithms used to process this data are explained. It is explained how the data related to a driver’s eyes and gaze can be used in ADAS to reduce the losses associated with road accidents occurring due to visual distraction of the driver. A discussion on the required features of current and future eye and gaze trackers is also presented.

show abstract

Section: Background and Motivationmentioning

confidence: 99%

Gaze and Eye Tracking: Techniques and Applications in ADAS

Khan

Lee

2019

Sensors

View full text Add to dashboard Cite

show abstract

“…from a face image. It plays an important role in popular applications such as face recognition [1,2], attribute computing [3,4], and expression recognition [5]. Face alignment technology is usually applied to get some anchor points for affine warping so that the face recognition procedure is robust against pose variations.…”

Section: Introductionmentioning

confidence: 99%

“…In [3], facial landmarks were used to outline a fetus's face. In [4], facial landmarks helped to localize and represent salient regions of the face. Figure 1 shows an example of a face alignment algorithm with the supervised descent method (SDM) [6].…”

Section: Introductionmentioning

confidence: 99%

Tightly-Coupled Data Compression for Efficient Face Alignment

et al. 2018

View full text Add to dashboard Cite

Face alignment is the key component for applications such as face and expression recognition, face based AR (Augmented Reality), etc. Among all the algorithms, cascaded-regression based methods have become popular in recent years for their low computational costs and satisfactory performances in uncontrolled environments. However, the size of the trained model is large for cascaded-regression based methods, which makes it difficult to be applied in resource restricted scenarios such as applications on mobile phones. In this paper, a data compression method for the trained model of supervised descent method (SDM) is proposed. Firstly, according to the distribution of the model data estimated with the non-parametric method, a K-means based data quantization algorithm with probability density-aware initialization was proposed to efficiently quantize the model data. Then, a tightly-coupled SDM training algorithm was proposed so that the training process reduced the errors caused by data quantization. Quantitative experimental results proved that our proposed method compressed the trained model to less than 19% of its original size with very similar feature localization performance. The proposed method opens the gates to efficient mobile face alignment applications based on SDM.

show abstract

“…Description of training and testing images from DDBC-DB1 [62]. was used for the implementation of training and testing algorithm of the CNN model.…”

mentioning

confidence: 99%

“…Description of training and testing images from DDBC-DB1. )[62] was used for the implementation of training and testing algorithm of the CNN model. For the training and testing of CNN, we used a desktop computer with an Intel ® Core™(Santa Clara, CA, USA) i7-3770K CPU @ 3.50 GHz, 16 GB memory, and a NVIDIA GeForce GTX 1070 (1920 CUDA cores and 8 GB memory) graphics card[63].…”

mentioning

confidence: 99%

Deep Learning-Based Drivers Emotion Classification System in Time Series Data for Remote Applications

et al. 2020

Self Cite

View full text Add to dashboard Cite

Aggressive driving emotions is indeed one of the major causes for traffic accidents throughout the world. Real-time classification in time series data of abnormal and normal driving is a keystone to avoiding road accidents. Existing work on driving behaviors in time series data have some limitations and discomforts for the users that need to be addressed. We proposed a multimodal based method to remotely detect driver aggressiveness in order to deal these issues. The proposed method is based on change in gaze and facial emotions of drivers while driving using near-infrared (NIR) camera sensors and an illuminator installed in vehicle. Driver's aggressive and normal time series data are collected while playing car racing and truck driving computer games, respectively, while using driving game simulator. Dlib program is used to obtain driver's image data to extract face, left and right eye images for finding change in gaze based on convolutional neural network (CNN). Similarly, facial emotions that are based on CNN are also obtained through lips, left and right eye images extracted from Dlib program. Finally, the score level fusion is applied to scores that were obtained from change in gaze and facial emotions to classify aggressive and normal driving. The proposed method accuracy is measured through experiments while using a self-constructed large-scale testing database that shows the classification accuracy of the driver's change in gaze and facial emotions for aggressive and normal driving is high, and the performance is superior to that of previous methods.Association Foundation for Traffic safety, published in 2009, that the aggressive behavior of driver causes 56% of traffic accidents [2]. Besides precious human lives, people, company, and government also lose billions of dollars due to road accidents. For this reason, aggressive driving behavior must be strongly discouraged that will result in reduction of the number of traffic accidents.The classification of aggressive and normal behavior is an important issue that can be used to increase awareness of driving habits of drivers as many drivers are over confident and are unaware of their bad driving habits [3]. If we can automatically identify the drivers driving behaviors, the drivers can be aware of their bad habits and assist them to avoid potential car accidents. Other than this if, monitoring results could be sent back to a security observing server of the local police station that could help to automatically detect aggressive drivers. The conventional method to keep a check on aggressive driving is by police patrolling, but, due to lack of police force, all roads cannot be simultaneously monitored and it also costs a lot [4]. The need of intelligent surveillance system is increasing with the increase in population. The advance driver assistance system (ADAS) that can monitor driver's attention and driving behavior can improve road safety, which will also enhance the effectiveness of the ADAS [5]. Many challenges are faced by these real time systems that ...

show abstract

Deep Learning-Based Gaze Detection System for Automobile Drivers Using a NIR Camera Sensor

Cited by 128 publications

References 56 publications

Gaze and Eye Tracking: Techniques and Applications in ADAS

Gaze and Eye Tracking: Techniques and Applications in ADAS

Tightly-Coupled Data Compression for Efficient Face Alignment

Deep Learning-Based Drivers Emotion Classification System in Time Series Data for Remote Applications

Contact Info

Product

Resources

About