Driver Eye Location and State Estimation Based on a Robust Model and Data Augmentation

Abstract

Eye state evaluation is crucial for vision-based driver fatigue detection. With the outbreak of COVID-19, many proposed models for eye location and state evaluation based on facial landmarks are unreliable due to mask coverings. In this paper, we proposed a robust facial landmark location model for eye location and state evaluation. First, we develop an existing lightweight face alignment model for eye key point locations that is robust in large poses. Then, to develop the performance of our model in a complex driving environment such as an environment with mask coverings, changing illumination, etc., we design a method to augment the training data set based on the original landmark data set without any extra cost. Finally, some facial landmarks around the eyes are extracted, and the eye aspect ratio (EAR) is introduced to evaluate the eye state based on eye key points. The experiment shows that our model achieves significantly improved landmark location performance on a driving simulation data set due to data augmentation. We tested our model on the BioID data set to measure the eye state evaluation performance, and the results showed that our model obtained satisfactory performance with an accuracy of approximately 97.7%. Further testing on the driving simulation data set shows that our model is robust in different driving scenarios with an average accuracy of approximately 93.9%.

Computer Vision-based Driver Fatigue Detection Framework with Personalization Threshold and Multi-feature Fusion

“…These features will be compared with pre-set fatigue thresholds to get the state of eyes and mouth. This threshold is mostly calculated using the average method currently [7,15], however, it is unreasonable to use the same fatigue threshold for different drivers to judge the state, because drivers' eyes and mouths have different sizes [16,17]. Li et al [15] and You et al [18] trained a classification library for each driver to alleviate the disadvantages caused by the above differences.…”

“…This work has yielded the current state-of-the-art results. Ling et al [7] proposed a robust facial landmark location model for eye location and state evaluation. However, in practice, only using few features will reduce the universality of the fatigue driving detection methods and affect their accuracy.…”

“…According to the data source, the existing methods for driver fatigue detecting can be divided into three categories: (1) driver's physiological characteristics-based methods [3,4]; (2) vehicle driving information-based methods [5,6] and (3) driver's facial expressions-based methods [7,8]. The driver's physiological characteristics-based method belongs to intrusive method which is not friendly enough to the driver.…”

Adjusting eye aspect ratio for strong eye blink detection based on facial landmarks

“…Left eye:(37, 38, 39, 40, 41, 42), (2). Right eye: (43, 44, 45, 46, 47, 48) ( Ling et al, 2021 ; Tang et al, 2018 ). After extracting the eye region, it is processed for detecting eye blinks.…”