Face salient points and eyes tracking for robust drowsiness detection

Abstract

Link to this article: http://journals.cambridge.org/abstract_S0263574711000749How to cite this article: J. Jimenez-Pinto and M. Torres-Torriti (2012). Face salient points and eyes tracking for robust drowsiness detection. SUMMARYMeasuring a driver's level of attention and drowsiness is fundamental to reducing the number of traffic accidents that often involve bus and truck drivers, who must work for long periods of time under monotonous road conditions. Determining a driver's state of alert in a noninvasive way can be achieved using computer vision techniques. However, two main difficulties must be solved in order to measure drowsiness in a robust way: first, detecting the driver's face location despite variations in pose or illumination; secondly, recognizing the driver's facial cues, such as blinks, yawns, and eyebrow rising. To overcome these challenges, our approach combines the well-known Viola-Jones face detector with the motion analysis of Shi-Tomasi salient features within the face. The location of the eyes and blinking is important to refine the tracking of the driver's head and compute the so-called PERCLOS, which is the percentage of time the eyes are closed over a given time interval. The latter cue is essential for noninvasive driver's alert state estimation as it has a high correlation with drowsiness. To further improve the location of the eyes under different conditions of illumination, the proposed method takes advantage of the high reflectivity of the retina to near infrared illumination employing a camera with an 850 nm wavelength filter. The paper shows that motion analysis of the salient points, in particular cluster mass centers and spatial distributions, yields better head tracking results compared to the state-ofthe-art and provides measures of the driver's alert state.

Video-based observer rated sleepiness versus self-reported subjective sleepiness in real road driving

“…Much research is currently devoted to developing automatic image processing systems capable of determining the level of sleepiness based on characteristics such as facial tone, slow eyelid closure, rubbing, yawning and nodding [35][36][37][38][39]. According to Vural et al [40], the ten facial actions that are most predictive of sleepiness are increased blink/eye closure, elevated outer brow raise, increased frown, chin raise, more nose wrinkle, less smiling, tightened eye lid, less compressed nostrils, less lowering of eye brows and less jaw drop.…”

A Review on Driver Face Monitoring Systems for Fatigue and Distraction Detection

“…Due to the simple extracted features and selection of the best features, this algorithm is relatively fast and robust. This algorithm was used in [7,26,27,28,29,44,51,59,74] for face detection.…”

“…In some recent researches such as [73,74,75], the salient points of face are detected after face detection. In these researches, the salient points are tracked over time.…”

Driver Distraction Using Visual-Based Sensors and Algorithms

“…The use of face salient points to track the head was introduced by Jimenez et al [132], instead of attempting to directly find the eyes using object recognition methods or the analysis of image intensities around the eyes. The camera was modified to include an 850 nm band-pass filter lens covering both the image sensor and the IR LEDs in order: (a) to improve the rejection of external sources of IR radiation and reduce changes in illumination and (b) to facilitate the detection of the pupils, because the retina is highly reflective of the NIR illumination of the LEDs.…”

“…They used a kinematic model of the driver’s motion and a grid of salient points tracked using the Lukas-Kanade optical flow method [132]. The advantage of this approach is that it does not require one to directly detect the eyes, and therefore, if the eyes are occluded or not visible from the camera when the head turns, the system does not loose the tracking of the eyes or the face, because it relies on the grid of salient points and the knowledge of the driver’s motion model.…”