Multi-sensor driver drowsiness monitoring

Abstract

A system for driver drowsiness monitoring is proposed, using multi-sensor data acquisition and investigating two decision-making algorithms, namely a fuzzy inference system (FIS) and an artificial neural network (ANN), to predict the drowsiness level of the driver. Drowsiness indicator signals are selected allowing non-intrusive measurements. The experimental set-up of a driver-drowsiness-monitoring system is designed on the basis of the soughtafter indicator signals. These selected signals are the eye closure via pupil area measurement, gaze vector and head motion acquired by a monocular computer vision system, steering wheel angle, vehicle speed, and force applied to the steering wheel by the driver. It is believed that, by fusing these signals, driver drowsiness can be detected and drowsiness level can be predicted. For validation of this hypothesis, 30 subjects, in normal and sleep-deprived conditions, are involved in a standard highway simulation for 1.5 h, giving a data set of 30 pairs. For designing a feature space to be used in decision making, several metrics are derived using histograms and entropies of the signals. An FIS and an ANN are used for decision making on the drowsiness level. To construct the rule base of the FIS, two different methods are employed and compared in terms of performance: first, linguistic rules from experimental studies in literature and, second, mathematically extracted rules by fuzzy subtractive clustering. The drowsiness levels belonging to each session are determined by the participants before and after the experiment, and videos of their faces are assessed to obtain the ground truth output for training the systems. The FIS is able to predict correctly 98 per cent of determined drowsiness states (training set) and 89 per cent of previously unknown test set states, while the ANN has a correct classification rate of 90 per cent for the test data. No significant difference is observed between the FIS and the ANN; however, the FIS might be considered better since the rule base can be improved on the basis of new observations.

Video-based observer rated sleepiness versus self-reported subjective sleepiness in real road driving

“…Much research is currently devoted to developing automatic image processing systems capable of determining the level of sleepiness based on characteristics such as facial tone, slow eyelid closure, rubbing, yawning and nodding [35][36][37][38][39]. According to Vural et al [40], the ten facial actions that are most predictive of sleepiness are increased blink/eye closure, elevated outer brow raise, increased frown, chin raise, more nose wrinkle, less smiling, tightened eye lid, less compressed nostrils, less lowering of eye brows and less jaw drop.…”

A review of fuzzy methods in automotive engineering applications

“…-Identifications and classifications of the drivers regarding fatigue, emotions and other human attributes, including the procedures of driver state recognition and forecasting through monitoring of various physiological parameters like electroencephalography-estimated brain activity, eye movement, gestures et al [10][11][12][13][14][15], -Structure and controllers of driver assistance systems and devices of human machine interface [16][17][18], -Models of driver actions on vehicle controlling devices (brake and throttle pedals, steering wheel) for authentic simulation of vehicle maneuvers on driving simulators; controllers of pedal and steering wheel robots [19][20][21][22], -Simulation of driver reasoning for controllers of (semi-) automated and unmanned ground vehicles [23][24][25], -Understanding of subjective evaluation of vehicle dynamics; driver feeling of vehicle dynamics parameters like velocity, road friction and other [26][27][28][29], -Advisory functions of human-machine interface systems supporting the driver in eco-and low-emission vehicle operation [30-32].…”

Fusion of Optimized Indicators from Advanced Driver Assistance Systems (ADAS) for Driver Drowsiness Detection

“…Then, this is the fusion technique with the best performance in driver fatigue studies, according to the state-of-the-art. For instance, in some of the previous works [ 22 , 44 , 46 , 47 ], this technique retrieves recall rates between 55% and 90%, depending on the inputs. Optimal combinations of a variety of drowsiness indicators proposed in the literature were evaluated by using a conveniently trained ANN.…”