Learning where to attend like a human driver

Abstract

Despite the advent of autonomous cars, it's likelyat least in the near future -that human attention will still maintain a central role as a guarantee in terms of legal responsibility during the driving task. In this paper we study the dynamics of the driver's gaze and use it as a proxy to understand related attentional mechanisms. First, we build our analysis upon two questions: where and what the driver is looking at? Second, we model the driver's gaze by training a coarse-tofine convolutional network on short sequences extracted from the DR(eye)VE dataset. Experimental comparison against different baselines reveal that the driver's gaze can indeed be learnt to some extent, despite i) being highly subjective and ii) having only one driver's gaze available for each sequence due to the irreproducibility of the scene. Eventually, we advocate for a new assisted driving paradigm which suggests to the driver, with no intervention, where she should focus her attention.

Dynamics of Driver's Gaze: Explorations in Behavior Modeling and Maneuver Prediction

“…This may happen with the driver in full control and getting active assistance from the robot, or the robot is in partial or full control and human drivers are passive observers "ready" to take over as deemed necessary by the machine or human [3], [4]. In the full spectrum from manual to autonomous mode, modeling the dynamics of driver's gaze is of particular interest because, if and how the driver is monitoring the driving environment is vital for driver assistance in manual mode [5], for takeover requests in highly automated mode [6] and for semantic perception of the surround in fully autonomous mode [7], [8].…”

Predicting Driver Attention in Critical Situations

“…Human driver's attention provides important visual cues for driving, and thus efforts to mimic human driver's attention have increasingly been introduced. Recently, several deep neural models have been utilized to predict where human drivers should pay attention [21,25]. Most of existing models were trained and tested on the DR(eye)VE dataset [1].…”

“…To our knowledge, [21] and [25] are the two deep neural models that use dash camera videos alone to predict human driver's gaze. They demonstrated similar results and were shown to surpass other deep learning models or traditional models that predict human gaze in non-driving-specific contexts.…”

“…Our model correctly attends to the crossing pedestrians and does not give false alarms to other irrelevant pedestrians attending to another object [6]. (ii) False positive gazes: human drivers also show eye movements to driving-irrelevant regions, such as sky, trees, and buildings [21]. It is challenging to separate these false positives from gazes that are dedicated to driving.…”

“…Drivers' gaze behavior has been studied as a proxy for their attention. Recently, a large driver attention dataset of routine driving [1] has been introduced and neural networks [21,25] have been trained end-to-end to estimate driver attention, mostly in lane-following and car-following situations. Nonetheless, datasets and prediction models for driver attention in rare and critical situations are still needed.…”