Predicting the Driver's Focus of Attention: The DR(eye)VE Project

Palazzi, Andrea; Abati, Davide; Calderara, Simone; Solera, Francesco; Cucchiara, Rita

doi:10.1109/tpami.2018.2845370

Cited by 220 publications

(228 citation statements)

References 73 publications

(105 reference statements)

Supporting

Mentioning

227

Contrasting

Order By: Relevance

“…1) Type1: We recorded the common attention maps, where an attention map contains five observers' fixations without temporal aggregation. Note that, different from the works [8], [1], we do not average the attention fixations of observers and maintain them in the same frame because of their subjectivity. We can observe that the fixations get close to the appeared crash-object, and vice versa for normal driving scenes.…”

Section: B Attention Collection 1) Protocolsmentioning

confidence: 99%

“…Benefit from the progress of saliency computation models, driver attention that directly links the driving task and eye fixations is focused, but concentrates on the designated objects [9], [10] for a long time. In order to mimic the real driver attention mechanism, the DR(eye)VE project [1] was launched, and on this basis, several models based on deep neural networks [6], [11], [7] were built for driver attention prediction. However, as aforementioned, there are some issues in this dataset.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

DADA-2000: Can Driving Accident be Predicted by Driver Attentionƒ Analyzed by A Benchmark

Fang

Yan

Qiao

et al. 2019

2019 IEEE Intelligent Transportation Systems Conference (ITSC)

View full text Add to dashboard Cite

Driver attention prediction is currently becoming the focus in safe driving research community, such as the DR(eye)VE project and newly emerged Berkeley DeepDrive Attention (BDD-A) database in critical situations. In safe driving, an essential task is to predict the incoming accidents as early as possible. BDD-A was aware of this problem and collected the driver attention in laboratory because of the rarity of such scenes. Nevertheless, BDD-A focuses the critical situations which do not encounter actual accidents, and just faces the driver attention prediction task, without a close step for accident prediction. In contrast to this, we explore the view of drivers' eyes for capturing multiple kinds of accidents, and construct a more diverse and larger video benchmark than ever before with the driver attention and the driving accident annotation simultaneously (named as DADA-2000), which has 2000 video clips owning about 658, 476 frames on 54 kinds of accidents. These clips are crowd-sourced and captured in various occasions (highway, urban, rural, and tunnel), weather (sunny, rainy and snowy) and light conditions (daytime and nighttime). For the driver attention representation, we collect the maps of fixations, saccade scan path and focusing time. The accidents are annotated by their categories, the accident window in clips and spatial locations of the crash-objects. Based on the analysis, we obtain a quantitative and positive answer for the question in this paper.

show abstract

Section: B Attention Collection 1) Protocolsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

DADA-2000: Can Driving Accident be Predicted by Driver Attentionƒ Analyzed by A Benchmark

Fang

Yan

Qiao

et al. 2019

2019 IEEE Intelligent Transportation Systems Conference (ITSC)

View full text Add to dashboard Cite

show abstract

“…A very related work [10] proposed a multi-branch deep neural network to predict eye gaze in urban driving scenarios. It attached much importance to gaze data analysis over different driving scenes and driving conditions.…”

Section: B Eye Gaze In Drivingmentioning

confidence: 99%

Gaze Training by Modulated Dropout Improves Imitation Learning

Chen

Liu

Tai

et al. 2019

2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

Imitation learning by behavioral cloning is a prevalent method that has achieved some success in vision-based autonomous driving. The basic idea behind behavioral cloning is to have the neural network learn from observing a human expert's behavior. Typically, a convolutional neural network learns to predict the steering commands from raw driver-view images by mimicking the behaviors of human drivers. However, there are other cues, such as gaze behavior, available from human drivers that have yet to be exploited. Previous researches have shown that novice human learners can benefit from observing experts' gaze patterns. We present here that deep neural networks can also profit from this. We propose a method, gaze-modulated dropout, for integrating this gaze information into a deep driving network implicitly rather than as an additional input. Our experimental results demonstrate that gaze-modulated dropout enhances the generalization capability of the network to unseen scenes. Prediction error in steering commands is reduced by 23.5% compared to uniform dropout. Running closed loop in the simulator, the gaze-modulated dropout net increased the average distance travelled between infractions by 58.5%. Consistent with these results, the gazemodulated dropout net shows lower model uncertainty.

show abstract

“…Our approach seeks to combine these approaches and use the representation learning power of deep networks to extract task-relevant visual features, given task-driven gaze data. A recent work that also takes this approach is [30], where they predict human gaze while driving from raw images using a multi-channel deep network.…”

Section: Modeling Visual Attention: Bottom-up Vs Top-downmentioning

confidence: 99%

AGIL: Learning Attention from Human for Visuomotor Tasks

Zhang

Liu

Zhang

et al. 2018

Computer Vision – ECCV 2018

View full text Add to dashboard Cite

When intelligent agents learn visuomotor behaviors from human demonstrations, they may benefit from knowing where the human is allocating visual attention, which can be inferred from their gaze. A wealth of information regarding intelligent decision making is conveyed by human gaze allocation; hence, exploiting such information has the potential to improve the agents' performance. With this motivation, we propose the AGIL (Attention Guided Imitation Learning) framework. We collect high-quality human action and gaze data while playing Atari games in a carefully controlled experimental setting. Using these data, we first train a deep neural network that can predict human gaze positions and visual attention with high accuracy (the gaze network) and then train another network to predict human actions (the policy network). Incorporating the learned attention model from the gaze network into the policy network significantly improves the action prediction accuracy and task performance.

show abstract

Predicting the Driver's Focus of Attention: The DR(eye)VE Project

Cited by 220 publications

References 73 publications

DADA-2000: Can Driving Accident be Predicted by Driver Attentionƒ Analyzed by A Benchmark

DADA-2000: Can Driving Accident be Predicted by Driver Attentionƒ Analyzed by A Benchmark

Gaze Training by Modulated Dropout Improves Imitation Learning

AGIL: Learning Attention from Human for Visuomotor Tasks

Contact Info

Product

Resources

About