DR(eye)VE: A Dataset for Attention-Based Tasks with Applications to Autonomous and Assisted Driving

Alletto, Stefano; Palazzi, Andrea; Solera, Francesco; Calderara, Simone; Cucchiara, Rita

doi:10.1109/cvprw.2016.14

Cited by 96 publications

(78 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The fourth dataset has data from DR(eye)VE [25], which consists of images from real life driving. The data is comprised of 74 videos with sequences of five minutes each with annotation of drivers' gaze fixations.…”

Section: Evaluation Methodsmentioning

confidence: 99%

Towards Structured Evaluation of Deep Neural Network Supervisors

Henriksson

Berger

Borg

et al. 2019

2019 IEEE International Conference on Artificial Intelligence Testing (AITest)

View full text Add to dashboard Cite

Deep Neural Networks (DNN) have improved the quality of several non-safety related products in the past years. However, before DNNs should be deployed to safety-critical applications, their robustness needs to be systematically analyzed. A common challenge for DNNs occurs when input is dissimilar to the training set, which might lead to high confidence predictions despite proper knowledge of the input.Several previous studies have proposed to complement DNNs with a supervisor that detects when inputs are outside the scope of the network. Most of these supervisors, however, are developed and tested for a selected scenario using a specific performance metric. In this work, we emphasize the need to assess and compare the performance of supervisors in a structured way. We present a framework constituted by four datasets organized in six test cases combined with seven evaluation metrics.The test cases provide varying complexity and include data from publicly available sources as well as a novel dataset consisting of images from simulated driving scenarios. The latter we plan to make publicly available. Our framework can be used to support DNN supervisor evaluation, which in turn could be used to motive development, validation, and deployment of DNNs in safety-critical applications.

show abstract

Section: Evaluation Methodsmentioning

confidence: 99%

Towards Structured Evaluation of Deep Neural Network Supervisors

Henriksson

Berger

Borg

et al. 2019

2019 IEEE International Conference on Artificial Intelligence Testing (AITest)

View full text Add to dashboard Cite

show abstract

“…There are several datasets [20], [24], [7], [18], [1] can be used for driver's attention prediction, but most of them are either restricted to limited settings or not publicly available. To the best of our knowledge, Dr(eye)ve [1] is the only public on-road driving dataset for the driver's attention prediction task. It consists of 555,000 frames divided into 74 video sequences.…”

Section: A Driver's Attention Predictionmentioning

confidence: 99%

“…Upperbound. We estimate importance scores for all the object proposals (tracklinks), so the final results depend on 1 We use 3-fold cross validation instead of 10-fold due to not enough data. 2 Since we do not have the object-category annotations.…”

Section: Baselinesmentioning

confidence: 99%

Goal-oriented Object Importance Estimation in On-road Driving Videos

Gao

Tawari

Martin

2019

2019 International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

We formulate a new problem as Object Importance Estimation (OIE) in on-road driving videos, where the road users are considered as important objects if they have influence on the control decision of the ego-vehicle's driver. The importance of a road user depends on both its visual dynamics, e.g., appearance, motion and location, in the driving scene and the driving goal, e.g., the planned path, of the ego vehicle. We propose a novel framework that incorporates both visual model and goal representation to conduct OIE. To evaluate our framework, we collect an on-road driving dataset at traffic intersections in the real world and conduct human-labeled annotation of the important objects. Experimental results show that our goal-oriented method outperforms baselines and has much more improvement on the left-turn and right-turn scenarios. Furthermore, we explore the possibility of using object importance for driving control prediction and demonstrate that binary brake prediction can be improved with the information of object importance. * Work done during an internship at the Honda Research Institute, USA.

show abstract

“…Drivers' gaze behavior has been studied as a proxy for their attention. Recently, a large driver attention dataset of routine driving [1] has been introduced and neural networks [21,25] have been trained end-to-end to estimate driver attention, mostly in lane-following and car-following situations. Nonetheless, datasets and prediction models for driver attention in rare and critical situations are still needed.…”

Section: Introductionmentioning

confidence: 99%

Predicting Driver Attention in Critical Situations

Xia

Zhang

Kim

et al. 2019

Lecture Notes in Computer Science

140

View full text Add to dashboard Cite

Robust driver attention prediction for critical situations is a challenging computer vision problem, yet essential for autonomous driving. Because critical driving moments are so rare, collecting enough data for these situations is difficult with the conventional in-car data collection protocol-tracking eye movements during driving. Here, we first propose a new in-lab driver attention collection protocol and introduce a new driver attention dataset, Berkeley DeepDrive Attention (BDD-A) dataset, which is built upon braking event videos selected from a large-scale, crowd-sourced driving video dataset. We further propose Human Weighted Sampling (HWS) method, which uses human gaze behavior to identify crucial frames of a driving dataset and weights them heavily during model training. With our dataset and HWS, we built a driver attention prediction model that outperforms the state-of-the-art and demonstrates sophisticated behaviors, like attending to crossing pedestrians but not giving false alarms to pedestrians safely walking on the sidewalk. Its prediction results are nearly indistinguishable from ground-truth to humans. Although only being trained with our in-lab attention data, the model also predicts in-car driver attention data of routine driving with state-of-the-art accuracy. This result not only demonstrates the performance of our model but also proves the validity and usefulness of our dataset and data collection protocol.

show abstract

DR(eye)VE: A Dataset for Attention-Based Tasks with Applications to Autonomous and Assisted Driving

Cited by 96 publications

References 34 publications

Towards Structured Evaluation of Deep Neural Network Supervisors

Towards Structured Evaluation of Deep Neural Network Supervisors

Goal-oriented Object Importance Estimation in On-road Driving Videos

Predicting Driver Attention in Critical Situations

Contact Info

Product

Resources

About