Toward Driving Scene Understanding: A Dataset for Learning Driver Behavior and Causal Reasoning

Ramanishka, Vasili; Chen, Yi-Ting; Misu, Teruhisa; Saenko, Kate

doi:10.1109/cvpr.2018.00803

Cited by 213 publications

(198 citation statements)

References 30 publications

(56 reference statements)

Supporting

Mentioning

196

Contrasting

Unclassified

Order By: Relevance

“…However, these prior efforts formulate the behavior as a goal-oriented task, which is not sufficient to learn how humans drive and interact with traffic scenes. In our work, we explore the cause-aware features by our proposed framework using the HDD [4].…”

Section: Self-driving Behavior Recognitionmentioning

confidence: 99%

“…To deal with the imbalanced training samples for different classes, oversampling of data was used during the training process. Evaluation: Different from [4], which performed per-frame evaluation on testing videos and calculated the average precision(AP), we measured the model performances based on the video recognition tasks. Video clips, where each contains a single target label, were used for evaluation.…”

Section: Model Implementationmentioning

confidence: 99%

“…Video recognition models can be categorized into Convolutional Recurrent Neural Networks (CRNNs) [11] and 3D Convolutional Neural Networks (CNNs) [12]. While [4] proposed a CRNN as a reasoning model of driving behaviors, the performance of 3D CNNs on those higher-level behaviors has not been explored. To further improve the models on reasoning driving behaviors, we introduced a Temporal Reasoning Block (TRB) to enhance the model understanding of the causes of driving behaviors.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Interpretable Self-Attention Temporal Reasoning for Driving Behavior Understanding

Liu¹,

Hsieh²,

Chen³

et al. 2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Performing driving behaviors based on causal reasoning is essential to ensure driving safety. In this work, we investigated how state-ofthe-art 3D Convolutional Neural Networks (CNNs) perform on classifying driving behaviors based on causal reasoning. We proposed a perturbation-based visual explanation method to inspect the models' performance visually. By examining the video attention saliency, we found that existing models could not precisely capture the causes (e.g., traffic light) of the specific action (e.g., stopping). Therefore, the Temporal Reasoning Block (TRB) was proposed and introduced to the models. With the TRB models, we achieved the accuracy of 86.3%, which outperform the state-of-the-art 3D CNNs from previous works. The attention saliency also demonstrated that TRB helped models focus on the causes more precisely. With both numerical and visual evaluations, we concluded that our proposed TRB models were able to provide accurate driving behavior prediction by learning the causal reasoning of the behaviors.

show abstract

Section: Self-driving Behavior Recognitionmentioning

confidence: 99%

Section: Model Implementationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Interpretable Self-Attention Temporal Reasoning for Driving Behavior Understanding

Liu¹,

Hsieh²,

Chen³

et al. 2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

show abstract

“…The Honda Research Institute Driving Dataset (HDD) [RCMS18] includes 104 hours of driving data in San Francisco Bay Area. A diverse set of traffic scenes is included.…”

Section: The Authors Computer Graphics Forum C 2019 Eurographimentioning

confidence: 99%

A Survey on Visual Traffic Simulation: Models, Evaluations, and Applications in Autonomous Driving

Chao

et al. 2019

Computer Graphics Forum

View full text Add to dashboard Cite

Virtualized traffic via various simulation models and real‐world traffic data are promising approaches to reconstruct detailed traffic flows. A variety of applications can benefit from the virtual traffic, including, but not limited to, video games, virtual reality, traffic engineering and autonomous driving. In this survey, we provide a comprehensive review on the state‐of‐the‐art techniques for traffic simulation and animation. We start with a discussion on three classes of traffic simulation models applied at different levels of detail. Then, we introduce various data‐driven animation techniques, including existing data collection methods, and the validation and evaluation of simulated traffic flows. Next, we discuss how traffic simulations can benefit the training and testing of autonomous vehicles. Finally, we discuss the current states of traffic simulation and animation and suggest future research directions.

show abstract

“…Dataset description: We evaluated our approach on a 150 hours HDD driving dataset [2], which is collected from February 2017 to March 2018, predominantly during daytime. The data consists of Controller Area Network (CAN) bus data that has information about six driving modalitiessteer angle, steer speed, speed, yaw, pedal angle and pedal pressure.…”

Section: Performance Evaluationmentioning

confidence: 99%

Deep Multi-Task Learning for Anomalous Driving Detection Using CAN Bus Scalar Sensor Data

Sadhu

Misu

Pompili

2019

2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Self Cite

View full text Add to dashboard Cite

Corner cases are the main bottlenecks when applying Artificial Intelligence (AI) systems to safety-critical applications. An AI system should be intelligent enough to detect such situations so that system developers can prepare for subsequent planning. In this paper, we propose semi-supervised anomaly detection considering the imbalance of normal situations. In particular, driving data consists of multiple positive/normal situations (e.g., right turn, going straight), some of which (e.g., U-turn) could be as rare as anomalous situations. Existing machine learning based anomaly detection approaches do not fare sufficiently well when applied to such imbalanced data. In this paper, we present a novel multi-task learning based approach that leverages domain-knowledge (maneuver labels) for anomaly detection in driving data. We evaluate the proposed approach both quantitatively and qualitatively on 150 hours of real-world driving data and show improved performance over baseline approaches.

show abstract

Toward Driving Scene Understanding: A Dataset for Learning Driver Behavior and Causal Reasoning

Cited by 213 publications

References 30 publications

Interpretable Self-Attention Temporal Reasoning for Driving Behavior Understanding

Interpretable Self-Attention Temporal Reasoning for Driving Behavior Understanding

A Survey on Visual Traffic Simulation: Models, Evaluations, and Applications in Autonomous Driving

Deep Multi-Task Learning for Anomalous Driving Detection Using CAN Bus Scalar Sensor Data

Contact Info

Product

Resources

About