MEDIRL: Predicting the Visual Attention of Drivers via Maximum Entropy Deep Inverse Reinforcement Learning

CoCAtt: A Cognitive-Conditioned Driver Attention Dataset

“…Four large-scale publicly available datasets have been popularly used in modeling driver attention: DR(eye)VE [31], BDD-Attention [43], DADA-2000 [7], and EyeCar [3]. With 6 hours of eye-tracking data, the DR(eye)VE dataset is the only dataset collected in-car and also provides distraction-related annotations for 20% of its frames [31].…”

“…The attention data in the BDD-A dataset was collected in an in-lab setup, in which participants were asked to imagine themselves as the driver in a driver-perspective video. This collection protocol was later reused by the DADA-2000 and EyeCar project with a focus on traffic accident scenarios [3,7]. Although this collection protocol is cost-effective, the collected gaze data is less credible as the participants were not the actual drivers in the setup.…”

“…Other early methods also use machine learning models, e.g., SVMs, to predict saliency maps [39]. More recent methods predict driver attention using stateof-the-art deep learning techniques [3,30,31]. For example, Alletto et al proposed a 3D convolution-based multi-branch deep network to predict driver attention directly from raw video sequences using RGB image features, optical flow, and semantic segmentation [31].…”

“…For example, Alletto et al proposed a 3D convolution-based multi-branch deep network to predict driver attention directly from raw video sequences using RGB image features, optical flow, and semantic segmentation [31]. Another example is the modular method proposed by Baee et al that combines the front view and eye fixation data to extract context features for Deep Inverse Reinforcement learning [3]. There have…”

“…However, as defined by SAE J3016 standards, human drivers still need to remain attentive at all times while driving at lower levels of autonomy, e.g., SAE Level 3 [37]. Thus, to better monitor driver behavior, researchers have shown great interest in modeling driver attention to capture the attention behavior of human drivers across driving scenes [3,16,25,30,31,43]. The primary focus of the pioneering work of driver attention prediction has been in modeling generic driver attention behavior at a task-driven level, i.e., modeling what most drivers would attend to in the driving scene [31].…”