Improving Driver Gaze Prediction With Reinforced Attention

Lv, Kai; Sheng, Hao; Zhang, Xiong; Li, Wei; Zheng, Liang

doi:10.1109/tmm.2020.3038311

Cited by 32 publications

(10 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are now studies showing improvement in simulated driving scenarios by training models in an end-to-end manner using driver gaze, so that models can observe the traffic as human drivers [25,30]. Based on new created real-world datasets, such as DR(eye)VE [34] and BDD-A [54], a variety of deep neural networks are proposed to predict pixel-wise gaze maps of drivers (e.g., [15,33,34,45,54]). The DR(eye)VE model [34] uses a multi-branch deep architecture with three different pathways for color, motion and semantics.…”

Section: :3mentioning

confidence: 99%

Where and What: Driver Attention-based Object Detection

Rong,

Kassautzki,

Fuhl

et al. 2022

Preprint

View full text Add to dashboard Cite

Human drivers use their attentional mechanisms to focus on critical objects and make decisions while driving. As human attention can be revealed from gaze data, capturing and analyzing gaze information has emerged in recent years to benefit autonomous driving technology. Previous works in this context have primarily aimed at predicting "where" human drivers look at and lack knowledge of "what" objects drivers focus on. Our work bridges the gap between pixel-level and object-level attention prediction. Specifically, we propose to integrate an attention prediction module into a pretrained object detection framework and predict the attention in a grid-based style. Furthermore, critical objects are recognized based on predicted attended-to areas. We evaluate our proposed method on two driver attention datasets, BDD-A and DR(eye)VE. Our framework achieves competitive state-of-the-art performance in the attention prediction on both pixel-level and object-level but is far more efficient (75.3 GFLOPs less) in computation.

show abstract

Section: :3mentioning

confidence: 99%

Where and What: Driver Attention-based Object Detection

Rong,

Kassautzki,

Fuhl

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The detection of distracted driving can be classified into three categories: physiological signals [11], eye attention tracking [12], and computer vision [13]. Physiological signals will change when the driver performs different driving operations [14].…”

Section: Introductionmentioning

confidence: 99%

AB-DLM: An Improved Deep Learning Model Based on Attention Mechanism and BiFPN for Driver Distraction Behavior Detection

Zhang

Li³

et al. 2022

IEEE Access

View full text Add to dashboard Cite

Driver distraction behavior causes a large number of traffic accidents every year, resulting in economic losses and injuries. Currently, the driver still plays an important role in the driving and control of the vehicle due to the low level of vehicle automation and the immature development of autonomous driving. Therefore, it is vital to research distraction detection for drivers. However, in realistic driving scenarios with uncertain information, they are still some challenges in efficient and accurate driver distraction detection. In this paper, an improved deep learning model based on attention mechanisms and bi-directional feature pyramid networks (BiFPN) is proposed to identify driver distractions. Firstly, an improved data augmentation strategy is introduced to increase the data diversity to enhance the generalization capability of the model. Secondly, the squeeze-and-excitation (SE) attention mechanism layer is used after the C3 module of the original backbone network to enhance the important feature information and suppress the minor feature information. Finally, the BiFPN module is introduced into the neck network to better achieve multi-scale feature fusion without increasing the calculation amount too much. The experimental results show that the method proposed in this paper has an average mean accuracy rate (mAP) of 0.956 on the test set. Compared to the original model the mAP has improved by 13.2%. The detection speed of the model is 71 frames per second, and the memory occupation is 15.9 MB. This method has the advantages of high recognition accuracy, fast detection speed, and small memory occupation of the model, which are important for achieving engineering deployment.

show abstract

“…Humans have a tremendous ability to rapidly direct gaze and fixate their high-resolution fovea on the most relevant information from the visual world. Simulating and understanding this attentive mechanism has attracted increasing attention in recent years since it has both scientific and economic impact and can be applied to a wide range of computer vision tasks and applications like object tracking [1,2], image retargeting [3], driver gaze prediction [4][5][6] and video summarisation [7,8]. Eye fixation prediction model predicts human fixation locations for a given image.…”

Section: Introductionmentioning

confidence: 99%

CFN: A coarse‐to‐fine network for eye fixation prediction

Liang

et al. 2022

IET Image Processing

View full text Add to dashboard Cite

Many image‐to‐image computer vision approaches have made great progress by an end‐to‐end framework with the encoder–decoder architecture. However, the same image‐to‐image eye fixation prediction task is not the same as those computer vision tasks in that it focuses more on salient regions rather than precise predictions for every pixel. Thus, it is not appropriate to directly apply the end‐to‐end encoder–decoder to the eye fixation prediction task. In addition, although high‐level feature is important, the contribution of low‐level feature should also be kept and balanced in computational model. Nevertheless, some low‐level features that attract attention are easily neglected while transiting through the deep network. Therefore, the effective way to integrate low‐level and high‐level features for improving eye fixation prediction performance is still a challenging task. In this paper, a coarse‐to‐fine network (CFN) that encompasses two pathways with different training strategies are proposed: coarse perceiving network (CFN‐Coarse) can be a simple encoder network or any of the existing pretrained network to capture the distribution of salient regions and generate high‐quality feature maps; fine integrating network (CFN‐Fine) uses fixed parameters from the CFN‐Coarse and combines features from deep to shallow in the deconvolution process by adding skip connections between down‐sampling and up‐sampling paths to efficiently integrate deep and shallow features. The saliency map obtained by the method is evaluated over 6 standard benchmark datasets, namely SALICON, MIT1003, MIT300, Toronto, OSIE, and SUN500. The results demonstrate that the method can surpass the state‐of‐the‐art accuracy of eye fixation prediction and achieves the competitive performance to date under most evaluation metrics on SALICON Saliency Prediction Challenge (LSUN2017).

show abstract

Improving Driver Gaze Prediction With Reinforced Attention

Cited by 32 publications

References 44 publications

Where and What: Driver Attention-based Object Detection

Where and What: Driver Attention-based Object Detection

AB-DLM: An Improved Deep Learning Model Based on Attention Mechanism and BiFPN for Driver Distraction Behavior Detection

CFN: A coarse‐to‐fine network for eye fixation prediction

Contact Info

Product

Resources

About