A Survey on Driver Behavior Analysis From In-Vehicle Cameras

Wang, Jiyang; Chai, Weiheng; Venkatachalapathy, Archana; Tan, Kai Liang; Haghighat, Arya; Velipasalar, Senem; Adu-Gyamfi, Yaw; Sharma, Anuj

doi:10.1109/tits.2021.3126231

Cited by 34 publications

(14 citation statements)

References 156 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The existing works for driver activity basically treat it as a classification problem, then it can be tackled by the efficient deep learning approach [34][35][36]. A commonly used input is an in-cabin image, and many convolutional neural network (CNN)-based approaches have been proposed from different perspectives [11,13,37]. [9] presented an ensemble of four CNN models to handle different parts of the driver, including the face, hands, and body, to recognize driver activity.…”

Section: Related Workmentioning

confidence: 99%

“…The demonstration can be found on the YouTube website. Driver anomaly quantification, as a typical task of a DMS, has been studied for a long time [9][10][11][12][13][14][15]. Recently, many researchers have revisited this topic by leveraging the powerful representation capabilities of deep learning, leading to impressive achievements [16][17][18][19][20][21][22][23][24].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Driver Anomaly Quantification for Intelligent Vehicles: A Contrastive Learning Approach With Representation Clustering

Xing

Gu³

et al. 2023

IEEE Trans. Intell. Veh.

View full text Add to dashboard Cite

Driver anomaly quantification is a fundamental capability to support human-centric driving systems of intelligent vehicles. Existing studies usually treat it as a classification task and obtain discrete levels for abnormalities. Meanwhile, the existing data-driven approaches depend on the quality of dataset and provide limited recognition capability for unknown activities.To overcome these challenges, this paper proposes a contrastive learning approach with the aim of building a model that can quantify driver anomalies with a continuous variable. In addition, a novel clustering supervised contrastive loss is proposed to optimize the distribution of the extracted representation vectors to improve the model performance. Compared with the typical contrastive loss, the proposed loss can better cluster normal representations while separating abnormal ones. The abnormality of driver activity can be quantified by calculating the distance to a set of representations of normal activities rather than being produced as the direct output of the model. The experiment results with datasets under different modes demonstrate that the proposed approach is more accurate and robust than existing ones in terms of recognition and quantification of unknown abnormal activities.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Driver Anomaly Quantification for Intelligent Vehicles: A Contrastive Learning Approach With Representation Clustering

Xing

Gu³

et al. 2023

IEEE Trans. Intell. Veh.

View full text Add to dashboard Cite

show abstract

“…Recent works in safety and advanced driver assistance systems utilize deep learning techniques in order to perform this driver analysis. In particular, deep learning allows researchers to extract driver state information and determine if they are distracted through analyzing driver characteristics such as eye-gaze, hand activity, or posture (Wang et al, 2021a). (Yang et al, 2020).…”

Section: Safety and Advanced Driver Assistance Systemsmentioning

confidence: 99%

Multi-View Ensemble Learning With Missing Data: Computational Framework and Evaluations using Novel Data from the Safe Autonomous Driving Domain

Greer¹,

Lulua²,

Gopalkrishnan³

et al. 2023

Preprint

View full text Add to dashboard Cite

Real-world applications with multiple sensors observing an event are expected to make continuously-available predictions, even in cases where information may be intermittently missing. We explore methods in ensemble learning and sensor fusion to make use of redundancy and information shared between four camera views, applied to the task of hand activity classification for autonomous driving. In particular, we show that a late-fusion approach between parallel convolutional neural networks can outperform even the best-placed single camera model. To enable this approach, we propose a scheme for handling missing information, and then provide comparative analysis of this late-fusion approach to additional methods such as weighted majority voting and model combination schemes.

show abstract

“…The human head and eye dynamics are fundamental to revealing the drivers’ gaze points that represent their current visual attention. Therefore, it has been widely used to detect the visual distraction and understand driver behaviors by exploiting the driver’s head and eye orientation [ 10 ]. In early works, several methods and devices were approached for accurate gaze information in driving environments, such as head-mounted eye trackers.…”

Section: Related Workmentioning

confidence: 99%

Dual-Cameras-Based Driver’s Eye Gaze Tracking System with Non-Linear Gaze Point Refinement

Ding

Yuan

2022

Sensors

View full text Add to dashboard Cite

The human eye gaze plays a vital role in monitoring people’s attention, and various efforts have been made to improve in-vehicle driver gaze tracking systems. Most of them build the specific gaze estimation model by pre-annotated data training in an offline way. These systems usually tend to have poor generalization performance during the online gaze prediction, which is caused by the estimation bias between the training domain and the deployment domain, making the predicted gaze points shift from their correct location. To solve this problem, a novel driver’s eye gaze tracking method with non-linear gaze point refinement is proposed in a monitoring system using two cameras, which eliminates the estimation bias and implicitly fine-tunes the gaze points. Supported by the two-stage gaze point clustering algorithm, the non-linear gaze point refinement method can gradually extract the representative gaze points of the forward and mirror gaze zone and establish the non-linear gaze point re-mapping relationship. In addition, the Unscented Kalman filter is utilized to track the driver’s continuous status features. Experimental results show that the non-linear gaze point refinement method outperforms several previous gaze calibration and gaze mapping methods, and improves the gaze estimation accuracy even on the cross-subject evaluation. The system can be used for predicting the driver’s attention.

show abstract

A Survey on Driver Behavior Analysis From In-Vehicle Cameras

Cited by 34 publications

References 156 publications

Driver Anomaly Quantification for Intelligent Vehicles: A Contrastive Learning Approach With Representation Clustering

Driver Anomaly Quantification for Intelligent Vehicles: A Contrastive Learning Approach With Representation Clustering

Multi-View Ensemble Learning With Missing Data: Computational Framework and Evaluations using Novel Data from the Safe Autonomous Driving Domain

Dual-Cameras-Based Driver’s Eye Gaze Tracking System with Non-Linear Gaze Point Refinement

Contact Info

Product

Resources

About