Forced Spatial Attention for Driver Foot Activity Classification

Rangesh, Akshay; Trivedi, Mohan M.

doi:10.1109/iccvw.2019.00308

Cited by 9 publications

(3 citation statements)

References 33 publications

(32 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We build upon the model proposed by Yuen et al [27] in this work, for driver hand analysis. Relatively few works have addressed the driver's foot activity [29]- [31]. However, we believe this is a significant cue for TOT estimation, especially since we estimate the footon-pedal time after the TOR.…”

Section: A Vision Based Driver Behavior Analysismentioning

confidence: 99%

Predicting Take-over Time for Autonomous Driving with Real-World Data: Robust Data Augmentation, Models, and Evaluation

Rangesh¹,

Deo²,

Greer³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Understanding occupant-vehicle interactions by modeling control transitions is important to ensure safe approaches to passenger vehicle automation. Models which contain contextual, semantically meaningful representations of driver states can be used to determine the appropriate timing and conditions for transfer of control between driver and vehicle. However, such models rely on real-world control take-over data from drivers engaged in distracting activities, which is costly to collect. Here, we introduce a scheme for data augmentation for such a dataset. Using the augmented dataset, we develop and train take-over time (TOT) models that operate sequentially on mid and high-level features produced by computer vision algorithms operating on different driver-facing camera views, showing models trained on the augmented dataset to outperform the initial dataset. The demonstrated model features encode different aspects of the driver state, pertaining to the face, hands, foot and upper body of the driver. We perform ablative experiments on feature combinations as well as model architectures, showing that a TOT model supported by augmented data can be used to produce continuous estimates of take-over times without delay, suitable for complex real-world scenarios.

show abstract

Section: A Vision Based Driver Behavior Analysismentioning

confidence: 99%

Predicting Take-over Time for Autonomous Driving with Real-World Data: Robust Data Augmentation, Models, and Evaluation

Rangesh¹,

Deo²,

Greer³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Traditional driver action recognition systems usually rely on a manual feature construction process, and a classification module such as support vector machines [17] and random forests [18]. The extracted feature vectors are derived from hand and body postures [11] as well as visually relevant inputs such as driver gaze [19], head posture [20], and dynamics during pedaling [21]. With the widespread employment of convolutional neural networks, DAR (Driver Action Recognization) has ushered in a new direction of development.…”

Section: Driver Action Recognitionmentioning

confidence: 99%

DSTANet: Learning a Dual-Stream Model for Anomaly Driving Action Detection Using Spatio-Temporal and Appearance Features

Peng,

Hu,

et al. 2024

Preprint

View full text Add to dashboard Cite

Driving action anomaly detection based on in-cab surveillance video has become the mainstream of current driving action research. However, there is a substantial redundancy of spatio-temporal information in the spatio-temporal action features extracted using only the 3D encoder, which will weaken the distinguishability between normal and anomaly videos, and there will be more difficult samples to be distinguished during the anomaly detection. To alleviate this problem, we propose a dual-stream model combining spatio-temporal and appearance features for anomaly driving action detection. Firstly, the model utilises 3D ResNet and 2D ResNet for feature extraction of spatio-temporal action and appearance information in the video. Next, to fully combine the respective advantages of the cross-dimension information and obtain fused features that can be more easily distinguished, a contrastive learning based alignment approach was used before fusion. In addition, cross-dimensional features are fused by introducing Transformer's cross attention block. Moreover, the supervised contrastive learning objective function is employed to guides the model to distinguish between normal and anomaly driving action. Finally, calculating the feature similarity between the target sample and the normal memory center is utilized to obtain the anomaly score of the target sample. In the experiment, the proposed model DSTANet in this paper achieved the 96.81% AUC in the publicly available DAD dataset, which is higher than the popular models in recent years. Furthermore, feature distinguishability was significantly improved in the visualization experiments. The experiment code and data are publicly available at: https://github.com/CreatedTRYNA/DCC\_Driving\_Anomaly\_Detection-.

show abstract

“…Traditional driver behavior recognition systems often rely on a manual feature construction process followed by a classification module like SVMs [17] and random forests [18]. The extracted feature vectors originate from hand-and body poses [14], [19], eyerelated inputs like driver gaze [20], [21], head patterns [17], [21], as well as foot dynamics [22]. Object recognition cues [23] and physiological signals [24], [25] are also associated for driver behavior observation.…”

Section: Related Workmentioning

confidence: 99%

TransDARC: Transformer-based Driver Activity Recognition with Latent Space Feature Calibration

Peng¹,

Roitberg²,

Yang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Traditional video-based human activity recognition has experienced remarkable progress linked to the rise of deep learning, but this effect was slower as it comes to the downstream task of driver behavior understanding. Understanding the situation inside the vehicle cabin is essential for Advanced Driving Assistant System (ADAS) as it enables identifying distraction, predicting driver's intent and leads to more convenient human-vehicle interaction. At the same time, driver observation systems face substantial obstacles as they need to capture different granularities of driver states, while the complexity of such secondary activities grows with the rising automation and increased driver freedom. Furthermore, a model is rarely deployed under conditions identical to the ones in the training set, as sensor placements and types vary from vehicle to vehicle, constituting a substantial obstacle for real-life deployment of data-driven models. In this work, we present a novel vision-based framework for recognizing secondary driver behaviours based on visual transformers and an additional augmented feature distribution calibration module. This module operates in the latent feature-space enriching and diversifying the training set at feature-level in order to improve generalization to novel data appearances, (e.g., sensor changes) and general feature quality. Our framework consistently leads to better recognition rates, surpassing previous state-of-the-art results of the public Drive&Act benchmark on all granularity levels. Our code will be made publicly available at https: //github.com/KPeng9510/TransDARC.

show abstract

Forced Spatial Attention for Driver Foot Activity Classification

Cited by 9 publications

References 33 publications

Predicting Take-over Time for Autonomous Driving with Real-World Data: Robust Data Augmentation, Models, and Evaluation

Predicting Take-over Time for Autonomous Driving with Real-World Data: Robust Data Augmentation, Models, and Evaluation

DSTANet: Learning a Dual-Stream Model for Anomaly Driving Action Detection Using Spatio-Temporal and Appearance Features

TransDARC: Transformer-based Driver Activity Recognition with Latent Space Feature Calibration

Contact Info

Product

Resources

About