Detection and Recognition of Driver Distraction Using Multimodal Signals

Abstract

Distracted driving is a leading cause of accidents worldwide. The tasks of distraction detection and recognition have been traditionally addressed as computer vision problems. However, distracted behaviors are not always expressed in a visually observable way. In this work, we introduce a novel multimodal dataset of distracted driver behaviors, consisting of data collected using twelve information channels coming from visual, acoustic, near-infrared, thermal, physiological and linguistic modalities. The data were collected from 45 subjects while being exposed to four different distractions (three cognitive and one physical). For the purposes of this paper, we performed experiments with visual, physiological, and thermal information to explore potential of multimodal modeling for distraction recognition. In addition, we analyze the value of different modalities by identifying specific visual, physiological, and thermal groups of features that contribute the most to distraction characterization. Our results highlight the advantage of multimodal representations and reveal valuable insights for the role played by the three modalities on identifying different types of driving distractions.

Neuromorphic Driver Monitoring Systems: A Computationally Efficient Proof-of-Concept for Driver Distraction Detection

“…Kapotaksha et al [16] proposed the detection and recognition of driver distraction using multi modal signals like visual, physiological and thermal groups of features with random forest and gradient boosting classifiers. Authors in [20], proposed an unsupervised deep learning algorithm that applies RepMLP-Res50 (replaces some blocks in ResNet50 with ResMLP).…”

Resource-Conscious Architecture Search for Driver Distraction Detection: A Pareto Optimised Multi-Objective Approach

“…Different data and sensor categories have been used for DDD in the literature [13]. Visual data remains the optimal choice due to its non-intrusive and practical nature for realworld deployment.…”