Two-stream neural architecture for unsafe maneuvers classification from dashcam videos and GPS/IMU sensors

Unsafe Maneuver Classification From Dashcam Video and GPS/IMU Sensors Using Spatio-Temporal Attention Selector

“…To the best of our knowledge, the only paper addressing unsafe maneuver classification is [1]. In this section we discuss the most relevant papers in the literature, focusing on closely related tasks using object detection and the usage of the attention mechanism in the computer vision literature.…”

“…T HE unsafe maneuver classification task was firstly introduced in [1], with the aim of classifying a safety-critical event (i.e., crashes and near crashes) recorded from a dashcam, according to the unsafe maneuver leading to the dangerous situation. This task is interesting for multiple reasons.…”

“…These maneuvers were grouped into ten classes, grounded on a Naturalistic Driving Study (NDS) dataset [3] and, thus, are representative of the real distribution of unsafe events. Specifically, the ten classes of unsafe maneuvers introduced in [1] and considered in this work are reported in Table I.…”

“…In this paper, we propose an extension of the approach of [1], where the authors considered video information and data from a GPS/IMU module (acceleration, angular velocity and…”

“…Such architecture processes the output of an object detection algorithm in combination with raw video frames and GPS/IMU data. At the core of the architecture there is a novel Spatio-Temporal Attention Selector (STAS) module, which (1) extracts features describing the evolution of each object in the scene over time and (2) leverages multi-head dot product attention to select the relevant ones, i.e., the dangerous ones or the ones in danger, to perform classification. We also introduce a simple but effective methodology to increase the benefit of finetuning the backbone network.…”