A Dynamic Spatial-Temporal Attention Network for Early Anticipation of Traffic Accidents

Abstract

Recently, autonomous vehicles and those equipped with an Advanced Driver Assistance System (ADAS) are emerging. They share the road with regular ones operated by human drivers entirely. To ensure guaranteed safety for passengers and other road users, it becomes essential for autonomous vehicles and ADAS to anticipate traffic accidents from natural driving scenes. The dynamic spatial-temporal interaction of the traffic agents is complex, and visual cues for predicting a future accident are embedded deeply in dashcam video data. Therefore, early anticipation of traffic accidents remains a challenge. To this end, the paper presents a dynamic spatial-temporal attention (DSTA) network for early anticipation of traffic accidents from dashcam videos. The proposed DSTA-network learns to select discriminative temporal segments of a video sequence with a module named Dynamic Temporal Attention (DTA). It also learns to focus on the informative spatial regions of frames with another module named Dynamic Spatial Attention (DSA). The spatial-temporal relational features of accidents, along with scene appearance features, are learned jointly with a Gated Recurrent Unit (GRU) network. The experimental evaluation of the DSTAnetwork on two benchmark datasets confirms that it has exceeded the state-of-the-art performance. A thorough ablation study evaluates the contributions of individual components of the DSTAnetwork, revealing how the network achieves such performance. Furthermore, this paper proposes a new strategy that fuses the prediction scores from two complementary models and verifies its effectiveness in further boosting the performance of early accident anticipation.

Application of Machine Learning in Ethical Design of Autonomous Driving Crash Algorithms

“…This problem leads to that in the practical application process, and deep neural network can give a particularly good result in most cases, but occasionally gives a particularly bad result in extreme cases. However, it is the particularly bad result that usually leads to the occurrence of traffic accidents in the end [ 29 , 30 ].…”

An Attention-guided Multistream Feature Fusion Network for Localization of Risky Objects in Driving Videos

“…Unlike video surveillance systems, dashcam videos capture moving traffic agents that not only rapidly move but appear and disappear quickly in the scene. Different advanced methods are developed to learn the spatiotemporal pattern of the traffic agents to provide an overall riskiness score of the scene, including an LSTM predictor [6], reinforced learning [8], Graph Neural Network [7], [11], and dynamic attention [9]. Although they only predict a risky event in the temporal domain, these studies have developed a solid methodological foundation for risky object localization in the spatial domain.…”

A New Approach to Traffic Accident Anticipation With Geometric Features for Better Generalizability

“…The method provided surprisingly good results (99.5% of the average precision (AP) with 4.74 seconds of mean time to accident (mTTA) in Table 4). Another similar approach (called Dynamic Spatial-Temporal Attention (DSTA)), using visual features of scene and the objects, further showed slightly better results (99.6% of AP with 4.87s mTTA in Table 4) [5]. However, we believe that these nearly perfect results seemed to be too good to.…”

“…It is particularly important for safety-critical driving systems that include several highly complex tasks involving interaction with dynamic environments. In recent years, a few methods have been proposed to solve this problem for autonomous or assisted driving [1]- [5]. Traffic accident prediction at every frame in a dashcam video can provide an anticipation of accident at any point in time, which in turn can issue timely preventive warnings or actions.…”