Conquering Ghosts: Relation Learning for Information Reliability Representation and End-to-End Robust Navigation

Abstract

Environmental disturbances, such as sensor data noises, various lighting conditions, challenging weathers and external adversarial perturbations, are inevitable in real selfdriving applications. Existing researches and testings have shown that they can severely influence the vehicle's perception ability and performance, one of the main issue is the false positive detection, i.e., the "ghost" object which is not real existed or occurs in the wrong position (such as a non-existent vehicle). Traditional navigation methods tend to avoid every detected objects for safety, however, avoiding a "ghost" object may lead the vehicle into a even more dangerous situation, such as a sudden break on the highway. Considering the various disturbance types, it is difficult to address this issue at the perceptual aspect. A potential solution is to detect the ghost through relation learning among the whole scenario and develop an integrated end-to-end navigation system. Our underlying logic is that the behavior of all vehicles in the scene is influenced by their neighbors, and normal vehicles behave in a logical way, while "ghost" vehicles do not. By learning the spatiotemporal relation among surrounding vehicles, an information reliability representation is learned for each detected vehicle and then a robot navigation network is developed. In contrast to existing works, we encourage the network to learn how to represent the reliability and how to aggregate all the information with uncertainties by itself, thus increasing the efficiency and generalizability. To the best of the authors' knowledge, this paper provides the first work on using graph relation learning to achieve end-to-end robust navigation in the presence of ghost vehicles. Simulation results in the CARLA platform demonstrate the feasibility and effectiveness of the proposed method in various scenarios.

Social Interaction‐Aware Dynamical Models and Decision‐Making for Autonomous Vehicles

“…Explicit interaction can be modelled in the Adjacency Matrix of the graph, whereas the implicit part consists of the graph convolutional layers. GCNs are widely used in traffic forecasting, [152][153][154][155] and have also been recently used in motion planning, [156][157][158][159] especially in combination with DRL.…”

Graph Neural Networks for Intelligent Transportation Systems: A Survey

“…Jin and Han [145] utilized relation learning on graphs to identify ghost objects (false positive detected objects). Their idea is that in a normal driving scene, all vehicles are affected by their neighbors, so the behavior of real vehicles is more or less logical, while the behavior of ghost vehicles is not.…”

RSPT: Reconstruct Surroundings and Predict Trajectory for Generalizable Active Object Tracking

“…Specifically, the agent must perceive the movement of the target and its surroundings and subsequently adjust its posture to continuously position the target at the center of its view with an appropriate size. AOT has a vast array of applications, including drones (Ci et al 2023), mobile robots (Wang et al 2018), and autonomous driving (Jin and Han 2022).…”