WIP: End-to-End Analysis of Adversarial Attacks to Automated Lane Centering Systems

Abstract

Machine learning techniques, particularly those based on deep neural networks (DNNs), are widely adopted in the development of advanced driver-assistance systems (ADAS) and autonomous vehicles. While providing significant improvement over traditional methods in average performance, the usage of DNNs also presents great challenges to system safety, especially given the uncertainty of the surrounding environment, the disturbance to system operations, and the current lack of methodologies for predicting DNN behavior. In particular, adversarial attacks to the sensing input may cause errors in systems' perception of the environment and lead to system failure. However, existing works mainly focus on analyzing the impact of such attacks on the sensing and perception results and designing mitigation strategies accordingly. We argue that as system safety is ultimately determined by the actions it takes, it is essential to take an end-to-end approach and address adversarial attacks with the consideration of the entire ADAS or autonomous driving pipeline, from sensing and perception to planing, navigation and control. In this paper, we present our initial findings in quantitatively analyzing the impact of a type of adversarial attack (that leverages road patch) on system planning and control, and discuss some of the possible directions to systematically address such attack with an end-to-end view.

Invisible for both Camera and LiDAR: Security of Multi-Sensor Fusion based Perception in Autonomous Driving Under Physical-World Attacks

“…In comparison, these works mainly focus on vulnerabilities at sensor level, while we focus on those at the higher autonomy software level, i.e., the "brain" of AD systems. At such level, prior works have studied the security of camera/LiDAR object detection [5], [6], [9], [10], [117] and tracking [118], localization [119], lane detection [120]- [122], traffic light detection [123], and end-toend AD [12], [13]. However, so far all of them only consider attacks on camera or LiDAR perception alone, while we are the first to study the security of MSF-based AD perception and address the corresponding design challenges ( §III-B).…”

End-to-end Uncertainty-based Mitigation of Adversarial Attacks to Automated Lane Centering

“…The planner module in OpenPilot generates an estimation of this desired path by calculating a weighted average of the detected left lane line, right lane line and predicted path, with the weights being the confidence scores outputted by the perception module. Intuitively, if the perception module is less confident on the predicted lanes, the generated desired path relies more on the predicted path; otherwise, it will be closer to the weighted average of the predicted left lane line and right lane line [2], [22].…”

“…Through our experiments, we will use the dirty road patch attack [12] as a case study, but we believe that our approach can be extended to other similar physical environment attacks. As discussed in our preliminary work [22], these physical attacks typically will render abnormal behavior in the perception output and then propagate through the entire pipeline.…”

“…In our preliminary work recently published in a work-inprogress paper [22], we studied the automated lane centring (ALC) system in OpenPilot [2], a popular open-source ADAS implementation, and investigated how the dirty road patch attack from [12] could affect perception, planning and control modules. We discovered that a confidence score generated by the perception module could serve as a sensitive signal for detecting such attack, however it does not quantitatively measure the extent of the attack and cannot be effectively used for mitigation.…”

“…We discovered that a confidence score generated by the perception module could serve as a sensitive signal for detecting such attack, however it does not quantitatively measure the extent of the attack and cannot be effectively used for mitigation. In this paper, motivated by the findings from [22], we propose a novel end-to-end approach for detecting and mitigating the adversarial attacks on the ALC system. Our approach quantitatively estimates the uncertainty of the perception results, and develops an adaptive planning and control method based on the uncertainty analysis to improve system safety and robustness.…”