PASS: A System-Driven Evaluation Platform for Autonomous Driving Safety and Security

Hu, Zhisheng; Shen, Junjie; Guo, Song; Zhang, Xinyang; Zhong, Zhenyu; Chen, Qi Alfred; Li, Kang

doi:10.14722/autosec.2022.23018

Cited by 3 publications

(2 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…VeReMi dataset focuses on detecting anomalies of an AV based on position and speed of the AV while the Sensor dataset is considering other communication-based data that the sensors mounted on AVs collect to detect anomalies. Nevertheless, there are other autonomous driving datasets (e.g., A2D2 [75], and Pass [76]) with other features that are not considered in our studied datasets. However, we emphasize that our proposed XAI-based framework and feature selection methods can be leveraged to know the contribution of different features for different types of datasets to enhance security of AVs.…”

Section: Limitations and Discussionmentioning

confidence: 99%

XAI-ADS: An Explainable Artificial Intelligence Framework for Enhancing Anomaly Detection in Autonomous Driving Systems

Nazat,

Li,

Abdallah

2024

IEEE Access

View full text Add to dashboard Cite

The advent of autonomous driving systems has given rise to pressing cybersecurity issues regarding the vulnerability of vehicular ad hoc networks (VANETs) to potential attacks. This critical security problem necessitates the application of artificial intelligence (AI) models for anomaly detection in VANETs of autonomous vehicles (AVs). However, the lack of explainability of such AI-based anomaly detection models presents challenges. This motivates an emerging research direction of utilizing explainable AI (XAI) techniques to elucidate the behaviors of anomaly detection models in AV networks. In this work, we propose an end-to-end XAI framework to interpret and visualize the anomaly detection classifications made by AI models securing VANETs. We evaluate the framework on two real-world autonomous driving datasets. The framework furnishes both global and local explanations for the black-box AI models using two XAI methods. Moreover, we introduce two novel feature selection techniques to identify the salient features contributing to anomaly detection, derived from the popular SHAP XAI method and the accuracy of six different black-box AI models. We compare our proposed feature selection approaches with six state-of-the-art feature selection techniques (including two wrapper-based feature selection methods), demonstrating superior performance on various evaluation metrics. To generalize the impact of our feature selection methods, we apply three independent classifiers to evaluate our proposed feature selection approaches. The novel feature selection methods effectively distill the most explanatory features, enhancing model interpretability. Finally, we assess the efficiency (how quickly the XAI models can yield explanatory findings) for each of the six black-box AI models we employed on our two datasets, identifying the most efficient model. By furnishing explanations and visualizations of anomaly detection by AI models, our XAI framework can help in enabling trust and transparency for securing vehicular networks.

show abstract

Section: Limitations and Discussionmentioning

confidence: 99%

XAI-ADS: An Explainable Artificial Intelligence Framework for Enhancing Anomaly Detection in Autonomous Driving Systems

Nazat,

Li,

Abdallah

2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…The VeReMi dataset focuses on detecting anomalies of an AV based on the position and speed of the AV while the Sensor dataset considers other communication-based data that the sensors mounted on AVs collect to detect anomalies. Nevertheless, there are other autonomous driving datasets (e.g., nuScenes [ 63 ], A2D2 [ 64 ], and Pass [ 65 ]) with other features that are not considered in our studied datasets. However, we emphasize that our proposed XAI evaluation framework can be leveraged to identify the effectiveness of XAI methods on these datasets and the contribution of different features for different types of datasets to enhance security of AVs.…”

Section: Limitations and Discussionmentioning

confidence: 99%

On Evaluating Black-Box Explainable AI Methods for Enhancing Anomaly Detection in Autonomous Driving Systems

Nazat,

Arreche,

Abdallah

2024

Sensors

View full text Add to dashboard Cite

The recent advancements in autonomous driving come with the associated cybersecurity issue of compromising networks of autonomous vehicles (AVs), motivating the use of AI models for detecting anomalies on these networks. In this context, the usage of explainable AI (XAI) for explaining the behavior of these anomaly detection AI models is crucial. This work introduces a comprehensive framework to assess black-box XAI techniques for anomaly detection within AVs, facilitating the examination of both global and local XAI methods to elucidate the decisions made by XAI techniques that explain the behavior of AI models classifying anomalous AV behavior. By considering six evaluation metrics (descriptive accuracy, sparsity, stability, efficiency, robustness, and completeness), the framework evaluates two well-known black-box XAI techniques, SHAP and LIME, involving applying XAI techniques to identify primary features crucial for anomaly classification, followed by extensive experiments assessing SHAP and LIME across the six metrics using two prevalent autonomous driving datasets, VeReMi and Sensor. This study advances the deployment of black-box XAI methods for real-world anomaly detection in autonomous driving systems, contributing valuable insights into the strengths and limitations of current black-box XAI methods within this critical domain.

show abstract

Simutack - An Attack Simulation Framework for Connected and Autonomous Vehicles

Finkenzeller

Mathur

Lauinger

et al. 2023

2023 IEEE 97th Vehicular Technology Conference (VTC2023-Spring)

View full text Add to dashboard Cite

PASS: A System-Driven Evaluation Platform for Autonomous Driving Safety and Security

Cited by 3 publications

References 34 publications

XAI-ADS: An Explainable Artificial Intelligence Framework for Enhancing Anomaly Detection in Autonomous Driving Systems

XAI-ADS: An Explainable Artificial Intelligence Framework for Enhancing Anomaly Detection in Autonomous Driving Systems

On Evaluating Black-Box Explainable AI Methods for Enhancing Anomaly Detection in Autonomous Driving Systems

Simutack - An Attack Simulation Framework for Connected and Autonomous Vehicles

Contact Info

Product

Resources

About