Adversarial Attacks and Defense in Deep Reinforcement Learning (DRL)-Based Traffic Signal Controllers

Haydari, Ammar; Zhang, Michael; Chuah, Chen-Nee

doi:10.1109/ojits.2021.3118972

Cited by 16 publications

(5 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Algorithm 1 summarizes the detailed algorithm steps of DLA. Input x n * to the classifier, calculate the feature loss L f according to (17), and determine the direction of the feature level perturbation according to (18); 4:…”

Section: Double Level Adversarial Attackmentioning

confidence: 99%

“…Since adversarial attack and defense are two parts of a game process, many researchers have designed a defense method against the attack after proposing a new attack method to improve the defense ability of the classifier [15]. For example, in the field of intelligent transportation, researchers have studied the adversarial attack and defense methods of the intelligent recognition model of urban road conditions, and improved the robustness of the model in automatic driving in the field of image recognition [16], [17]. Today, compared with the image field, there is little work to apply adversarial examples to wireless communication, which means that the communication jamming recognition model will be easily attacked.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Adversarial Attack and Defense on Deep Learning for Air Transportation Communication Jamming

Liu

Zhang

Chen

et al. 2024

IEEE Trans. Intell. Transport. Syst.

View full text Add to dashboard Cite

Air transportation communication jamming recognition model based on deep learning (DL) can quickly and accurately identify and classify communication jamming, to improve the safety and reliability of air traffic. However, due to the vulnerability of deep learning, the jamming recognition model can be easily attacked by the attacker's carefully designed adversarial examples. Although some defense methods have been proposed, they have strong pertinence to attacks. Thus, new attack methods are needed to improve the defense performance of the model. In this work, we improve the existing attack methods and propose a double level attack method. By constructing the dynamic iterative step size and analyzing the class characteristics of the signals, this method can use the adversarial losses of feature layer and decision layer to generate adversarial examples with stronger attack performance. In order to improve the robustness of the recognition model, we use adversarial examples to train the model, and transfer the knowledge learned from the model to the jamming recognition models in other wireless communication environments by transfer learning. Simulation results show that the proposed attack and defense methods have good performance.

show abstract

Section: Double Level Adversarial Attackmentioning

confidence: 99%

mentioning

confidence: 99%

Adversarial Attack and Defense on Deep Learning for Air Transportation Communication Jamming

Liu

Zhang

Chen

et al. 2024

IEEE Trans. Intell. Transport. Syst.

View full text Add to dashboard Cite

show abstract

“…They concluded that when training in scenarios with sensor failures, the RL approach can be quite robust to the wide sensor failure and demand surge problems. Haydari et al (2021) investigated security vulnerabilities in Deep Reinforcement Learning (DRL) based Traffic Signal Control (TSC) systems under adversarial attacks. They explored two threat models using white-box and black-box attacks on different DRL-based TSC algorithms in single and multiple intersections.…”

Section: Robustness In Traffic Signal Controlmentioning

confidence: 99%

Improving the Generalizability and Robustness of Large-Scale Traffic Signal Control

Shi,

Devailly,

Larocque

et al. 2024

IEEE Open J. Intell. Transp. Syst.

View full text Add to dashboard Cite

A number of deep reinforcement-learning (RL) approaches propose to control traffic signals.Compared to traditional approaches, RL approaches can learn from higher-dimensionality input road and vehicle sensors and better adapt to varying traffic conditions resulting in reduced travel times (in simulation). However, these RL methods require training from massive traffic sensor data. To offset this relative inefficiency, some recent RL methods have the ability to first learn from small-scale networks and then generalize to unseen city-scale networks without additional retraining (zero-shot transfer). In this work, we study the robustness of such methods along two axes. First, sensor failures and GPS occlusions create missing-data challenges and we show that recent methods remain brittle in the face of these missing data. Second, we provide a more systematic study of the generalization ability of RL methods to new networks with different traffic regimes. Again, we identify the limitations of recent approaches. We then propose using a combination of distributional and vanilla reinforcement learning through a policy ensemble. Building upon the state-of-the-art previous model which uses a decentralized approach for large-scale traffic signal control with graph convolutional networks (GCNs), we first learn models using a distributional reinforcement learning (DisRL) approach. In particular, we use implicit quantile networks (IQN) to model the state-action return distribution with quantile regression. For traffic signal control problems, an ensemble of standard RL and DisRL yields superior performance across different scenarios, including different levels of missing sensor data and traffic flow patterns. Furthermore, the learning scheme of the resulting model can improve zero-shot transferability to different road network structures, including both synthetic networks and real-world networks (e.g., Luxembourg, Manhattan). We conduct extensive experiments to compare our approach to multi-agent reinforcement learning and traditional transportation approaches. Results show that the proposed method improves robustness and generalizability in the face of missing data, varying road networks, and traffic flows.

show abstract

“…Recently, Haydari et al, [35] used reinforcement learning based to model attacks on traffic control and study their impacts. They used SUMO simulations and observed anomalous behavior to be detected well.…”

Section: Review Of Past Studiesmentioning

confidence: 99%

Cyberattack Monitoring Architectures for Resilient Operation of Connected and Automated Vehicles

Khattak,

Smith,

Fontaine

2024

IEEE Open J. Intell. Transp. Syst.

View full text Add to dashboard Cite

The deployment of connected and automated vehicles (CAVs) may enhance operations and safety with little human feedback. Automation requires the use of communication and smart devices, thus introducing potential access points for adversaries. This paper develops a prototype real-time monitoring system for a vehicle to infrastructure (V2I) based CAV system that generates cyberattack data for CAV operations under realistic traffic conditions. The monitoring system detects any deviations from the normal operation of CAVs using a long-short term memory (LSTM) neural network proposed by the authors and reverts the system back to a safe state of operation using a set of countermeasures. The proposed algorithm was also compared to convolutional neural network (CNN) and other classical algorithms. The monitoring system detected three different emulated cyberattacks with high accuracy. The LSTM showed the highest accuracy of 98% and outperformed the other algorithms. Further, the performance of the monitoring systems was assessed in terms of the impact on traffic stream stability and safety. The results reveal that a fake basic safety message (BSM) attack on even a single CAV causes the traffic stream to become significantly unstable and increase safety risk without the monitoring system. The monitoring system, however, reverts the system to a safe state of operation and reduces the negative impacts of cyberattacks. The monitoring system improves flow stability by an average of 38% as quantified through acceleration variation and volatility. This is comparable to the base case without attacks. The findings have implications for the design of future resilient systems. INDEX TERMS -Cybersecurity; monitoring system; volatility; cooperative game; connected and automated vehicles; intelligent transportation I. INTRODUCTION Cooperative intelligent transportation technologies including connected and automated vehicles (CAVs) offer the potential to revolutionize transportation systems by improving efficiency and safety [1]. Since driver error is estimated to be a contributing factor in 93% of crashes [2], CAVs may reduce these crashes by reducing the effect of operator errors. CAVs utilize communication between vehicles (V2V) and infrastructure (V2I) to communicate location, position, etc., to surrounding vehicles and roadside units (RSUs). This communication is vital for the wider deployment of CAVs. The operation of multiple CAVs on the road network will require resilient communications between vehicles and infrastructure. This communication dependence provides access points to hackers, however. While vulnerability to cyberattacks also exists in traditional transportation systems [3], the additional reliance and interdependency on communication and automation may exacerbate the situation for CAVs and result in severe crashes and traffic instability. This makes it imperative to analyze the cyber risks of the CAV environment before reaching a mass deployment stage so that the design of these applications can be enhanced to perform...

show abstract

Adversarial Attacks and Defense in Deep Reinforcement Learning (DRL)-Based Traffic Signal Controllers

Cited by 16 publications

References 25 publications

Adversarial Attack and Defense on Deep Learning for Air Transportation Communication Jamming

Adversarial Attack and Defense on Deep Learning for Air Transportation Communication Jamming

Improving the Generalizability and Robustness of Large-Scale Traffic Signal Control

Cyberattack Monitoring Architectures for Resilient Operation of Connected and Automated Vehicles

Contact Info

Product

Resources

About