A Dynamic Deep Reinforcement Learning-Bayesian Framework for Anomaly Detection

Watts, Jeremy; Wyk, Franco van; Rezaei, Shahrbanoo; Wang, Yiyang; Masoud, Neda; Khojandi, Anahita

doi:10.1109/tits.2022.3200906

Cited by 16 publications

(6 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Deep reinforcement learning (DRL) was first introduced by Mnih et al [58] by developing a deep Q-network to learn optimal policies on the Attari 2600 games. It then was applied to several applications such as anomaly detection [59], cyber security [60], digital twin networks [61], and energy management [62], [63]. Although DRL has achieved success, it lacks the comprehension of physical laws, resulting in practically infeasible solutions.…”

Section: Pi Strategies In Reinforcement Learning Methodsmentioning

confidence: 99%

Physics-Informed Machine Learning for Data Anomaly Detection, Classification, Localization, and Mitigation: A Review, Challenges, and Path Forward

Zideh,

Chatterjee,

Srivastava

2024

IEEE Access

View full text Add to dashboard Cite

Advancements in digital automation for smart grids have led to the installation of measurement devices like phasor measurement units (PMUs), micro-PMUs (µ-PMUs), and smart meters. However, large amount of data collected by these devices bring several challenges as control room operators need to use this data with models to make confident decisions for reliable and resilient operation of the cyber-power systems. Machine-learning (ML) based tools can provide a reliable interpretation of the deluge of data obtained from the field. For the decision-makers to ensure reliable network operation under all operating conditions, these tools need to identify solutions that are feasible and satisfy the system constraints, while being efficient, trustworthy and interpretable. This resulted in the increasing popularity of physics-informed machine learning (PIML) approaches, as these methods overcome challenges that model-based or datadriven ML methods face in silos. This work aims at the following: a) review existing strategies and techniques for incorporating underlying physical principles of the power grid into different types of ML approaches (supervised/semi-supervised learning, unsupervised learning, and reinforcement learning (RL)); b) explore the existing works on PIML methods for anomaly detection, classification, localization, and mitigation in power transmission and distribution systems, c) discuss improvements in existing methods through consideration of potential challenges while also addressing the limitations to make them suitable for real-world applications.

show abstract

Section: Pi Strategies In Reinforcement Learning Methodsmentioning

confidence: 99%

Physics-Informed Machine Learning for Data Anomaly Detection, Classification, Localization, and Mitigation: A Review, Challenges, and Path Forward

Zideh,

Chatterjee,

Srivastava

2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Watts et al, in [128], have presented another technique for detecting anomalies caused by anomalous/faulty information by integrating a CNN classification model into a Bayesian framework comprising a Partially Observable Markov Decision Process (POMDP) model. In this method, the CNN model first analyzes past sensor readings and provides probabilities of anomalies at each epoch.…”

Section: Deep Learning-based Techniquesmentioning

confidence: 99%

Anomaly Detection in Connected and Autonomous Vehicles: A Survey, Analysis, and Research Challenges

Baccari,

Hadded,

Ghazzai

et al. 2024

IEEE Access

View full text Add to dashboard Cite

In Intelligent Transportation Systems (ITS), ensuring road safety has paved the way for innovative advancements such as autonomous driving. These self-driving vehicles, with their variety of sensors, harness the potential to minimize human driving errors and enhance transportation efficiency via sophisticated AI modules. However, the reliability of these sensors remains challenging, especially as they can be vulnerable to anomalies resulting from adverse weather, technical issues, and cyber-attacks. Such inconsistencies can lead to imprecise or erroneous navigation decisions for autonomous vehicles that can result in fatal consequences, e.g., failure in recognizing obstacles. This survey delivers a comprehensive review of the latest research on solutions for detecting anomalies in sensor data. After laying the foundation on the workings of the connected and autonomous vehicles, we categorize anomaly detection methods into three groups: statistical, classical machine learning, and deep learning techniques. We provide a qualitative assessment of these methods to underline existing research limitations. We conclude by spotlighting key research questions to enhance the dependability of autonomous driving in forthcoming studies.

show abstract

“…A CNN was used to extract spatial features from the sensor data, while LSTM was used to analyze the temporal features. The author of [21] proposed a new approach that combines deep reinforcement learning and Bayesian inference to detect real-time anomalies. They used a deep reinforcement learning agent that learns to detect anomalies in a dynamic environment by interacting with the environment and receiving rewards based on its actions.…”

Section: Related Workmentioning

confidence: 99%

Single and Mixed Sensory Anomaly Detection in Connected and Automated Vehicle Sensor Networks

Kim,

Ojo,

Krichen

et al. 2024

Electronics

View full text Add to dashboard Cite

Connected and automated vehicles (CAVs), integrated with sensors, cameras, and communication networks, are transforming the transportation industry and providing new opportunities for consumers to enjoy personalized and seamless experiences. The fast proliferation of connected vehicles on the road and the growing trend of autonomous driving create vast amounts of data that need to be analyzed in real time. Anomaly detection in CAVs refers to identifying any unusual or unforeseen behavior in the data generated by vehicles’ various sensors and components. Anomaly detection aims to identify any unusual behavior that might indicate a problem or a malfunction in the vehicle. To identify and detect anomalies efficiently, a method must deal with noisy data, missing data, dynamic frequency data, and low- and high-magnitude data, and it must be accurate enough to detect anomalies in a dynamic sensor streaming environment. Therefore, this paper proposes a fast and efficient hard-voting-based technique named FT-HV, comprising three fine-tuned machine learning algorithms to detect and classify anomaly behavior in CAVs for single and mixed sensory datasets. In experiments, we evaluate our approach on the benchmark Sensor Anomaly dataset that contains data from various vehicle sensors at low and high magnitudes. Further, it contains single and mixed anomaly types that are challenging to detect and identify. The results reveal that the proposed approach outperforms existing solutions for detecting single anomaly types at low magnitudes and detecting mixed anomaly types in all settings. Furthermore, this research is envisioned to help detect and identify anomalies early and efficiently promote safer and more resilient CAVs.

show abstract

A Dynamic Deep Reinforcement Learning-Bayesian Framework for Anomaly Detection

Cited by 16 publications

References 25 publications

Physics-Informed Machine Learning for Data Anomaly Detection, Classification, Localization, and Mitigation: A Review, Challenges, and Path Forward

Physics-Informed Machine Learning for Data Anomaly Detection, Classification, Localization, and Mitigation: A Review, Challenges, and Path Forward

Anomaly Detection in Connected and Autonomous Vehicles: A Survey, Analysis, and Research Challenges

Single and Mixed Sensory Anomaly Detection in Connected and Automated Vehicle Sensor Networks

Contact Info

Product

Resources

About