Temporal Detection of Anomalies via Actor-Critic Based Controlled Sensing

Joseph, Geethu; Gursoy, M. Cenk; Varshney, Pramod K.

doi:10.1109/globecom46510.2021.9685919

Cited by 4 publications

(1 citation statement)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Areas for future research include monitoring of sensor networks for abrupt changes, [65], or protection against adversarial attacks.…”

Section: Discussionmentioning

confidence: 99%

Deep reinforcement learning for active hypothesis testing with heterogeneous agents and cost constraints

Stamatelis¹,

Kalouptsidis²

2023

Preprint

View full text Add to dashboard Cite

<p>We consider active hypothesis testing with multiple heterogeneous agents. Each agent has access to its own set of experiments, has different action costs and forms its own beliefs. Additionally, each experiment carries a global cost, and the agents must try to keep the expected cumulative cost below a certain threshold. We study a centralized and a decentralized scenario. Under the centralized scenario, the agents are instructed how to act by a central controller. Under the decentralized scenario they communicate and exchange information over a directed graph. For each scenario we propose separate deep reinforcement learning algorithms based on proximal policy optimization. Solutions to the decentralized problem start from fully decentralised training, and progressively introduce two levels of centralisation during training. We assess the proposed algorithms in an example of anomaly detection over sensor networks, considering three different decentralised communication settings. We infer that all algorithms achieve the required accuracy level considerably faster than a single deep reinforcement learning agent, while satisfying the expected cost constraints when required. Under the assumption that the communication graph is fully connected, the decentralised agents perform just as well, and sometimes better than the centralised controller. Out of all decentralised algorithms, the one that uses a global critic is by far the best performing and can compete with the central controller even when the communication graph is not complete.</p>

show abstract

“…Areas for future research include monitoring of sensor networks for abrupt changes, [65], or protection against adversarial attacks.…”

Section: Discussionmentioning

confidence: 99%

Deep reinforcement learning for active hypothesis testing with heterogeneous agents and cost constraints

Stamatelis¹,

Kalouptsidis²

2023

Preprint

View full text Add to dashboard Cite

show abstract

Deep Reinforcement Learning for Anomaly Detection: A Systematic Review

et al. 2022

View full text Add to dashboard Cite

Anomaly detection has been used to detect and analyze anomalous elements from data for years. Various techniques have been developed to detect anomalies. However, the most convenient one is Machine learning which is performing well but still has limitations for large-scale unlabeled datasets. Deep Reinforcement Learning (DRL) based techniques outperform the existing supervised or unsupervised and other alternative techniques for anomaly detection. This study presents a Systematic Literature Review (SLR), which analyzes DRL models that detect anomalies in their application. This SLR aims to analyze the DRL frameworks for anomaly detection applications, proposed DRL methods, and their performance comparisons against alternative methods. In this review, we have identified 32 research articles from 2017-2022 that discuss DRL techniques for various anomaly detection applications. After analyzing the selected research articles, this paper presents 13 different applications of anomaly detection found in the selected research articles. We identified 50 different datasets applied in experiments on anomaly detection and demonstrated 17 distinct DRL models used in the selected papers to detect anomalies. Finally, we analyzed the performance of these DRL models and reviewed them. Additionally, we observed that detecting anomalies using DRL frameworks is a promising area of research and showed that DRL had shown better performance for anomaly detection where other models lack. Therefore, we provide researchers with recommendations and guidelines based on this review.

show abstract