Policy Design for Active Sequential Hypothesis Testing using Deep Learning

Kartik, Dhruva; Sabir, Ekraam; Mitra, Urbashi; Natarajan, Prem

doi:10.1109/allerton.2018.8636086

Cited by 26 publications

(30 citation statements)

References 21 publications

(25 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It was shown in [10] that maximizing J g N can be formulated as a Partially Observable Markov Decision Problem (POMDP). Using heuristics for solving POMDPs, we can approximately optimize J g N and one such heuristic was presented in [10]. The strategy based on EJS divergence [9] happens to be a one-step greedy policy with respect to this POMDP.…”

Section: Discussion On Strategy Designmentioning

confidence: 99%

Active Hypothesis Testing: Beyond Chernoff-Stein

Kartik

Nayyar

Mitra

2019

2019 IEEE International Symposium on Information Theory (ISIT)

Self Cite

View full text Add to dashboard Cite

An active hypothesis testing problem is formulated. In this problem, the agent can perform a fixed number of experiments and then decide on one of the hypotheses. The agent is also allowed to declare its experiments inconclusive if needed. The objective is to minimize the probability of making an incorrect inference (misclassification probability) while ensuring that the true hypothesis is declared conclusively with moderately high probability. For this problem, lower and upper bounds on the optimal misclassification probability are derived and these bounds are shown to be asymptotically tight. In the analysis, a sub-problem, which can be viewed as a generalization of the Chernoff-Stein lemma, is formulated and analyzed. A heuristic approach to strategy design is proposed and its relationship with existing heuristic strategies is discussed.

show abstract

Section: Discussion On Strategy Designmentioning

confidence: 99%

Active Hypothesis Testing: Beyond Chernoff-Stein

Kartik

Nayyar

Mitra

2019

2019 IEEE International Symposium on Information Theory (ISIT)

Self Cite

View full text Add to dashboard Cite

show abstract

“…There is a plethora of studies related to active hypothesis testing that explore the model-based approaches in the literature [13]- [17]. Recently, the active hypothesis testing framework is combined with deep learning algorithms to design data-driven anomaly detection algorithms [11], [12], [18], [19]. However, all these algorithms assume a static setting where the states of the processes remain unchanged throughout the decision-making process.…”

Section: A Related Literaturementioning

confidence: 99%

Monitoring and Anomaly Detection Actor-Critic Based Controlled Sensing

Joseph¹,

Gursoy²,

Varshney³

2022

Preprint

View full text Add to dashboard Cite

We address the problem of monitoring a set of binary stochastic processes and generating an alert when the number of anomalies among them exceeds a threshold. For this, the decision-maker selects and probes a subset of the processes to obtain noisy estimates of their states (normal or anomalous). Based on the received observations, the decisionmaker first determines whether to declare that the number of anomalies has exceeded the threshold or to continue taking observations. When the decision is to continue, it then decides whether to collect observations at the next time instant or defer it to a later time. If it chooses to collect observations, it further determines the subset of processes to be probed. To devise this three-step sequential decision-making process, we use a Bayesian formulation wherein we learn the posterior probability on the states of the processes. Using the posterior probability, we construct a Markov decision process and solve it using deep actor-critic reinforcement learning. Via numerical experiments, we demonstrate the superior performance of our algorithm compared to the traditional model-based algorithms.

show abstract

“…We note that this approach is different from the Bayesian log likelihood ratio based-approach in [3], [4], [10]. Therefore, we define the instantaneous objective function that the agent aims to minimize at time k as follows:…”

Section: B Preferencesmentioning

confidence: 99%

Anomaly Detection via Controlled Sensing and Deep Active Inference

Joseph

Zhong

Gursoy

et al. 2020

GLOBECOM 2020 - 2020 IEEE Global Communications Conference

View full text Add to dashboard Cite

In this paper, we address the anomaly detection problem where the objective is to find the anomalous processes among a given set of processes. To this end, the decision-making agent probes a subset of processes at every time instant and obtains a potentially erroneous estimate of the binary variable which indicates whether or not the corresponding process is anomalous. The agent continues to probe the processes until it obtains a sufficient number of measurements to reliably identify the anomalous processes. In this context, we develop a sequential selection algorithm that decides which processes to be probed at every instant to detect the anomalies with an accuracy exceeding a desired value while minimizing the delay in making the decision and the total number of measurements taken. Our algorithm is based on active inference which is a general framework to make sequential decisions in order to maximize the notion of free energy. We define the free energy using the objectives of the selection policy and implement the active inference framework using a deep neural network approximation. Using numerical experiments, we compare our algorithm with the state-of-the-art method based on deep actor-critic reinforcement learning and demonstrate the superior performance of our algorithm.

show abstract

Policy Design for Active Sequential Hypothesis Testing using Deep Learning

Cited by 26 publications

References 21 publications

Active Hypothesis Testing: Beyond Chernoff-Stein

Active Hypothesis Testing: Beyond Chernoff-Stein

Monitoring and Anomaly Detection Actor-Critic Based Controlled Sensing

Anomaly Detection via Controlled Sensing and Deep Active Inference

Contact Info

Product

Resources

About