Cost-Bounded Active Classification Using Partially Observable Markov Decision Processes

Ahmadi, Mohamadreza; Bharadwaj, Suda; Topcu, Ufuk

doi:10.23919/acc.2019.8814415

Cited by 5 publications

(4 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The article develops a point-based value iteration algorithm that incorporates this greedy strategy to pick perception actions for each sampled belief point in each iteration. A sequential decision making process using POMDPs is studied in [WABT19]. The work aims to find strategies that actively interact with the dynamical system, and observe its reactions so that the true model is determined efficiently and with high confidence.…”

Section: Directions For Open Researchmentioning

confidence: 99%

Automated Verification and Synthesis of Stochastic Hybrid Systems: A Survey

Lavaei¹,

Soudjani²,

Abate³

2021

Preprint

View full text Add to dashboard Cite

Stochastic hybrid systems have received significant attentions as a relevant modelling framework describing many systems, from engineering to the life sciences: they enable the study of numerous applications, including transportation networks, biological systems and chemical reaction networks, smart energy and power grids, and beyond. Automated verification and policy synthesis for stochastic hybrid systems can be inherently challenging: this is due to the heterogeneity of their dynamics (presence of continuous and discrete components), the presence of uncertainty, and in some applications the large dimension of state and input sets. Over the past few years, a few hundred articles have investigated these models, and developed diverse and powerful approaches to mitigate difficulties encountered in the analysis and synthesis of such complex stochastic systems. In this survey, we overview the most recent results in the literature and discuss different approaches, including (in)finite abstractions, verification and synthesis for temporal logic specifications, stochastic similarity relations, (control) barrier certificates, compositional techniques, and a selection of results on continuous-time stochastic systems; we finally survey recently developed software tools that implement the discussed approaches. Throughout the manuscript we discuss a few open topics to be considered as potential future research directions: we hope that this survey will guide younger researchers through a comprehensive understanding of the various challenges, tools, and solutions in this enticing and rich scientific area.

show abstract

Section: Directions For Open Researchmentioning

confidence: 99%

Automated Verification and Synthesis of Stochastic Hybrid Systems: A Survey

Lavaei¹,

Soudjani²,

Abate³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…On the other hand, different techniques are used in these studies. Yaylali and Karamustafa, 13 Wu et al, 27 Dai and Tayur, 28 Denton, 29 and many other studies used MDPs. Belciug and Gorunescu 30 employed intelligent decision support systems and machine learning techniques.…”

Section: Literature Reviewmentioning

confidence: 99%

An integrated methodology to control the risk of cardiovascular disease in patients with hypertension and type 1 diabetes

Shetaban

Esfahani

Saghaei

et al. 2020

Computational Intelligence

View full text Add to dashboard Cite

Today, air pollution, smoking, use of fatty acids and ready-made foods, and so on, have exacerbated heart disease. Therefore, controlling the risk of such diseases can prevent or reduce their incidence. The present study aimed at developing an integrated methodology including Markov decision processes (MDP) and genetic algorithm (GA) to control the risk of cardiovascular disease in patients with hypertension and type 1 diabetes. First, the efficiency of GA is evaluated against Grey Wolf optimization (GWO) algorithm, and then, the superiority of GA is revealed. Next, the MDP is employed to estimate the risk of cardiovascular disease. For this purpose, model inputs are first determined using a validated micro-simulation model for screening cardiovascular disease developed at Tehran University of Medical Sciences, Iran by GA. The model input factors are then defined accordingly and using these inputs, three risk estimation models are identified. The results of these models support WHO guidelines that provide medicine with a high discount to patients with high expected LYs. To develop the MDP methodology, policies should be adopted that work well despite the difference between the risk model and the actual risk. Finally, a sensitivity

show abstract

“…detection problem for MDPs studied in this paper is relevant in applications such as medical decision-making [2], active intrusion detection [3], and recommendation systems [4]. In an MDP-based recommendation system [4], [5], the MDPs model different types of customer behavior depending on their characteristics (e.g., gender or age).…”

Section: Introductionmentioning

confidence: 99%

“…In this paper, we synthesize policies for detecting the ground truth MDP model asymptotically for multi-model MDPs (MMDPs). The authors in [3] formulate a similar problem as a general POMDP and study cost-bounded policies. However, the intrinsic detectability issue has not been addressed.…”

Section: Introductionmentioning

confidence: 99%

On the Detection of Markov Decision Processes

Duan¹,

Savas²,

Yan³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

We study the detection problem for a finite set of Markov decision processes (MDPs) where the MDPs have the same state and action spaces but possibly different probabilistic transition functions. Any one of these MDPs could be the model for some underlying controlled stochastic process, but it is unknown a priori which MDP is the ground truth. We investigate whether it is possible to asymptotically detect the ground truth MDP model perfectly based on a single observed history (stateaction sequence). Since the generation of histories depends on the policy adopted to control the MDPs, we discuss the existence and synthesis of policies that allow for perfect detection. We start with the case of two MDPs and establish a necessary and sufficient condition for the existence of policies that lead to perfect detection. Based on this condition, we then develop an algorithm that efficiently (in time polynomial in the size of the MDPs) determines the existence of policies and synthesizes one when they exist. We further extend the results to the more general case where there are more than two MDPs in the candidate set, and we develop a policy synthesis algorithm based on the breadthfirst search and recursion. We demonstrate the effectiveness of our algorithms through numerical examples.

show abstract

Cost-Bounded Active Classification Using Partially Observable Markov Decision Processes

Cited by 5 publications

References 32 publications

Automated Verification and Synthesis of Stochastic Hybrid Systems: A Survey

Automated Verification and Synthesis of Stochastic Hybrid Systems: A Survey

An integrated methodology to control the risk of cardiovascular disease in patients with hypertension and type 1 diabetes

On the Detection of Markov Decision Processes

Contact Info

Product

Resources

About