Chance-constrained dynamic programming with application to risk-aware robotic space exploration

Ono, Masahiro; Pavone, Marco; Kuwata, Y.; Balaram, J.

doi:10.1007/s10514-015-9467-7

Cited by 112 publications

(107 citation statements)

References 37 publications

Supporting

Mentioning

100

Contrasting

Order By: Relevance

“…For a known MDP M = (S, A, P, µ 0 , r), that is when the set P is a singleton, the approximation introduced by considering Problem 1 with the cost function c safety precisely coincides with the approximation used in [11]. In [11] the probability of exiting a set of feasible (in our case, safe) states is over-approximated by the sum of probabilities of doing so in each step of the execution.…”

Section: A Cost Function For Safety-threshold Constraintsmentioning

confidence: 97%

“…In [11] the probability of exiting a set of feasible (in our case, safe) states is over-approximated by the sum of probabilities of doing so in each step of the execution. For a known MDP, the cost c(s, a, s ) is exactly the probability of entering a state in S err , and, in this case, we impose an upper bound on the sum of the probabilities for the individual steps by the total cost of the optimal policy for Problem 1.…”

Section: A Cost Function For Safety-threshold Constraintsmentioning

confidence: 99%

See 1 more Smart Citation

Robust optimal policies for Markov decision processes with safety-threshold constraints

Dimitrova

Topcu

2016

2016 IEEE 55th Conference on Decision and Control (CDC)

View full text Add to dashboard Cite

Abstract-We study the synthesis of robust optimal control policies for Markov decision processes with transition uncertainty (UMDPs) and subject to two types of constraints: (i) constraints on the worst-case, maximal total cost and (ii) safetythreshold constraints that bound the worst-case probability of visiting a set of error states. For maximal total cost constraints, we propose a state-augmentation method and a twostep synthesis algorithm to generate deterministic, memoryless optimal policies given the reward to be maximized. For safety threshold constraints, we introduce a new cost function and provide an approximately optimal solution by a reduction to an uncertain Markov decision process under a maximal total cost constraint. The safety-threshold constraints require memory and randomization for optimality. We discuss the use and the limitations of the proposed solution.

show abstract

Section: A Cost Function For Safety-threshold Constraintsmentioning

confidence: 97%

Section: A Cost Function For Safety-threshold Constraintsmentioning

confidence: 99%

Robust optimal policies for Markov decision processes with safety-threshold constraints

Dimitrova

Topcu

2016

2016 IEEE 55th Conference on Decision and Control (CDC)

View full text Add to dashboard Cite

show abstract

“…In [16], the problem of state tracking with active observation control is also tackled in a similar fashion where a Kalman-Like state estimator is developed. Next, AL problems with constraints were developed by the research community which exploited Constrained DP [17], [18] to actively classify human body states with biometric device sensing costs [19] and to operate a sensor network with communication costs [20]. In this paper, we combine this Constrained DP framework with a sophisticated Bayesian Learning tool, the EP.…”

Section: Related Workmentioning

confidence: 99%

Constrained Bayesian Active Learning of Interference Channels in Cognitive Radio Networks

Tsakmalis

Chatzinotas

Ottersten

2018

IEEE J. Sel. Top. Signal Process.

View full text Add to dashboard Cite

Abstract-In this paper, a sequential probing method for interference constraint learning is proposed to allow a centralized Cognitive Radio Network (CRN) accessing the frequency band of a Primary User (PU) in an underlay cognitive scenario with a designed PU protection specification. The main idea is that the CRN probes the PU and subsequently eavesdrops the reverse PU link to acquire the binary ACK/NACK packet. This feedback indicates whether the probing-induced interference is harmful or not and can be used to learn the PU interference constraint. The cognitive part of this sequential probing process is the selection of the power levels of the Secondary Users (SUs) which aims to learn the PU interference constraint with a minimum number of probing attempts while setting a limit on the number of harmful probing-induced interference events or equivalently of NACK packet observations over a time window. This constrained design problem is studied within the Active Learning (AL) framework and an optimal solution is derived and implemented with a sophisticated, accurate and fast Bayesian Learning method, the Expectation Propagation (EP). The performance of this solution is also demonstrated through numerical simulations and compared with modified versions of AL techniques we developed in earlier work.

show abstract

“…In particular, a chance constraint specifies an upper bound on the probability of incurring a cost higher than a given threshold. Chance constraints have been widely studied in robotics for motion planning under uncertainty [6,10,23]. While chance constraints are suitable for capturing risk corresponding to boolean events (e.g., collisions with obstacles), they do not take into account variations in the tails of cost distributions (since they are not affected by changes to the value of the cost above the given threshold).…”

Section: Introductionmentioning

confidence: 99%

How Should a Robot Assess Risk? Towards an Axiomatic Theory of Risk in Robotics

Majumdar

Pavone

2019

Springer Proceedings in Advanced Robotics

Self Cite

155

145

View full text Add to dashboard Cite

Endowing robots with the capability of assessing risk and making riskaware decisions is widely considered a key step toward ensuring safety for robots operating under uncertainty. But, how should a robot quantify risk? A natural and common approach is to consider the framework whereby costs are assigned to stochastic outcomes-an assignment captured by a cost random variable. Quantifying risk then corresponds to evaluating a risk metric, i.e., a mapping from the cost random variable to a real number. Yet, the question of what constitutes a "good" risk metric has received little attention within the robotics community. The goal of this paper is to explore and partially address this question by advocating axioms that risk metrics in robotics applications should satisfy in order to be employed as rational assessments of risk. We discuss general representation theorems that precisely characterize the class of metrics that satisfy these axioms (referred to as distortion risk metrics), and provide instantiations that can be used in applications. We further discuss pitfalls of commonly used risk metrics in robotics, and discuss additional properties that one must consider in sequential decision making tasks. Our hope is that the ideas presented here will lead to a foundational framework for quantifying risk (and hence safety) in robotics applications.

show abstract

Chance-constrained dynamic programming with application to risk-aware robotic space exploration

Cited by 112 publications

References 37 publications

Robust optimal policies for Markov decision processes with safety-threshold constraints

Robust optimal policies for Markov decision processes with safety-threshold constraints

Constrained Bayesian Active Learning of Interference Channels in Cognitive Radio Networks

How Should a Robot Assess Risk? Towards an Axiomatic Theory of Risk in Robotics

Contact Info

Product

Resources

About