Approximate dynamic programming for stochastic reachability

Kariotoglou, Nikolaos; Summers, Sean; Summers, Tyler H.; Kamgarpour, Maryam; Lygeros, John

doi:10.23919/ecc.2013.6669603

Cited by 38 publications

(51 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…State of the art methods that include (uniform) gridding reach a maximum of 5-6 dimensions and as a result research is focusing on approximate dynamic programming (ADP) techniques, (see, for example, Refs. [27,34] and the books in Refs. [35,36] for a more general approach).…”

Section: Complexity Denote By O(c)mentioning

confidence: 99%

Multi-Agent Autonomous Surveillance: A Framework Based on Stochastic Reachability and Hierarchical Task Allocation

Kariotoglou

Raimondo

Summers

et al. 2014

Journal of Dynamic Systems, Measurement, and Control

Self Cite

View full text Add to dashboard Cite

We develop and implement a framework to address autonomous surveillance problems with a collection of pan-tilt (PT) cameras. Using tools from stochastic reachability with random sets, we formulate the problems of target acquisition, target tracking, and acquisition while tracking as reach-avoid dynamic programs for Markov decision processes (MDPs). It is well known that solution methods for MDP problems based on dynamic programming (DP), implemented by state space gridding, suffer from the curse of dimensionality. This becomes a major limitation when one considers a network of PT cameras. To deal with larger problems we propose a hierarchical task allocation mechanism that allows cameras to calculate reach-avoid objectives independently while achieving tasks collectively. We evaluate the proposed algorithms experimentally on a setup involving industrial PT cameras and mobile robots as targets.

show abstract

Section: Complexity Denote By O(c)mentioning

confidence: 99%

Multi-Agent Autonomous Surveillance: A Framework Based on Stochastic Reachability and Hierarchical Task Allocation

Kariotoglou

Raimondo

Summers

et al. 2014

Journal of Dynamic Systems, Measurement, and Control

Self Cite

View full text Add to dashboard Cite

show abstract

“…Thus, the limiting factor in computation is the continuous space dimensions. Adaptive gridding [24] as well as approximation methods which do not require gridding [16,26] have been proposed to alleviate the Curse of Dimensionality.…”

Section: Proof X Satisfies the Specification Of The Automaton If Andmentioning

confidence: 99%

“…The infinite constraints may be handled either using sampling approaches [9,12,7] or sum-of-squares [28]. Currently we are exploring the application of these methods for infinite horizon dynamic programming [16,26].…”

mentioning

confidence: 99%

Control design for specifications on stochastic hybrid systems

Kamgarpour

Summers

Lygeros

2013

Proceedings of the 16th International Conference on Hybrid Systems: Computation and Control

Self Cite

View full text Add to dashboard Cite

We synthesize controllers for discrete-time stochastic hybrid systems such that the probability of satisfying a given specification on the system is maximized. The specifications are defined with finite state automata. It is shown that automata satisfaction is equivalent to a reachability problem in an extended state space consisting of the system and the automaton state spaces. The control policy is defined as a map from this extended state space to the input space. Using existing results on maximizing reachability probability, the control policy is designed to maximize probability of satisfying the specification.

show abstract

“…Let t > 0 and x ∈ D t (T , K, E). We are interested in underapproximating L t (β) = {x : V * N −t (x) ≥ β} as defined in (10). From (8),…”

Section: Conservative Approximation Of Stochastic Reach-avoid Levmentioning

confidence: 99%

Underapproximation of reach-avoid sets for discrete-time stochastic systems via Lagrangian methods

Gleason

Vinod

Oishi

2017

2017 IEEE 56th Annual Conference on Decision and Control (CDC)

View full text Add to dashboard Cite

We examine Lagrangian techniques for computing underapproximations of finite-time horizon, stochastic reachavoid level-sets for discrete-time, nonlinear systems. We use the concept of reachability of a target tube in the control literature to define robust reach-avoid sets which are parameterized by the target set, safe set, and the set in which the disturbance is drawn from. We unify two existing Lagrangian approaches to compute these sets and establish that there exists an optimal control policy of the robust reach-avoid sets which is a Markov policy. Based on these results, we characterize the subset of the disturbance space whose corresponding robust reachavoid set for the given target and safe set is a guaranteed underapproximation of the stochastic reach-avoid level-set of interest. The proposed approach dramatically improves the computational efficiency for obtaining an underapproximation of stochastic reach-avoid level-sets when compared to the traditional approaches based on gridding. Our method, while conservative, does not rely on a grid, implying scalability as permitted by the known computational geometry constraints. We demonstrate the method on two examples: a simple twodimensional integrator, and a space vehicle rendezvous-docking problem.

show abstract

Approximate dynamic programming for stochastic reachability

Cited by 38 publications

References 11 publications

Multi-Agent Autonomous Surveillance: A Framework Based on Stochastic Reachability and Hierarchical Task Allocation

Multi-Agent Autonomous Surveillance: A Framework Based on Stochastic Reachability and Hierarchical Task Allocation

Control design for specifications on stochastic hybrid systems

Underapproximation of reach-avoid sets for discrete-time stochastic systems via Lagrangian methods

Contact Info

Product

Resources

About