SARSOP: Efficient Point-Based POMDP Planning by Approximating Optimally Reachable Belief Spaces

Kurniawati, Hanna; Hsu, David; Lee, Wee Sun

doi:10.15607/rss.2008.iv.009

Cited by 615 publications

(677 citation statements)

References 13 publications

(19 reference statements)

Supporting

Mentioning

674

Contrasting

Unclassified

Order By: Relevance

“…We evaluate our methods using policies computed by two state-of-the-art pointbased POMDP algorithms: GapMin [12] and SARSOP [13]. GapMin returns α i , b i , a i -tuples and therefore we can compile its policies into finite-state controllers using both of our methods.…”

Section: Methodsmentioning

confidence: 99%

“…Thus, this time could be considerably shorter if one stops the planning algorithms as soon as a policy of sufficient quality is obtained. This could lead to a substantial reduction of the planning/initialization time since longer planning times (e.g., 10 4 seconds instead of 10 3 seconds in the case of SARSOP [13]) do not usually lead to dramatically improved policies.…”

Section: Methodsmentioning

confidence: 99%

“…We demonstrate our techniques on a set of large benchmark POMDP problems (Sec. 5), and we use policies generated by two state-of-the-art point-based techniques, namely GapMin [12] and SARSOP [13]. We show how we can construct very compact controllers that are equivalent, and sometimes better, than the policies they are derived from.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Controller Compilation and Compression for Resource Constrained Applications

Grzes

Poupart

Hoey

2013

Algorithmic Decision Theory

View full text Add to dashboard Cite

Abstract. Recent advances in planning techniques for partially observable Markov decision processes have focused on online search techniques and offline point-based value iteration. While these techniques allow practitioners to obtain policies for fairly large problems, they assume that a non-negligible amount of computation can be done between each decision point. In contrast, the recent proliferation of mobile and embedded devices has lead to a surge of applications that could benefit from state of the art planning techniques if they can operate under severe constraints on computational resources. To that effect, we describe two techniques to compile policies into controllers that can be executed by a mere table lookup at each decision point. The first approach compiles policies induced by a set of alpha vectors (such as those obtained by point-based techniques) into approximately equivalent controllers, while the second approach performs a simulation to compile arbitrary policies into approximately equivalent controllers. We also describe an approach to compress controllers by removing redundant and dominated nodes, often yielding smaller and yet better controllers. The compilation and compression techniques are demonstrated on benchmark problems as well as a mobile application to help Alzheimer patients to way-find.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Controller Compilation and Compression for Resource Constrained Applications

Grzes

Poupart

Hoey

2013

Algorithmic Decision Theory

View full text Add to dashboard Cite

show abstract

“…POMDPs provide a general framework for planning under uncertainty. Although solving POMDPs exactly is computationally intractable in the worst case [16], point-based approximation algorithms have greatly improved the speed of POMDP planning in recent years [12,17,20]. Today the fastest algorithms, such as HSVI [20] and SARSOP [12], can solve POMDPs with hundreds of thousands states in reasonable time.…”

Section: Related Workmentioning

confidence: 99%

“…• Our MOMDP model treats intention as a single partially observable state variable and limits uncertainty over intention to a small portion of the state space. The latest MOMDP algorithm exploits this modeling feature to achieve dramatic computational efficiency gain over standard POMDP algorithms [12,15]. The scalability of MOMDPs makes them useful for modeling moderately complex robotic tasks.…”

Section: Introductionmentioning

confidence: 99%

Intention-Aware Motion Planning

Bandyopadhyay

Won

Frazzoli

et al. 2013

Springer Tracts in Advanced Robotics

Self Cite

161

104

View full text Add to dashboard Cite

As robots venture into new application domains as autonomous vehicles on the road or as domestic helpers at home, they must recognize human intentions and behaviors in order to operate effectively. This paper investigates a new class of motion planning problems with uncertainty in human intention. We propose a method for constructing a practical model by assuming a finite set of unknown intentions. We first construct a motion model for each intention in the set and then combine these models together into a single Mixed Observability Markov Decision Process (MOMDP), which is a structured variant of the more common Partially Observable Markov Decision Process (POMDP). By leveraging the latest advances in POMDP/MOMDP approximation algorithms, we can construct and solve moderately complex models for interesting robotic tasks. Experiments in simulation and with an autonomous vehicle show that the proposed method outperforms common alternatives because of its ability in recognizing intentions and using the information effectively for decision making.

show abstract

A general optimal adaptive framework for managing a threatened species

Nicol

Ferrer‐Mestres

Lloyd

et al. 2022

Ecol Sol and Evidence

View full text Add to dashboard Cite

Managers must determine which interventions best protect threatened species when the outcomes of interventions are uncertain. Adaptive management is a dynamic optimization approach that generates optimal management actions based on current knowledge while learning to improve future management outcomes. Although adaptive management theory is well‐developed, uptake has been impeded by its complexity and a tendency to develop bespoke solutions with high implementation costs for problem‐specific returns. To increase uptake of adaptive management and improve threat management for species recovery, we developed a general adaptive management decision model, framed as a Mixed Observability Markov Decision process. We embraced principles of generality, simplicity and interpretability to overcome previous implementation challenges. We created a general model structure that is applicable to any species–threat combination, thus avoiding the need to develop customized models for every species. Simplicity was achieved by minimizing states to reduce the information requirements for parameterization. To improve interpretability, we implemented our method as a Shiny application and employed a recent artificial intelligence approach to simplify the optimal strategy. We applied our approach to a case study of fox impacts on a threatened marsupial. Our case study shows that when one management action is robust to uncertainty, the value of information of optimal adaptive management may be low. Cases like these highlight species–threat combinations where investment in adaptive management is not required. Our tool provides a rapid prototype adaptive management approach with minimal cost to management agencies. Our simple yet general model structure improves efficiency for implementing adaptive management for large numbers of threatened species, improving the effectiveness of conservation investments.

show abstract

SARSOP: Efficient Point-Based POMDP Planning by Approximating Optimally Reachable Belief Spaces

Cited by 615 publications

References 13 publications

Controller Compilation and Compression for Resource Constrained Applications

Controller Compilation and Compression for Resource Constrained Applications

Intention-Aware Motion Planning

A general optimal adaptive framework for managing a threatened species

Contact Info

Product

Resources

About