Monte-Carlo-based partially observable Markov decision process approximations for adaptive sensing

Chong, Edwin K. P.; Kreucher, Chris; Hero, Alfred O.

doi:10.1109/wodes.2008.4605941

Cited by 22 publications

(9 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [34], the authors, presented a Monte Carlo based approximation methods to POMDP. Their method does not require analytical tractability; the system is modular and can be treated as plug and play.…”

Section: Some Variations and Improvements Of Pomdps Solution Methodsmentioning

confidence: 99%

Partially Observable Markov Decision Processes (POMDPs) and Wireless Body Area Networks (WBAN): A Survey

2013

KSII TIIS

View full text Add to dashboard Cite

Wireless body area network (WBAN) is a promising candidate for future health monitoring system. Nevertheless, the path to mature solutions is still facing a lot of challenges that need to be overcome. Energy efficient scheduling is one of these challenges given the scarcity of available energy of biosensors and the lack of portability. Therefore, researchers from academia, industry and health sectors are working together to realize practical solutions for these challenges. The main difficulty in WBAN is the uncertainty in the state of the monitored system. Intelligent learning approaches such as a Markov Decision Process (MDP) were proposed to tackle this issue. A Markov Decision Process (MDP) is a form of Markov Chain in which the transition matrix depends on the action taken by the decision maker (agent) at each time step. The agent receives a reward, which depends on the action and the state. The goal is to find a function, called a policy, which specifies which action to take in each state, so as to maximize some utility functions (e.g., the mean or expected discounted sum) of the sequence of rewards. A partially Observable Markov Decision Processes (POMDP) is a generalization of Markov decision processes that allows for the incomplete information regarding the state of the system. In this case, the state is not visible to the agent. This has many applications in operations research and artificial intelligence. Due to incomplete knowledge of the system, this uncertainty makes formulating and solving POMDP models mathematically complex and computationally expensive. Limited progress has been made in terms of applying POMPD to real applications. In this paper, we surveyed the existing methods and algorithms for solving POMDP in the general domain and in particular in Wireless body area network (WBAN). In addition, the papers discussed recent real implementation of POMDP on practical problems of WBAN. We believe that this work will provide valuable insights for the newcomers who would like to pursue related research in the domain of WBAN.

show abstract

Section: Some Variations and Improvements Of Pomdps Solution Methodsmentioning

confidence: 99%

Partially Observable Markov Decision Processes (POMDPs) and Wireless Body Area Networks (WBAN): A Survey

2013

KSII TIIS

View full text Add to dashboard Cite

show abstract

“…Another body of literature frames NBV planning as a partially observable Markov decision process (POMDP) (Ahmad and Yu, 2013; Butko and Movellan, 2010; Chong et al, 2008). Unlike MABs, these have the advantage of allowing non-myopic planning.…”

Section: Related Workmentioning

confidence: 99%

Placing and scheduling many depth sensors for wide coverage and efficient mapping in versatile legged robots

Brandão

Figueiredo

Kazuki

et al. 2019

The International Journal of Robotics Research

View full text Add to dashboard Cite

This article tackles the problem of designing 3D perception systems for robots with high visual requirements, such as versatile legged robots capable of different locomotion styles. In order to guarantee high visual coverage in varied conditions (e.g., biped walking, quadruped walking, ladder climbing), such robots need to be equipped with a large number of sensors, while at the same time managing the computational requirements that arise from such a system. We tackle this problem at both levels: sensor placement (how many sensors to install on the robot and where) and run-time acquisition scheduling under computational constraints (not all sensors can be acquired and processed at the same time). Our first contribution is a methodology for designing perception systems with a large number of depth sensors scattered throughout the links of a robot, using multi-objective optimization for optimal trade-offs between visual coverage and the number of sensors. We estimate the Pareto front of these objectives through evolutionary optimization, and implement a solution on a real legged robot. Our formulation includes constraints on task-specific coverage and design symmetry, which lead to reliable coverage and fast convergence of the optimization problem. Our second contribution is an algorithm for lowering the computational burden of mapping with such a high number of sensors, formulated as an information-maximization problem with several sampling techniques for speed. Our final system uses 20 depth sensors scattered throughout the robot, which can either be acquired simultaneously or optimally scheduled for low CPU usage while maximizing mapping quality. We show that, when compared with state-of-the-art robotic platforms, our system has higher coverage across a higher number of tasks, thus being suitable for challenging environments and versatile robots. We also demonstrate that our scheduling algorithm allows higher mapping performance to be obtained than with naïve and state-of-the-art methods by leveraging on measures of information gain and self-occlusion at low computational costs.

show abstract

“…This may lead to missed or lost targets as well as a lack of robustness to model mismatch. Chong [6] shows that there are significant gains to be had by using non-myopic policies, which trade off short-term performance gains for long-term benefits.…”

Section: Problem Formulationmentioning

confidence: 99%

“…However, exact DP methods generally are intractable when the state space is large as in the work considered here. Chong [6] shows that many adaptive sensing problems can be formulated as partially observable Markov decision processes (POMDP) and provides several approximate techniques. These POMDP approximate solutions allow for non-myopic sensing, but also suffer from large computational burdens for large state spaces.…”

Section: Introductionmentioning

confidence: 99%

Adaptive search for sparse dynamic targets

Newstadt

Wei

Hero

2013

2013 5th IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP)

Self Cite

View full text Add to dashboard Cite

Abstract-We consider the problem of energy constrained and noise-limited search for targets that are sparsely distributed over a large area. We propose a multiple-stage search algorithm that accounts for complex time-varying target behavior such as transitions among neighboring cells and varying target amplitudes. This work extends the adaptive resource allocation policy (ARAP) introduced in [Bashan et al, 2008] to policies with T 2 stages. The proposed search strategy is driven by minimization of a surrogate function for energy constrained mean-squared error within locations containing targets. Exact optimization of the multi-stage objective function is infeasible, but myopic optimization yields a closed-form solution. We extend the myopic solution with non-myopic considerations that save a percentage of resources for exploring the scene at large. Empirical evidence suggests that the non-myopic policy performs significantly better than the myopic solution in terms of estimation error, probability of detection, and robustness to model mismatch. Moreover, the provided search policy has low computational complexity compared to state-of-the-art dynamic programming solutions.

show abstract

Monte-Carlo-based partially observable Markov decision process approximations for adaptive sensing

Cited by 22 publications

References 19 publications

Partially Observable Markov Decision Processes (POMDPs) and Wireless Body Area Networks (WBAN): A Survey

Partially Observable Markov Decision Processes (POMDPs) and Wireless Body Area Networks (WBAN): A Survey

Placing and scheduling many depth sensors for wide coverage and efficient mapping in versatile legged robots

Adaptive search for sparse dynamic targets

Contact Info

Product

Resources

About