Online High-Level Model Estimation for Efficient Hierarchical Robot Navigation

Stadler, Martina; Liu, Katherine; Roy, Nicholas

doi:10.1109/iros51168.2021.9636565

Cited by 1 publication

(1 citation statement)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A more promising approach may be to directly learn the value of information based on experience. Previous methods, such as Liu et al [106], Stadler et al [167], Stein et al [168] have demonstrated that auxiliary metrics of planning success, such as likelihood of a given exploratory action succeeding, can be learned effectively from data. This style of learning framework may be a promising way of scaling VoI approximations to large and continuous problems.…”

Section: Recommendations For Future Workmentioning

confidence: 99%

Balancing exploration and exploitation: task-targeted exploration for scientific decision-making

Flaspohler¹

View full text Add to dashboard Cite

How do we collect observational data that reveal fundamental properties of scientific phenomena? This is a key challenge in modern scientific discovery. Scientific phenomena are complex—they have high-dimensional and continuous state, exhibit chaotic dynamics, and generate noisy sensor observations. Additionally, scientific experimentation often requires significant time, money, and human effort. In the face of these challenges, we propose to leverage autonomous decision-making to augment and accelerate human scientific discovery. Autonomous decision-making in scientific domains faces an important and classical challenge: balancing exploration and exploitation when making decisions under uncertainty. This thesis argues that efficient decision-making in real-world, scientific domains requires task-targeted exploration—exploration strategies that are tuned to a specific task. By quantifying the change in task performance due to exploratory actions, we enable decision-makers that can contend with highly uncertain real-world environments, performing exploration parsimoniously to improve task performance. The thesis presents three novel paradigms for task-targeted exploration that are motivated by and applied to real-world scientific problems. We first consider exploration in partially observable Markov decision processes (POMDPs) and present two novel planners that leverage task-driven information measures to balance exploration and exploitation. These planners drive robots in simulation and oceanographic field trials to robustly identify plume sources and track targets with stochastic dynamics. We next consider the exploration- exploitation trade-off in online learning paradigms, a robust alternative to POMDPs when the environment is adversarial or difficult to model. We present novel online learning algorithms that balance exploitative and exploratory plays optimally under real-world constraints, including delayed feedback, partial predictability, and short regret horizons. We use these algorithms to perform model selection for subseasonal temperature and precipitation forecasting, achieving state-of-the-art forecasting accuracy. The human scientific endeavor is poised to benefit from our emerging capacity to integrate observational data into the process of model development and validation. Realizing the full potential of these data requires autonomous decision-makers that can contend with the inherent uncertainty of real-world scientific domains. This thesis highlights the critical role that task-targeted exploration plays in efficient scientific decision-making and proposes three novel methods to achieve task-targeted exploration in real-world oceanographic and climate science applications.

show abstract

Section: Recommendations For Future Workmentioning

confidence: 99%

Balancing exploration and exploitation: task-targeted exploration for scientific decision-making

Flaspohler¹

View full text Add to dashboard Cite

show abstract

Online High-Level Model Estimation for Efficient Hierarchical Robot Navigation

Cited by 1 publication

References 22 publications

Balancing exploration and exploitation: task-targeted exploration for scientific decision-making

Balancing exploration and exploitation: task-targeted exploration for scientific decision-making

Contact Info

Product

Resources

About