Continuous real time POMCP to find-and-follow people by a humanoid service robot

Goldhoorn, Alex; Garrell, Anaís; Alquézar, René; Sanfeliu, Alberto

doi:10.1109/humanoids.2014.7041445

Cited by 25 publications

(33 citation statements)

References 19 publications

(27 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Regarding the use of machine learning techniques in related problems, Goldhoorn et al [12] proposed solving a hide-and-seek game using reinforcement learning with the aim to find a human and follow him/her to a goal location. The reward here relies on minimizing the distance between the following robot and the human during the following task.…”

Section: Related Workmentioning

confidence: 99%

Learning optimal navigation actions for foresighted robot behavior during assistance tasks

Bayoumi

Bennewitz

2016

2016 IEEE International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

We present an approach to learn optimal navigation actions for assistance tasks in which the robot aims at efficiently reaching the final navigation goal of a human where service has to be provided. Always following the human at a close distance might hereby result in inefficient trajectories, since people regularly do not move on the shortest path to their destination (e.g., they move to grab the phone or make a note). Therefore, a service robot should infer the human's intended navigation goal and compute its own motion based on that prediction. We developed an approach that applies reinforcement learning to get a Q-function that determines for each pair of the robot's and human's relative positions the best navigation action for the robot. Our approach applies a prediction of the human's motion based on a softened Markov decision process (MDP). This MDP is independent from the navigation learning framework and is learned beforehand on previously observed trajectories. We thoroughly evaluated our method in simulation and on a real robot. As the experimental results show, our approach leads to foresighted navigation behavior and significantly reduces the path length and completion time compared to naive following strategies.

show abstract

Section: Related Workmentioning

confidence: 99%

Learning optimal navigation actions for foresighted robot behavior during assistance tasks

Bayoumi

Bennewitz

2016

2016 IEEE International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

show abstract

“…POMCP has successfully been applied to problems with large discrete state and observation spaces beyond what offline algorithms can usually approach. Due to the fact that it only requires a 'blackbox' generative model of the problem to function, POMCP has also been shown to function well with continuous state spaces [42]. POMCP also acts as an online 'anytime algorithm', where computation can be cut short at some threshold and return the best answer found to that point.…”

Section: Comparison To Online Algorithmsmentioning

confidence: 99%

Optimal Continuous State POMDP Planning With Semantic Observations: A Variational Approach

2019

View full text Add to dashboard Cite

This work develops novel strategies for optimal planning with semantic observations using continuous state Partially Observable Markov Decision Processes (CPOMDPs). Two major innovations are presented in relation to Gaussian mixture (GM) CPOMDP policy approximation methods. While existing methods have many theoretically nice properties, they are hampered by the inability to efficiently represent and reason over hybrid continuous-discrete probabilistic models. The first major innovation is the derivation of closed-form variational Bayes GM approximations of Point-Based Value Iteration Bellman policy backups, using softmax models of continuous-discrete semantic observation probabilities. A key benefit of this approach is that dynamic decision-making tasks can be performed with complex non-Gaussian uncertainties, while also exploiting continuous dynamic state space models (thus avoiding cumbersome and costly discretization). The second major innovation is a new clusteringbased technique for mixture condensation that scales well to very large GM policy functions and belief functions. Simulation results for a target search and interception task with semantic observations show that the GM policies resulting from these innovations are more effective than those produced by other state of the art GM and Monte Carlo based policy approximations, but require significantly less modeling overhead and runtime cost. Additional results demonstrate the robustness of this approach to model errors.

show abstract

“…However this increased power comes at the cost of increased computational expense. Tree searches, such as Monte-Carlo Tree Search (MCTS) [9] and its variants [10] have been applied successfully to robotic systems [11], and some research has extended tree-based search strategies to handle intentions by 1…”

Section: Introductionmentioning

confidence: 99%

Interactive Decision Making for Autonomous Vehicles in Dense Traffic

Isele

2019

2019 IEEE Intelligent Transportation Systems Conference (ITSC)

View full text Add to dashboard Cite

Dense urban traffic environments can produce situations where accurate prediction and dynamic models are insufficient for successful autonomous vehicle motion planning. We investigate how an autonomous agent can safely negotiate with other traffic participants, enabling the agent to handle potential deadlocks. Specifically we consider merges where the gap between cars is smaller than the size of the ego vehicle. We propose a game theoretic framework capable of generating and responding to interactive behaviors. Our main contribution is to show how game-tree decision making can be executed by an autonomous vehicle, including approximations and reasoning that make the tree-search computationally tractable. Additionally, to test our model we develop a stochastic rule-based traffic agent capable of generating interactive behaviors that can be used as a benchmark for simulating traffic participants in a crowded merge setting.

show abstract

Continuous real time POMCP to find-and-follow people by a humanoid service robot

Cited by 25 publications

References 19 publications

Learning optimal navigation actions for foresighted robot behavior during assistance tasks

Learning optimal navigation actions for foresighted robot behavior during assistance tasks

Optimal Continuous State POMDP Planning With Semantic Observations: A Variational Approach

Interactive Decision Making for Autonomous Vehicles in Dense Traffic

Contact Info

Product

Resources

About