Stein Variational Probabilistic Roadmaps

Lambert, Alexander; Hou, Brian; Scalise, Rosario; Srinivasa, Siddhartha S.; Boots, Byron

doi:10.1109/icra46639.2022.9811656

Cited by 6 publications

(3 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We can integrate value functions learned by reinforcement learning (Haarnoja et al, 2018b) or optimal control (Lutter et al, 2021). Alternatively, we could compute the distribution for an optimal trajectory distribution by particles as in Stein variational MPC (Lambert et al, 2021) and exploit this multi-modal distribution as a guiding policy. These learned models can be afterward integrated with additional energy policies to deal with specific parts that were not covered in the RL or optimal control problem.…”

Section: Q-function In Optimal Control and Reinforcement Learningmentioning

confidence: 99%

Composable energy policies for reactive motion generation and reinforcement learning

Urain

Liu

et al. 2023

The International Journal of Robotics Research

View full text Add to dashboard Cite

In this work, we introduce composable energy policies (CEP), a novel framework for multi-objective motion generation. We frame the problem of composing multiple policy components from a probabilistic view. We consider a set of stochastic policies represented in arbitrary task spaces, where each policy represents a distribution of the actions to solve a particular task. Then, we aim to find the action in the configuration space that optimally satisfies all the policy components. The presented framework allows the fusion of motion generators from different sources: optimal control, data-driven policies, motion planning, and handcrafted policies. Classically, the problem of multi-objective motion generation is solved by the composition of a set of deterministic policies, rather than stochastic policies. However, there are common situations where different policy components have conflicting behaviors, leading to oscillations or the robot getting stuck in an undesirable state. While our approach is not directly able to solve the conflicting policies problem, we claim that modeling each policy as a stochastic policy allows more expressive representations for each component in contrast with the classical reactive motion generation approaches. In some tasks, such as reaching a target in a cluttered environment, we show experimentally that CEP additional expressivity allows us to model policies that reduce these conflicting behaviors. A field that benefits from these reactive motion generators is the one of robot reinforcement learning. Integrating these policy architectures with reinforcement learning allows us to include a set of inductive biases in the learning problem. These inductive biases guide the reinforcement learning agent towards informative regions or improve collision safety while exploring. In our work, we show how to integrate our proposed reactive motion generator as a structured policy for reinforcement learning. Combining the reinforcement learning agent exploration with the prior-based CEP, we can improve the learning performance and explore safer.

show abstract

Section: Q-function In Optimal Control and Reinforcement Learningmentioning

confidence: 99%

Composable energy policies for reactive motion generation and reinforcement learning

Urain

Liu

et al. 2023

The International Journal of Robotics Research

View full text Add to dashboard Cite

show abstract

“…SVGD has proven useful in a number of robotic applications in recent years, including control, planning, and point cloud matching [31]- [34]. SVGD has been applied to graphical models to approximate joint distributions using kernels over local node neighborhoods [35] and conditional distributions over nodes [36].…”

Section: Stein Variational Inferencementioning

confidence: 99%

“…At each timestep, we execute the first action in the trajectory and rerun the optimization, as in model predictive control (MPC). This approach is akin to a multi-robot version of Stein MPC [31].…”

Section: A Problem Formulationmentioning

confidence: 99%

Parts-Based Articulated Object Localization in Clutter Using Belief Propagation

Pavlasek

Lewis

Desingh

et al. 2020

2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

Decentralized coordination for multi-robot systems involves planning in challenging, high-dimensional spaces. The planning problem is particularly challenging in the presence of obstacles and different sources of uncertainty such as inaccurate dynamic models and sensor noise. In this paper, we introduce Stein Variational Belief Propagation (SVBP), a novel algorithm for performing inference over nonparametric marginal distributions of nodes in a graph. We apply SVBP to multi-robot coordination by modelling a robot swarm as a graphical model and performing inference for each robot. We demonstrate our algorithm on a simulated multi-robot perception task, and on a multi-robot planning task within a Model-Predictive Control (MPC) framework, on both simulated and real-world mobile robots. Our experiments show that SVBP represents multi-modal distributions better than sampling-based or Gaussian baselines, resulting in improved performance on perception and planning tasks. Furthermore, we show that SVBP's ability to represent diverse trajectories for decentralized multi-robot planning makes it less prone to deadlock scenarios than leading baselines.

show abstract