Combining Learning-Based Locomotion Policy With Model-Based Manipulation for Legged Mobile Manipulators

Abstract

Deep reinforcement learning produces robust locomotion policies for legged robots over challenging terrains. To date, few studies have leveraged model-based methods to combine these locomotion skills with the precise control of manipulators. Here, we incorporate external dynamics plans into learning-based locomotion policies for mobile manipulation. We train the base policy by applying a random wrench sequence on the robot base in simulation and add the noisified wrench sequence prediction to the policy observations. The policy then learns to counteract the partially-known future disturbance. The random wrench sequences are replaced with the wrench prediction generated with the dynamics plans from model predictive control to enable deployment. We show zero-shot adaptation for manipulators unseen during training. On the hardware, we demonstrate stable locomotion of legged robots with the prediction of the external wrench.

RoLoMa: Robust Loco-Manipulation for Quadruped Robots with Arms

“…However, the most important difference and our main contribution is that we focus on planning motions that are not only physically feasible, but that also maximize robustness against unknown external disturbances. This aspect is something that none of the previous work mentioned [1]- [3], [6], [9] have considered.…”

“…Ma et al [3] combined manipulation using model predictive control (MPC) with a locomotion policy obtained from reinforcement learning (RL). First, they modeled the wrenches (arising from the motion of the arm) applied to the base of the robot as external disturbances that can be predicted.…”

“…All of the previous work mentioned thus far [1]- [3] have one thing in common: they all see the arm as a disturbance to be compensated for. In contrast, Bellicoso et al [9] approached the problem differently, and took into account the dynamics of the whole system.…”

“…The robot experiments shown in [3] and in [6] demonstrate some capability of resistance against external forces; but this robustness is reactive, in the sense that it is either the learned locomotion policy or the MPC that compensate for the disturbances online in a reactive fashion. In contrast, research on proactive robustness [10]- [12] attempts to take uncertainty into account at the planning stage, i.e., ahead (or just in time) of online execution.…”

“…Most of the previous research done on loco-manipulation, [1]- [3], has tackled the problem by splitting the arm from the base, planning the manipulation separately from the locomotion, and then considering arm movement as a disturbance that the base should compensate for. Furthermore, most of the existing research [3]- [6] has focused on reactive robustness at the control stage, rather than taking into account robustness proactively during planning.…”