Learning Kinematic Feasibility for Mobile Manipulation Through Deep Reinforcement Learning

Robot Learning of Mobile Manipulation with Reachability Behavior Priors

“…Recently, several RL algorithms were proposed for solving MM tasks as interactive navigation, where the hierarchical structure would be employed to decide possible sub-goals for the arm or the base in [7], [8], that would be executed either through RL policies or by motion planning respectively, both however employing precomputed policies for executing the manipulation tasks like pushing or door-opening. On the other part, [9] learns a policy that controls the base velocity using an augmented state-space while maintaining a reward function that would account for the kinematic feasibility of the next end-effector pose, which was, interestingly, sampled from the path that connected the end-effector and the goal. Learning whole-body control for MM seems to benefit from structural information, as shown in [26], [27].…”

“…SAC-hybrid, which is, in essence, the implementation of HyRL with maximum entropy exploration; SAC with no discrete action space (SAC-continuous), which resembles the method of [8], where the selection of the embodiment can be tackled as thresholding of a continuous value, but this does not affect the learning as it happens outside the MDP; iv. a variant of learning kinematic feasibility (LKF) [9], in which instead of predicting base velocities, we predict only base sub-goals, so that it can be directly comparable to ours, but there is no policy for arm activation -IK is checked at every step. Please note, that we extended all methods to consider 6D goal poses to be reached by the robot.…”

“…While significant research on MM was delivered in the last decades [1], the limitation of the proposed algorithms to structured and well-defined environments does not allow the extrapolation of such methods to unstructured realworld setups. On the other hand, recent breakthroughs in reinforcement and imitation learning [2], [3], led to an increased deployment of learning methods in robotics [4]- [6], with some early results on MM tasks too [7]- [9]. Among the many challenges of autonomous robotics, one major issue in MM is the embodiment coordination, i.e., the coordinated motion of base, arms, torso, head, etc., for accomplishing a manipulation task.…”

“…In the context of MM, classical approaches would consider planning, and a task scheduler that coordinates the robot actions [13], or an Inverse Reachability Map (IRM) that can indicate the areas where there is a higher chance for an MM robot to reach a point, leading to some greedy trialand-error of possible good base placements [14], [15]. The learning-based methods employ deep RL algorithms to learn in an end-to-end fashion MM behaviors, however leading to impractically long training times [8], [9], that could be hazardous to a real robotic system. Real-world RL training of MM was only showcased in simplistic scenarios with a reduced action space and on a setup that is not easily transferrable to robots with higher degrees of freedom and more complex structure [16].…”

Self-Supervised Learning of Multi-Object Keypoints for Robotic Manipulation

“…However, this severely limits the utility of this method for diverse applications. For example, it can neither be used for mobile robots [14], nor for manipulation tasks where a wrist camera is required for precise alignment of the gripper [15].…”