Let's Push Things Forward: A Survey on Robot Pushing

Abstract

As robot make their way out of factories into human environments, outer space, and beyond, they require the skill to manipulate their environment in multifarious, unforeseeable circumstances. With this regard, pushing is an essential motion primitive that dramatically extends a robot's manipulation repertoire. In this work, we review the robotic pushing literature. While focusing on work concerned with predicting the motion of pushed objects, we also cover relevant applications of pushing for planning and control. Beginning with analytical approaches, under which we also subsume physics engines, we then proceed to discuss work on learning models from data. In doing so, we dedicate a separate section to deep learning approaches which have seen a recent upsurge in the literature. Concluding remarks and further research perspectives are given at the end of the paper. Stüber et al. Let's Push Things Forward PROBLEM STATEMENTEven in ideal conditions, such as structured environments where an agent has a complete model of the environment and perfect sensing abilities, the problems of robotic grasping and manipulation are not trivial. By complete model of the environment we mean that physical and geometric properties of the world, such as pose, shape, friction parameters and the mass of the object we wish to manipulate, are exactly known. In fact, the object to be manipulated is indirectly controlled by contacts with a robot manipulator (e.g. pushing by a contacting finger part), and an inverse model (IM), which computes an action to produce the desired motion or set of forces on the object, may not be known. Sometimes forward models (FM) may be fully or partially known, even where IMs are not available. In such cases, an FM can be used to estimate the next state of a system, given the current state and a set of executable actions. This enables planning to be 4. Physics engines. It employs a physics engine as a "black box" to make predictions about the interactions.5. Data-driven. It learns how to predict physical interaction from examples. 6. Deep learning. As the data-driven approaches, it learns how to construct an FM from examples. The key insight is that the deep learning approaches are based on feature extraction.The features highlighted for each approach are as follows.• The assumptions made by the authors on their approach. We highlight i) the quasi-static assumption in the model, ii) if it is a seminal work on 2D shapes, and iii) if the method required a known model of the object to be manipulated.• The type of motion analysed in the paper, such as 1D, planar (2D translation and 1D rotation around the x−axis), or full 3D (3D translation and 3D rotation).• The aim of the paper. We distinguish between predicting the motion of the object, estimating physical parameters, planning pushes, and analysing a push to reach a stable grasp.

Human-Robot Interaction with Robust Prediction of Movement Intention Surpasses Manual Control

“…However, despite being performed in simulations, due to three sources of noise that were added on long trajectories and the reasoning described above, this proof of principle approach should scale to real-world scenarios. Moreover, further investigation is required to extend this framework to tasks beyond grasping, such as peg-in-the-hole problems when the interactive forces arisen by the physical interaction become critical ( 27 ).…”

A model predictive approach for online mobile manipulation of non-holonomic objects using learned dynamics

“…The estimation of dynamic parameters of a manipulated object by an autonomous mobile robot has received some attention in the past (Stüber et al, 2020). Most of the existing approaches either require an extensive training dataset (Fitzpatrick et al, 2003) or use kinematics-based methods for a specific task (Vithani and Gupta, 2002).…”

Review of Deep Reinforcement Learning-Based Object Grasping: Techniques, Open Challenges, and Recommendations

“…Although this method shows effective generalization to novel objects, it is constrained in terms of complex tasks and the time scales at which these tasks should be executed. In addition, this method cannot be applied to long-term planning and is only effective for short motions [110], [111]. To overcome this limitation, the authors updated their work in another paper by using a learning-based approach of hand-eye coordination for robotic grasping from monocular images [112].…”