Action Conditioned Tactile Prediction: a case study on slip prediction

Abstract

Tactile predictive models can be useful across several robotic manipulation tasks, e.g. robotic pushing, robotic grasping, slip avoidance, and in-hand manipulation. However, available tactile prediction models are mostly studied for image-based tactile sensors and there is no comparison study indicating the best performing models. In this paper, we presented two novel data-driven action-conditioned models for predicting tactile signals during real-world physical robot interaction tasks (1) action condition tactile prediction and (2) action conditioned tactile-video prediction models. We use a magnetic-based tactile sensor that is challenging to analyse and test state-of-the-art predictive models and the only existing bespoke tactile prediction model. We compare the performance of these models with those of our proposed models. We perform the comparison study using our novel tactile enabled dataset containing 51,000 tactile frames of a real-world robotic manipulation task with 11 flat-surfaced household objects. Our experimental results demonstrate the superiority of our proposed tactile prediction models in terms of qualitative, quantitative and slip prediction scores.

Deep Functional Predictive Control for Strawberry Cluster Manipulation using Tactile Prediction

“…The core of such controllers is a forward model that can generate predicted tactile readings (we call them tactile images). For instance, action-conditioned tactile predictive models are utilised with a taxel-based tactile sensor in pick and place tasks [10], demonstrating the approach performs well only for flat surface objects.…”

“…Our approach uses a time-series model for tactile prediction based on [10]. We form a deep Predictive Functional Control (d-FPC) [20], [21] which enables the robot to control the strawberry pushing actions.…”

“…a 2D problem). While interactive movement primitives [10] can be used to plan pushing actions, an accurate interaction model is crucial for effectively controlling the planned motion of the strawberry during pushing in this scenario.…”

Tactile-Sensing Technologies: Trends, Challenges and Outlook in Agri-Food Manipulation

“…More recent works have attempted to improve robustness, safety and trajectory optimisation through slip prediction [148]. Where tactile sensations are predicted into the future during object manipulation [149], this tactile prediction can then be classified into a predictive slip signal, with which the robot can adapt its path in an optimal manner to avoid future slip [150].…”