Viewpoint optimization for aiding grasp synthesis algorithms using reinforcement learning

Abstract

Grasp synthesis for unknown objects is a challenging problem as the algorithms are expected to cope with missing object shape information. This missing information is a function of the vision sensor viewpoint. The majority of the grasp synthesis algorithms in literature synthesize a grasp by using one single image of the target object and making assumptions on the missing shape information. On the contrary, this paper proposes the use of robot's depth sensor actively: we propose an active vision methodology that optimizes the viewpoint of the sensor for increasing the quality of the synthesized grasp over time. By this way we aim to relax the assumptions on the sensor's viewpoint and boost the success rates of the grasp synthesis algorithms. A reinforcement learning technique is employed to obtain a viewpoint optimization policy, and a training process and automated training data generation procedure are presented. The methodology is applied to a simple force-moment balance-based grasp synthesis algorithm, and a thousand simulations with five objects are conducted with random initial poses in which the grasp synthesis algorithm was not able to obtain a good grasp with the initial viewpoint. In 94% of these cases, the policy achieved to find a successful grasp.

Aiding Grasp Synthesis for Novel Objects Using Heuristic-Based and Data-Driven Active Vision Methods

“…Two of the data-driven methods in this paper are based on the general strategy in our prior work in Calli et al (2018a). In that work, we presented a preliminary study in simulation.…”

“…We assume an eye-in-hand system that allows us to move the camera to any viewpoint within the manipulator workspace. To reduce the dimension of the active vision algorithm's action space, the camera movement is constrained to move along a viewsphere, always pointing towards and centered around the target object [a common strategy also adopted in Paletta and Pinz (2000), Arruda et al (2016), andCalli et al (2018a)]. The radius of the viewsphere (v r ) is set based on the manipulator workspace and sensor properties.…”

“…The point cloud library (PCL) (Rusu and Cousins, 2011) has modules for calculating the normal vector and curvature which are used in the following process. As our gripper is a parallel jaw gripper, we use a force-closure-based approach similar to Calli et al (2018a), with the following constraints:…”

“…Our data-driven policies utilize a fixed-size state vector as input. This state vector is obtained by modeling the object point cloud and unexplored regions point cloud with Height Accumulated Features (HAF), developed by Fischinger and Vincze (2012) and used in Calli et al (2018a) along with the camera position. We experimented with grid sizes of 5 and 7 height maps, both of which provide similar performance in our implementation, so we chose to use 5 for performance reasons.…”

Multi-AUV Collaborative Target Recognition Based on Transfer-Reinforcement Learning

“…These algorithms increase the accuracy of target recognition by improving the correlation between multiple views. In terms of algorithm optimization, the reinforcement learning [23]- [25] algorithm extracts new features of the target by maximizing the cumulative reward training to achieve accurate target recognition in different interference environments. The transfer learning [26]- [29] algorithm can reduce the repeated calculation of the same data and improve the real-time performance of the algorithm.…”