Siamese Convolutional Neural Network for Sub-millimeter-accurate Camera Pose Estimation and Visual Servoing

Abstract

Visual Servoing (VS), where images taken from a camera typically attached to the robot end-effector are used to guide the robot motions, is an important technique to tackle robotic tasks that require a high level of accuracy. We propose a new neural network, based on a Siamese architecture, for highly accurate camera pose estimation. This, in turn, can be used as a final refinement step following a coarse VS or, if applied in an iterative manner, as a standalone VS on its own. The key feature of our neural network is that it outputs the relative pose between any pair of images, and does so with submillimeter accuracy. We show that our network can reduce pose estimation errors to 0.6 mm in translation and 0.4 degrees in rotation, from initial errors of 10 mm / 5 degrees if applied once, or of several cm / tens of degrees if applied iteratively. The network can generalize to similar objects, is robust against changing lighting conditions, and to partial occlusions (when used iteratively). The high accuracy achieved enables tackling low-tolerance assembly tasks downstream: using our network, an industrial robot can achieve 97.5% success rate on a VGAconnector insertion task without any force sensing mechanism.

Convolutional Neural Network based Visual Servoing for Eye-to-Hand Manipulator

“…Convolutional neural networks (CNN) have shown superior performance with state-of-the-art method in some areas, such as object identification [22], [23], camera relocalization [24], [25], and pose estimation of objects [26], [27], etc.. Recently, CNN has been applied to visual servoing scheme [28], [29], [30] in order to overcome the limitation of visual servoing, such as the requirement of hand-crafted image features, and the sensitivity to lighting conditions and occlusions.…”

“…Yu et al [30] proposed a new network based on Siamese architecture [33] for camera pose estimation to position an eye-in-hand manipulator. The network proposed by C. Yu et al [30] processes images through two branches of convolutional layers which have the same structure and weights. The network regresses the camera pose from concatenated two flattened image features that are extracted from two backbones.…”

“…DEFINet consists of two parts, the feature extraction part and the regression part. Inspired by the architecture proposed in [30], the feature extraction part consists of two networks with the same structure that share weights to process two images in parallel. In contrast to the network in [30], the difference between the two encoded features is fed into the regression part to regress the relative pose, which results in high positioning accuracy.…”

“…Inspired by the architecture proposed in [30], the feature extraction part consists of two networks with the same structure that share weights to process two images in parallel. In contrast to the network in [30], the difference between the two encoded features is fed into the regression part to regress the relative pose, which results in high positioning accuracy. The network is trained by a dataset for eye-tohand configuration ( Fig.1) generated from a sample dataset of images collected by operating a manipulator automatically for a given task space.…”

Toward Fast Convergence and Calibration-Free Visual Servoing Control: A New Image Based Uncalibrated Finite Time Control Scheme

“…Various uncalibrated visual servoing (UVS) control techniques have been proposed. Some publications present the neural networking method [3]- [6] and the genetic algorithm [7], [8] to cope with the estimation of visual nonlinear mapping. These artificial intelligence schemes employ off-line data to train the network models to approximate the nonlinear mapping models and usually require a mass of the data for the training stage, which is time-consuming.…”