Pose Determination of a Robot Manipulator Based on Monocular Vision

Abstract

This paper presents an approach to determine the pose of a robot manipulator by using a single fixed camera. Conventionally, the pose determination is usually achieved by using the encoders to sense the joint angles, and then the pose of the end effector is obtained by using the direct kinematics of the manipulator. However, when the encoders or the manipulators are malfunctioning, the pose may not be accurately determined. This paper presents an approach based on machine vision, where a single camera is fixed away from the base of the manipulator. Besides, based on the kinematics of the manipulator and a calibrated camera, the pose of the manipulator can be determined. Furthermore, a graphical user interface is developed, which is convenient for users to operate the entire system. Two examples are demonstrated, and the estimated results are compared with those from the encoders. The proposed approach does not compete with the encoders. Instead, the approach can be treated as a backup method, which can provide a reference solution.INDEX TERMS Pose determination, robot manipulator, monocular vision.

Simultaneous Pose and Correspondence Determination Combining Softassign and Orthogonal Iteration

“…First, N object points (vertices) which are randomly distributed in the range of [−2, 2] × [−2, 2] × [4,8] are generated. Next, the object is rotated to an arbitrary orientation and translated to arbitrary point according to a generated pose which can be denoted by R true and T true .…”

“…Pose estimation is widely used in virtual reality [1], unmanned air vehicle navigation [2], aircrafts docking [3] and robot control [4]. In industrial measurement, point-based pose estimation is computing the pose of an object through 3D known geometry and the corresponding 2D image, which is also known as the PnP (perspective-n-point) problem [5].…”

SCARA Self Posture Recognition Using a Monocular Camera

“…R OBOTIC manipulators usually rely on data obtained from rotary encoders to determine their joint angles and to relay it to a closed-loop control system. However, rotary encoders have flaws [1], [2], due to effects of elastic joints, joint frictions, flexible links, gearbox backlashes, etc [1]. Furthermore, using an incremental rotary encoder requires an absolute position sensor, such as a hall sensor, to obtain a reference point, and requires the current position saved in memory before shutting down the system.…”

“…The authors concluded that their system is reliable in real-world applications with millimetric and near 0°errors. Kuo et al [1] proposed a single cam-era vision-based system for estimating a manipulator joint angles. The authors used visual markers and pointed at the manipulator with a fixed camera.…”

“…Given this, or to attenuate the previously mentioned problems, a vision-based joint angle estimation method is proposed using a monocular camera on the manipulator base. This method, known as visual servoing [10], [11], offers a backup in case the encoders fail [1], providing absolute angular positioning. The sensor (camera) can be used by other systems simultaneously.…”