Target object identification and localization in mobile manipulations

Abstract

Abstract-How to make mobile manipulator autonomously identify and locate target object in unknown environment, this is a very challenging question. In this paper, a multi-sensor fusion method based on camera and laser range finder (LRF) for mobile manipulations is proposed. Although the camera can acquire rich perceptual information, the image processing is very complex and easily influenced from the change in ambient light. Moreover, it can not directly provide the depth information of the environment. Since the LRF has the ability to directly measure 3D coordinates and the stability against the ambient light influence, meanwhile, the camera has the ability to acquire color information, the combination of the two sensors by making use of their advantages is utilized to obtain more accurate measurements as well as to simplify information processing. To overlay the camera image with the measurement points of the LRF pitching scan and to reconstruct the 3D image which includes the depth-of-field information, the model and the calibration of the system are built. Based on the combination of the color features extracted from the color image and the shape, size features extracted from the 3D depth-of-field image, the target object identification and localization are implemented autonomously. In order to extract the shape and size features, a triangular facet normal vector clustering (TFNVC) algorithm is introduced. The effectiveness of the proposed method and algorithm are validated by some experimental testing and analysis carried out on the mobile manipulator platform.

Joint calibration of a camera and a laser range finder from grid calibration target

“…According to the above method, the depth information is converted to the point cloud matrix. Then By making the adjacent three points of the point cloud matrix are connected into a triangular plane, a reconstructed triangular mesh is generated [10]. As shown in Fig.…”

Improvement of localization with artificial landmark for mobile manipulator

“…Jiang and Xi 8 proposed a multi-sensor fusion method based on camera and laser range finder to enhance the accuracy of mobile manipulator. This method can obtain more environmental information so that the precise position can be acquired.…”

“…Moreno et al 7 provided Markov chain Monte Carlo algorithms combined with the differential evolution methods to efficiently solve optimization problems for global localization. Jiang and Xi 8 proposed a multi-sensor fusion method based on camera and laser range finder to enhance the accuracy of mobile manipulator. This method can obtain more environmental information so that the precise position can be acquired.…”