Optimal Image-Based Guidance of Mobile Manipulators using Direct Visual Servoing

Abstract

This paper presents a direct image-based controller to perform the guidance of a mobile manipulator using image-based control. An eye-in-hand camera is employed to perform the guidance of a mobile differential platform with a seven degrees-of-freedom robot arm. The presented approach is based on an optimal control framework and it is employed to control mobile manipulators during the tracking of image trajectories taking into account robot dynamics. The direct approach allows us to take both the manipulator and base dynamics into account. The proposed image-based controllers consider the optimization of the motor signals sent to the mobile manipulator during the tracking of image trajectories by minimizing the control force and torque. As the results show, the proposed direct visual servoing system uses the eye-in-hand camera images for concurrently controlling both the base platform and robot arm. The use of the optimal framework allows us to derive different visual controllers with different dynamical behaviors during the tracking of image trajectories.

Visual Servoing in Robotics

“…In this case, the dynamic model parameters are identified by using a nonlinear optimisation technique. In [3], a direct image-based visual servoing system is used for the guidance of a mobile manipulator. This approach considers not only kinematic properties of the robot, but also dynamic ones for guiding both the robot base and the manipulator arm.…”

Dynamic visual servoing with Kalman filter-based depth and velocity estimator

“…The concept of IBVS is exploited to find o dT j . Similar to the derivation of equations ( 10) and (11), one can calculate the corresponding image Jacobian L that describes the relationship between the virtual velocity screw c V vir ¼ u T ðtÞ ! T ðtÞ Â Ã T of P o and the time derivative of image feature m using the following equation…”

“…Although many approaches on visual servoing have been proposed, [6][7][8][9][10][11][12] only a few of them have taken into account system dynamics in the control design of a visual servoing system. 6,7,11 For a robotic system involving highly nonlinear dynamics, its control performance will not be satisfactory unless the nonlinear dynamics of the system is carefully dealt with. In the work by Corke and Good, 6,7 the dynamics issue of a visual servoing system is investigated and the idea of feedforward control is exploited to cope with the vision latency problem.…”

“…[1][2][3][4][5] Despite visual servoing systems having many attractive features, their performance has been hindered by issues such as camera calibration error, nonlinear dynamics, and vision latency. Although many approaches on visual servoing have been proposed, [6][7][8][9][10][11][12] only a few of them have taken into account system dynamics in the control design of a visual servoing system. 6,7,11 For a robotic system involving highly nonlinear dynamics, its control performance will not be satisfactory unless the nonlinear dynamics of the system is carefully dealt with.…”

Visual servoing of robot manipulator with weak field-of-view constraints

“…To overcome the shortcomings of the trajectory planning method to solve the problem of "FOV constraints," researchers at home and abroad put forward a series of nontrajectory planning methods. By combining different visual servo structures or designing a new VS structure, the object is always in the camera FOV, such as combining IBVS and PBVS approach, 21 switching approach, 22 zooming VS approach, 23 direct VS approach, 24 and IBVS with a Kalman-neural-network filtering approach. 25 Besides, some advanced control laws are designed to deal with the FOV constraint problem, such as sliding mode control, 26,27 model predictive control, [28][29][30] prescribed performance control, 31,32 switched controller, 6,33 and so on.…”