Coupling active depth estimation and visual servoing via a large projection operator

Abstract

The goal of this paper is to propose a coupling between the execution of a Image-Based Visual Servoing (IBVS) task and an active Structure from Motion (SfM) strategy. The core idea is to modify online the camera trajectory in the null-space of the (main) servoing task for rendering the camera motion 'more informative' w.r.t. the estimation of the 3-D structure. Consequently, the SfM convergence rate and accuracy is maximized during the servoing transient. The improved SfM performance also benefits the servoing execution, since a higher accuracy in the 3-D parameters involved in the interaction matrix improves the IBVS convergence by significantly mitigating the negative effects (instability, loss of feature visibility) of a poor knowledge of the scene structure. Active maximization of the SfM performance results, in general, in a deformed camera trajectory w.r.t. what would be obtained with a classical IBVS: therefore, we also propose an adaptive strategy able to automatically activate/deactivate the SfM optimization as a function of the current level of accuracy in the estimated 3-D structure. We finally report a thorough experimental validation of the overall approach under different conditions and case studies. The reported experiments support well the theoretical analysis and clearly show the benefits of the proposed coupling between visual control and active perception.

Trifocal tensor-based 6-DOF visual servoing

“…To make the vision-based robotic system versatile, it would be interesting to simultaneously accomplish the distance estimation during the visual servoing. The active optimization strategy (Spica et al, 2017) or concurrent learning technique (Parikh et al, 2018) could be coupled into the proposed control scheme in this regard.…”

“…Specifically, an image is prerecorded in the teaching process to express a desired pose, and then by utilizing the visual feedback, the robot is driven from an initial pose to the desired pose automatically. According to the feature information used for the feedback signals, the visual servoing can be mainly divided into image-based methods (Collewet and Marchand, 2011; Dame and Marchand, 2011; Kallem et al, 2007; Liu et al, 2006; Spica et al, 2017) and pose-based methods (Fujita et al, 2007; Lippiello et al, 2007; Wilson et al, 1996).…”

AquaVis: A Perception-Aware Autonomous Navigation Framework for Underwater Vehicles

“…The work of Spica et al [20] combined visual servoing with Structure from Motion, but their primary focus was mapping and their method did not consider obstacles and operations in cluttered environments. Constante et al [21] proposed a photometric method to drive the robot close to regions with rich texture, but as with Forster et al [18], direct methods do not perform well underwater and the motions were constrained to fly-overs and near-hovering.…”

Active Estimation of 3D Lines in Spherical Coordinates

“…In order to have the remaining state dynamics we have to defineη 1 , andη 2 . Let us take the time derivative of (19):…”