Feature Depth Observation for Image-based Visual Servoing: Theory and Experiments

Abstract

In the classical image-based visual servoing framework, error signals are directly computed from image feature parameters, allowing, in principle, control schemes to be obtained that need neither a complete three-dimensional (3D) model of the scene nor a perfect camera calibration. However, when the computation of control signals involves the interaction matrix, the current value of some 3D parameters is required for each considered feature, and typically a rough approximation of this value is used. With reference to the case of a point feature, for which the relevant 3D parameter is the depth Z, we propose a visual servoing approach where Z is observed and made available for servoing. This is achieved by interpreting depth as an unmeasurable state with known dynamics, and by building a non-linear observer that asymptotically recovers the actual value of Z for the selected feature. A byproduct of our analysis is the rigorous characterization of camera motions that actually allow such observation. Moreover, in the case of a partially uncalibrated camera, it is possible to exploit complementary camera motions in order to preliminarily estimate the focal length without knowing Z. Simulations and experimental results are presented for a mobile robot with an on-board camera in order to illustrate the benefits of integrating the depth observation within classical visual servoing schemes.

A framework for active estimation: Application to Structure from Motion

“…We focus our attention on the issue of 3D structure identification with a monocular camera, in particular on the problems of depth estimation for a point feature [1] and identification of the normal and distance to a planar scene using image moments, namely area and barycenter [2]. These two examples have been chosen because of their practical relevance and also because of their complementarity w.r.t.…”

“…For instance, one could mention [1], [2] which address the problem of recovering the structure of the scene (depth for feature points or normal vector for planar scenes) from a moving camera with known linear/angular velocity. The proposed method exploits the linearity of the state dynamics w.r.t.…”

“…VI concludes the paper with some future directions. and to the global exponential conrvergence of the unperturbed system (4) (see [1], [2] for some examples in this sense exploiting bounds on g and on e(t 0 ) ).…”

A new sensor self-calibration framework from velocity measurements

“…However, if the robotic platform is not equipped with such sensors, it is possible to use structure from motion (SFM) techniques [3], [15], signal processing methods [13], or even pose relative estimation [16]. Recent work has also shown that Z-depth can be retrieved in the case of partially uncalibrated camera [5].…”

A bio-plausible design for visual pose stabilization

“…A possible solution would be to estimate µ by solving a structure from motion problem (e.g., [17]- [19]). Our approach, instead, will show that it is possible to use a control law that does not depend on µ, if we are only concerned with local stability near q g .…”