Abstract-Tracking is a very important research subject in a real-time augmented reality context. The main requirements for trackers are high accuracy and little latency at a reasonable cost. In order to address these issues, a real-time, robust, and efficient 3D modelbased tracking algorithm is proposed for a "video see through" monocular vision system. The tracking of objects in the scene amounts to calculating the pose between the camera and the objects. Virtual objects can then be projected into the scene using the pose. Here, nonlinear pose estimation is formulated by means of a virtual visual servoing approach. In this context, the derivation of point-to-curves interaction matrices are given for different 3D geometrical primitives including straight lines, circles, cylinders, and spheres. A local moving edges tracker is used in order to provide real-time tracking of points normal to the object contours. Robustness is obtained by integrating an M-estimator into the visual control law via an iteratively reweighted least squares implementation. This approach is then extended to address the 3D model-free augmented reality problem. The method presented in this paper has been validated on several complex image sequences including outdoor environments. Results show the method to be robust to occlusion, changes in illumination, and mistracking.

Control Issue Visual servoing techniques consist of using the data provided by one or several cameras in order to control the motion of a robotic system [13], [19]. A large variety of positioning or target tracking tasks can be implemented by controlling from one to all degrees of freedom of the system. Whatever the sensor configuration, which can vary from one camera mounted on the robot end effector to several free-standing cameras, a set of visual features s must be designed from the visual measurements x(t) (s = s(x(t))), allowing control of the desired degrees of freedom. A control law also must be designed so that these features s reach a desired value s * , defining a correct realization of the task. A desired trajectory s * (t) can also be tracked [1], [29]. The control principle is thus to regulate the error vector s − s * to zero.

Augmented reality (AR) allows to seamlessly insert virtual objects in an image sequence. In order to accomplish this goal, it is important that synthetic elements are rendered and aligned in the scene in an accurate and visually acceptable way. The solution of this problem can be related to a pose estimation or, equivalently, a camera localization process. This paper aims at presenting a brief but almost self-contented introduction to the most important approaches dedicated to vision-based camera localization along with a survey of several extension proposed in the recent years. For most of the presented approaches, we also provide links to code of short examples. This should allow readers to easily bridge the gap between theoretical aspects and practical implementations.

This report proposes a new way to achieve robotic tasks by 2D visual servoing. Indeed, instead of using classical geometric features such as points, straight lines, pose or an homography, as it is usually done, the luminance of all pixels in the image is here considered. The main advantage of this new approach is that it does not require any tracking or matching process. The key point of our approach relies on the analytic computation of the interaction matrix that links the time variation of the luminance to the camera motions. This computation is based either on a simple Lambertian model or on the Phong one so that complex illumination changes can be considered. However, since most of the classical control laws fail when considering the luminance as a visual feature, we turn the visual servoing problem into an optimization one leading to a new control law. Experimental results on positioning and tracking tasks validate the proposed approach and show its robustness regarding to approximated depths, Lambertian and non Lambertian objects, low textured objects and partial occlusions. Key-words:

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Abstract-This paper proposes a new way to achieve robotic tasks by visual servoing. Instead of using geometric features (points, straight lines, pose, homography, etc.) as it is usually done, we use directly the luminance of all pixels in the image. Since most of the classical control laws fail in this case, we turn the visual servoing problem into an optimization problem leading to a new control law. Experimental results validate the proposed approach and show its robustness regarding to approximated depths, non Lambertian objects and partial occlusions.

Abstract

To cite this version:Grégory Flandin, François Chaumette, E. Marchand. Eye-in-hand / eye-to-hand cooperation for visual servoing. IEEE Int.