Classical and Deep Learning based Visual Servoing Systems: a Survey on State of the Art

Abstract

Computer vision, together with bayesian estimation algorithms, sensors and actuators are used in robotics to solve a variety of critical tasks such as localization, obstacle avoidance, and navigation. Visual servoing uses computer vision algorithms to guide robot movements. Classical approaches in visual servoing systems relied on extracting features from images to control robot movements. Now, state of the art computer vision systems use deep neural networks for object recognition, detection, segmentation, and tracking. These networks and specialized controllers play a predominant role in the design and implementation of modern visual servoing systems due to their accuracy, flexibility, and adaptability. Recent research in direct systems for visual servoing has created robotic systems that rely only on the information extracted from images. Furthermore, end-to-end systems eliminate entirely the controller by learning the control laws during training.This paper presents a comprehensive survey on the state of the art in visual servoing systems, discussing the latest classical methods not included in other surveys but emphasizing the new approaches based on deep neural networks and their applications within robotics.

Visual Servoing on Wheels: Robust Robot Orientation Estimation in Remote Viewpoint Control

“…This is challenging as neither depth information nor 3D model of the robot is available. Techniques that blend PBVS and IBVS are known as hybrid visual servoing [12], some of which have exploited modern Deep Learning (DL) to learn the robot control signals directly from the raw images [13]. These latter methods, however, are mainly limited in use to robot-in-hand systems.…”

Image moment-based visual positioning and robust tracking control of ultra-redundant manipulator

“…To improve the interaction ability of robots with the environment or other intelligent agents in unstructured environments, robot visual servoing control technology has attracted extensive attention from many researchers and has been widely used in positioning tasks [1],aerospace [2],ultrasound image [3] and agricultural fields [4]. Visual servoing method can be divided into three categories [5]: 1) image-based visual servoing (IBVS), 2) position-based visual servoing (PBVS), and 3) hybrid visual servo control (HVS or 2.5-D VS). IBVS achieves robot motion control directly by extracting pixel features of the target in the image plane.…”

A Real-Time Application of Singular Spectrum Analysis to Object Tracking with SIFT

“…The main problems of object tracking are common with object detection, such as occlusion, clutter in the background, viewpoint changes due to affine transformation such as translation and rotation, scale changes (zoom in and out), and photometric deformation such as distortion, blur, and illumination changes. The most important challenge is still the speed or real-time performance of the online tracker, such as the tracker used in visual servoing in robotics [3]. On the other hand, object tracking has some problems of its own.…”