ToF cameras for active vision in robotics

Abstract

ToF cameras are now a mature technology that is widely being adopted to provide sensory input to robotic applications. Depending on the nature of the objects to be perceived and the viewing distance, we distinguish two groups of applications: those requiring to capture the whole scene and those centered on an object. It will be demonstrated that it is in this last group of applications, in which the robot has to locate and possibly manipulate an object, where the distinctive characteristics of ToF cameras can be better exploited.After presenting the physical sensor features and the calibration requirements of such cameras, we review some representative works highlighting for each one which of the distinctive ToF characteristics have been more essential. Even if at low resolution, the acquisition of 3D images at frame-rate is one of the most important features, as it enables quick background/foreground segmentation. A common use is in combination with classical color cameras. We present three developed applications, using a mobile robot and a robotic arm, to exemplify with real images some of the stated advantages.

Using ToF and RGBD cameras for 3D robot perception and manipulation in human environments

“…Observe that ToF cameras offer good depth estimation of the shirt, and lots of details (even small wrinkles) can be identified. Similar results can be obtained using high resolution stereo cameras or ToF-color cameras combination [3]. On the contrary, Kinect detailed image reveals acquisition difficulties yielding some small holes in the surface.…”

“…Their main use was in robot navigation and other long range tasks, in short range tasks like object modeling and grasping, and in less extend in human activity recognition and robot-human interaction (for a complete survey see [3,6]). The appearance of RGBD cameras around 2011 revolutionized robotics applications, as the sensor is very easy to use and offers data of enough quality for most applications.…”

An affordable deep learning based solution to support pick and place robotic tasks

“…The applications of vision technology for robots in manufacturing processes include identification and locating parts, inspection, assembly, quality control, human-robot collaboration, track the objects, robot navigation and etc. The vision system for robotics can be either scene-related or object-related [2]. The scene-oriented applications involve pathfinding, obstacle avoidance, localization, mapping for mobile robots.…”

Locomotion Strategy Selection for a Hybrid Mobile Robot Using Time of Flight Depth Sensor

“…Nowadays, ToF devices are broadly utilized in humancomputer interaction, computer games and graphics, robotics, activity recognition, and so forth [24][25][26][27][28]. In [29], Alenyà et al review applications of ToF sensors for vision systems in robotics pointing out the distinctive features ToF sensors for each of the discussed works. General characteristics of Fotonic E70 [30], MESA SR4000 [31], Brainvision BV-TOF [32], Argos 3D p100 [33], Microsoft Kinect [34], and SoftKinetic DS325 [35], commercial offthe-shelf depth sensors, are given in Table 1 with their illustrations in Figure 3.…”