Plenoptic cameras in real-time robotics

Abstract

Real-time vision-based navigation is a difficult task largely due to the limited optical properties of single cameras that are usually mounted on robots. Multiple camera systems such as polydioptric sensors provide more efficient and precise solutions for autonomous navigation. They are particularly suitable for motion estimation because they allow one to formulate a linear optimization. These sensors capture the visual information in a more complete form called the plenoptic function that encodes the spatial and temporal light radiance of the scene. The polydioptric sensors are rarely used in robotics because they are usually thought to increase the amount of data produced and require more computational power. This paper shows that these cameras provide more accurate estimation results in mobile robotics navigation if designed properly. It also shows that a plenoptic vision sensor with a resolution ranging from 3 × 3 to 40 × 30 pixels camera, provides higher accuracy than a mono-SLAM running on a 320 × 240 pixels camera. The paper also gives a complete scheme to design usable real-time plenoptic cameras for mobile robotics applications by establishing the link between velocity, resolution and motion estimation accuracy. Finally, experiments on a mobile robot are shown allowing for a comparison between optimal plenoptic visual sensors and single high-resolution cameras. The estimation with the plenoptic sensor is more accurate than a monocular high-definition camera with a processing time 100 times lower.

Geometric Calibration of Micro-Lens-Based Light-Field Cameras Using Line Features

“…Light-field panorama [2] and disparity estimation [13,16] are also interesting applications. In the robotics field, light-field cameras composed of a camera array have shown their usefulness for visual odometry and visual SLAM [6,4]. As is widely known, the performance of these applications can be enhanced if the geometric information of handheld light-field cameras is available.…”

Reliable Planar Object Pose Estimation in Light Fields From Best Subaperture Camera Pairs

“…The information obtained by the LF camera is inherently equivalent to a set of images captured from different viewpoints, so-called sub-aperture images (SAI). SAIs, made from the transformation of the image taken by the LF camera or made from several actual camera arrays are full of interest in robotics [9], [10]. This is due to the high redundancy and strong constraints brought by the configuration of sub-aperture cameras (SAC) in a LF camera, different from general multi-view settings.…”

Benefits of plenoptic cameras for robot vision during close range on-orbit servicing maneuvers

“…Dansereau discusses methods for the calibration of plenoptic cameras as well as the processing of light fields with respect to the usage for field robotics [7]; the application of plenoptic cameras for underwater robotics with respect to challenging illumination conditions is discussed in [27]. Dong et al explore the potential of real-time vision-based navigation with plenoptic cameras for robotics [28]. Visual odometry is important for autonomous robot navigation and can be improved with plenoptic cameras as shown by Dansereau et al for underwater robotics [29] and by Zeller et al for indoor navigation [30] [31].…”