Benefits of plenoptic cameras for robot vision during close range on-orbit servicing maneuvers

Lingenauber, Martin; Strobl, Klaus H.; Oumer, Nassir W.; Kriegel, Simon

doi:10.1109/aero.2017.7943666

Cited by 9 publications

(9 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…All of this helps for scientific, engineering and robotic tasks. It is achieved by shifting the focus operation from hardware to software, which requires more computational ressource and the development of space grade plenoptic processing as it is discussed in [40]. As a plenoptic camera can be created by modifying existing camera hardware, it is possible to gain a new type of sensor whithout the need of a complete and expensive hardware development from scratch.…”

Section: Discussionmentioning

confidence: 99%

“…As said, the real challenge in the maturation of this technology is the development of the space grade plenoptic data processing, which might also require the development of more efficient algorithms for parts of the processing chain. First results for more efficient processing were reported in literature and are for example discussed in [40].…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

In-Situ Close-Range Imaging with Plenoptic Cameras

Lingenauber

Fröhlich

Krutz

et al. 2019

2019 IEEE Aerospace Conference

Self Cite

View full text Add to dashboard Cite

This paper discusses the concept of plenoptic hand lens imagers for in-situ close-range imaging during planetary exploration missions. Hand lens imagers, such as the Mars Hand Lens Imager on-board the Mars rover Curiosity, are important cameras for in-situ investigations, e.g. of rock layer, minerals or dust. They are also important for the preparation and documentation of other instrument operations and for rover health assessment. Due to the small working distance between object and the camera's main lens, significant physical limitations affect the imaging performance. Most evident is the limited depth of field of a few millimeters for working distances of a few centimeters. This requires a highly accurate positioning of the camera and also limits the in-focus content of an image significantly. Hence, in order to have an extended object completely in focus, a sequence of images, each being focused to a different distance, is required. A single, passive camera is insufficient to compute depth from a single shot; only the combination of multiple images, either taken from different vantage points or at different focal settings, allows this. To overcome those limitations, we propose the use of plenoptic cameras as hand lens imagers. From a single exposure, they allow to create an extended depth of field image and at the same time a metric depth map while maintaining a more open aperture. These and other advantages might make it possible to omit space grade focus mechanisms in the future. A plenoptic camera is achieved by adding an additional matrix of lenslets shortly in front of the image sensor of a conventional camera. Hence, available space camera hardware can be used to form a new type of sensor. Due to its recording concept, a plenoptic camera maintains the depth of the scene as it is projected into the camera by the main lens. Thanks to the parallax between the lenslets, it is possible to compute depth via triangulation for each image point as well as a high resolution 2-D extended depth of field image. This paper provides an overview of the state of the art of hand lens imaging from which we derive a set of common requirements for future devices. We briefly introduce the plenoptic camera technology and provide first experimental results on the imaging performance based on samples of test targets and rocks. The results show that our preliminary plenoptic camera setup can comply with the requirements for in-situ hand lens imaging in terms of image quality, depth estimation and the usability for planetology.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

In-Situ Close-Range Imaging with Plenoptic Cameras

Lingenauber

Fröhlich

Krutz

et al. 2019

2019 IEEE Aerospace Conference

Self Cite

View full text Add to dashboard Cite

show abstract

“…Estimations were used for visual navigation in relation to non-cooperative targets by Gaias et al [ 2 ], Sharma et al [ 3 ] and Bennighoff et al [ 4 ]. Lingenauber et al [ 5 ] presented a plenoptic camera for autonomous robot vision during OOS missions at very close range (as close as 2 m from a satellite mockup). The use of stereo vision allows the tracking [ 6 , 7 ] and the identification [ 8 ] of an illuminated non-cooperative target.…”

Section: Introductionmentioning

confidence: 99%

Hardware-in-the-Loop Simulations with Umbra Conditions for Spacecraft Rendezvous with PMD Visual Sensors

Klionovska

Burri

2021

Sensors

View full text Add to dashboard Cite

This paper addresses the validation of a robust vision-based pose estimation technique using a Photonic Mixer Device (PMD) sensor as a single visual sensor in the close-range phase of spacecraft rendezvous. First, it was necessary to integrate the developed hybrid navigation technique for the PMD sensor into the hardware-in-the-loop (HIL) rendezvous system developed by the German Aerospace Center (DLR). Thereafter, HIL tests were conducted using the European Proximity Operation Simulator (EPOS) with sun simulation and in total darkness. For the future missions with an active sensor, e.g., a PMD camera, it could be useful to use only its own illumination during the rendezvous phase in penumbra or umbra, instead of additional flash light. In some tests, the rotational rate of the target object was also tuned. Unlike the rendezvous tests in other works, here we present for the first time closed-loop approaches with only depth and amplitude images of a PMD sensor. For the rendezvous tests in the EPOS laboratory, the Argos3D camera was used at the range of 8 to 5.5 meters; the performance showed promising results.

show abstract

“…Further explorations in this application field can be found in [57,58,59]. Also some studies focused on using the technology in space applications [60,61]. The light field features in scale space and depth proposed in [62] can also be included in this category of algorithms.…”

Section: Introductionmentioning

confidence: 99%

Implementation of a Depth from Light Field Algorithm on FPGA

Conde

Luke

Rosa

2019

Sensors

View full text Add to dashboard Cite

A light field is a four-dimensional function that grabs the intensity of light rays traversing an empty space at each point. The light field can be captured using devices designed specifically for this purpose and it allows one to extract depth information about the scene. Most light-field algorithms require a huge amount of processing power. Fortunately, in recent years, parallel hardware has evolved and enables such volumes of data to be processed. Field programmable gate arrays are one such option. In this paper, we propose two hardware designs that share a common construction block to compute a disparity map from light-field data. The first design employs serial data input into the hardware, while the second employs view parallel input. These designs focus on performing calculations during data read-in and producing results only a few clock cycles after read-in. Several experiments were conducted. First, the influence of using fixed-point arithmetic on accuracy was tested using synthetic light-field data. Also tests on actual light field data were performed. The performance was compared to that of a CPU, as well as an embedded processor. Our designs showed similar performance to the former and outperformed the latter. For further comparison, we also discuss the performance difference between our designs and other designs described in the literature.

show abstract

Benefits of plenoptic cameras for robot vision during close range on-orbit servicing maneuvers

Cited by 9 publications

References 49 publications

In-Situ Close-Range Imaging with Plenoptic Cameras

In-Situ Close-Range Imaging with Plenoptic Cameras

Hardware-in-the-Loop Simulations with Umbra Conditions for Spacecraft Rendezvous with PMD Visual Sensors

Implementation of a Depth from Light Field Algorithm on FPGA

Contact Info

Product

Resources

About