Robust Depth Estimation for Light Field Microscopy

Abstract

Light field technologies have seen a rise in recent years and microscopy is a field where such technology has had a deep impact. The possibility to provide spatial and angular information at the same time and in a single shot brings several advantages and allows for new applications. A common goal in these applications is the calculation of a depth map to reconstruct the three-dimensional geometry of the scene. Many approaches are applicable, but most of them cannot achieve high accuracy because of the nature of such images: biological samples are usually poor in features and do not exhibit sharp colors like natural scene. Due to such conditions, standard approaches result in noisy depth maps. In this work, a robust approach is proposed where accurate depth maps can be produced exploiting the information recorded in the light field, in particular, images produced with Fourier integral Microscope. The proposed approach can be divided into three main parts. Initially, it creates two cost volumes using different focal cues, namely correspondences and defocus. Secondly, it applies filtering methods that exploit multi-scale and super-pixels cost aggregation to reduce noise and enhance the accuracy. Finally, it merges the two cost volumes and extracts a depth map through multi-label optimization.

High-Accuracy Correction of a Microlens Array for Plenoptic Imaging Sensors

“…The MLA determines the angular sampling distribution of the rays passing through a point at the main lens pupil and projects it onto the sensor pixels assigned to the corresponding microlenses to form subimages. Such a system structure can achieve maximal light-field depth resolution and provide depth constraints to estimate the object depth map without calibration [2,3]. Ideally, each microlens has an identical geometric form and a uniform arrangement, and the MLA is aligned with the image sensor.…”

“…Plenoptic imaging is an advanced computational photography technique that can obtain both spatial and angular information of light rays in a single exposure [1,2]. The retention of radiation information in the angular dimension provides the necessary light-field data for altering the viewpoints or focusing depth of the target scene, which enables the capabilities of depth estimation, dynamic capture, and volumetric reconstruction [2,3,4,5]. This means that plenoptic imaging systems can be used as sensors for making measurements in complex scenarios, such as high-temperature flames [6,7,8], 3D fluid flow fields [9,10], and organ microtopography [11].…”

Super-Resolution Enhancement Method Based on Generative Adversarial Network for Integral Imaging Microscopy

“…An optical microscope is a piece of magnification equipment for viewing and observing microscopic objects. The microscope is extensively used in many different fields, such as biomedical science, nanophysics, and medical science [ 1 , 2 , 3 ]. Generally, it consists of an objective lens and a tube lens.…”

Depth-of-Field-Extended Plenoptic Camera Based on Tunable Multi-Focus Liquid-Crystal Microlens Array

“…Unlike conventional cameras, plenoptic cameras can record the 3D information of objects (i.e., the 4D plenoptic function or radiance, also called the light-field) including the radiation power and the ray direction. The capture of high-dimensional data sets that contain rich scene information enable many unique applications of plenoptic cameras, such as digital refocusing [ 1 , 2 ], multi-view imaging [ 3 , 4 ], depth estimation [ 5 , 6 ], and 3D reconstruction [ 7 , 8 , 9 ]. The first prototype of the plenoptic camera was proposed by Adelson and Wang in 1992 [ 10 ].…”