High-Quality 3D Visualization System for Light-Field Microscopy with Fine-Scale Shape Measurement through Accurate 3D Surface Data

Abstract

We propose a light-field microscopy display system that provides improved image quality and realistic three-dimensional (3D) measurement information. Our approach acquires both high-resolution two-dimensional (2D) and light-field images of the specimen sequentially. We put forward a matting Laplacian-based depth estimation algorithm to obtain nearly realistic 3D surface data, allowing the calculation of depth data, which is relatively close to the actual surface, and measurement information from the light-field images of specimens. High-reliability area data of the focus measure map and spatial affinity information of the matting Laplacian are used to estimate nearly realistic depths. This process represents a reference value for the light-field microscopy depth range that was not previously available. A 3D model is regenerated by combining the depth data and the high-resolution 2D image. The element image array is rendered through a simplified direction-reversal calculation method, which depends on user interaction from the 3D model and is displayed on the 3D display device. We confirm that the proposed system increases the accuracy of depth estimation and measurement and improves the quality of visualization and 3D display images.

Comprehensive High-Quality Three-Dimensional Display System Based on a Simplified Light-Field Image Acquisition Method and a Full-Connected Deep Neural Network

“…The key to success is the acquisition of high-quality light field images. There are two main types of light field image acquisition techniques: single- and multicamera-based methods [ 3 , 4 , 5 , 6 , 7 , 8 , 9 , 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 , 22 ]. Single-camera methods can be divided into depth camera and lens array types.…”

“…Recently, several methods of generating high-quality 3D models from given perspectives based on convolutional neural network models have been introduced [ 15 , 16 , 23 , 24 , 25 , 26 , 27 , 28 ]. For example, some methods estimate the depth of information from the perspectives and generate 3D models for the object [ 23 , 24 ], and other methods represent the view synthesis of 2D images as 3D visualizations [ 25 , 26 , 27 , 28 ].…”

“…However, the resolution of each elemental image tends to be low as it is affected by optical aberrations of the lenses, resulting in poor image quality. Even though resolution and quality improvement methods based on convolutional neural networks have been applied, they are still insufficient for comfortable observation [13,15,16]. Multicamera-based methods, also known as camera array methods, acquire light field images with high resolution and high quality via multiple interconnected digital cameras [17][18][19][20][21][22].…”