SiSPRNet: end-to-end learning for single-shot phase retrieval

Abstract

With the success of deep learning methods in many image processing tasks, deep learning approaches have also been introduced to the phase retrieval problem recently. These approaches are different from the traditional iterative optimization methods in that they usually require only one intensity measurement and can reconstruct phase images in real-time. However, because of tremendous domain discrepancy, the quality of the reconstructed images given by these approaches still has much room to improve to meet the general application requirements. In this paper, we design a novel deep neural network structure named SiSPRNet for phase retrieval based on a single Fourier intensity measurement. To effectively utilize the spectral information of the measurements, we propose a new feature extraction unit using the Multi-Layer Perceptron (MLP) as the front end. It allows all pixels of the input intensity image to be considered together for exploring their global representation. The size of the MLP is carefully designed to facilitate the extraction of the representative features while reducing noises and outliers. A dropout layer is also equipped to mitigate the possible overfitting problem in training the MLP. To promote the global correlation in the reconstructed images, a self-attention mechanism is introduced to the Up-sampling and Reconstruction (UR) blocks of the proposed SiSPRNet. These UR blocks are inserted into a residual learning structure to prevent the weak information flow and vanishing gradient problems due to their complex layer structure. Extensive evaluations of the proposed model are performed using different testing datasets of phase-only images and images with linearly related magnitude and phase. Experiments were conducted on an optical experimentation platform (with defocusing to reduce the saturation problem) to understand the performance of different deep learning methods when working in a practical environment. The results demonstrate that the proposed approach consistently outperforms other deep learning methods in single-shot maskless phase retrieval. The source codes of the proposed method have been released in Github [see references].

Towards Practical Single-shot Phase Retrieval with Physics-Driven Deep Neural Network

“…For the first category, a feedforward network is used to estimate the target images directly from a Fourier intensity measurement [18], [23]- [25], [28], [30]- [32]. Specifically, [25] proposed a two-branch CNN to reconstruct the magnitude and phase part from an oversampled Fourier intensity measurement, and [30] applied this network to the 3D crystal PR problem.…”

“…Only the result of using 2 measurements is shown in the paper, and the blurring effect is quite significant, as shown in the result. Recently, our team also developed a feedforward DNN structure to tackle the PR problem [28]. It has an MLP front end for feature extraction and a residual attention-based reconstruction unit to generate the phase images.…”

“…Researchers have suggested a few solutions to the problem. One of them is by using the defocusing method [28]. Specifically, we can reduce the dynamic range of the intensity measurement by convolving it with a defocus kernel H. The convolution operation can be easily implemented by moving the camera beyond the Fourier plane.…”

“…and RAAR [7]; and eleven deep learning-based methods: ResNet [31], [33], PRCGAN [23], LearnInitNet [35], 3-scale UNet [34], LenlessNet [18], CPR-FS [24], SiSPRNet [28], NNPhase [25], and MCNN [32], prDeep [22], and HIO-UNet [27]. Similar to that in the simulation, we modified the input of some of the deep learning-based approaches since they originally could only handle much smaller size intensity measurements.…”

“…All data in the intensity image were capped to be represented by 12-bit integers to simulate the saturation problem with quantization errors. We then adopted the defocusing method [28] to mitigate the saturation problem in the intensity images. As shown in the simulation results, the proposed PPRNet significantly outperforms the state-of-the-art deep learningbased PR methods.…”