Imaging error compensation method for through‐focus scanning optical microscopy images based on deep learning

Abstract

Through-focus scanning optical microscopy (TSOM) is a model-based nanoscale metrology technique which combines conventional bright-field microscopy and the relevant numerical simulations. A TSOM image is generated after throughfocus scanning and data processing. However, the mechanical vibration and optical noise introduced into the TSOM image during image generation can affect the measurement accuracy. To reduce this effect, this paper proposes a imaging error compensation method for the TSOM image based on deep learning with U-Net.Here, the simulated TSOM image is regarded as the ground truth, and the U-Net is trained using the experimental TSOM images by means of a supervised learning strategy. The experimental TSOM image is first encoded and then decoded with the U-shaped structure of the U-Net. The difference between the experimental and simulated TSOM images is minimised by iteratively updating the weights and bias factors of the network, to obtain the compensated TSOM image. The proposed method is applied for optimising the TSOM images for nanoscale linewidth estimation. The results demonstrate that the proposed method performs as expected and provides a significant enhancement in accuracy.

3D microscopy in industrial measurements

“…Different from computational microscopy that mines hidden information from data based on physical models, deep learning algorithms, which are also a research hotspot in microscopy in recent years, describe data from a mathematical perspective and mine information using a data‐driven paradigm. Deep learning algorithms can be versatile and efficient in solving many computational inverse problems, including image quality improvement, 137–140 breaking through the optical diffraction limit, 141 phase retrieval, 142 imaging through a scattering medium 143 and optical tomography 144 . The success of deep learning is attributable to the deep mining of prior knowledge in measurements to automatically establish complex constraints and provide strong and reasonable regularisation to specific inverse problems in metrology systems 145 …”

Research on TSOM detection method for subsurface defects of optical components

“…Through-focus scanning optical microscopy (TSOM) is a new, fast, non-destructive micro/nanoscale measurement method based on model-based, computational imaging, which can efficiently and low-cost, non-destructive measurement of targets ranging from nanometers to micrometers in size [12][13] . The features of the TSOM image are sensitive to nanoscale size changes of the measurement target and can circumvent the diffraction limit of optical imaging [14] . The measurement sensitivity of TSOM can be comparable to typical optical scattering methods, SEM, and AFM [15] .…”

The analysis of the structural parameter influences on measurement errors in a binocular 3D reconstruction system: a portable 3D system

“… Kumar, Bansal & Saluja (2021) obtained an efficient feature point recognition method based on Shi Tomasi’s corner detection algorithm. Qu et al (2021) calculated and corrected the imaging error of the through-focus scanning microscopic image, and Jihoon, Seonghyeon & Kwanghee (2018) corrected the image distortion of the Kinect projection system, indicating that the manufacturing accuracy of imaging devices, pixel digitization, residual distortion, and other factors in the reconstruction process introduce imaging errors, which have a significant impact on error measurements.…”