Deep learning super‐resolution electron microscopy based on deep residual attention network

Abstract

Field-emission scanning electron microscopy has become a fundamental research tool in the fields of medicine and materials science owing to its effectiveness. However, an inherent contradiction exists between the resolution of field-emission scanning electron microscopy and its field-of-view. To solve this problem, we propose a deep learning-based method for electron microscopy that can simultaneously obtain a large field-of-view and ultrahigh resolution.To solve the super-resolution problem, a deep residual attention network is designed based on residual learning and the attention mechanism, wherein the backbone network employs several residual groups to effectively extract the features; moreover, attention groups append the backbone part to refine and fuse the features. Besides, a high-frequency information retention module is added to acquire high-frequency signals, acting as an effective complement to the deep residual attention network. Owing to the lack of super-resolution datasets for electron microscopy, we created a microscopic butterfly wing dataset. In the experiments, MixUp was also applied to super-resolution problem as a simple and effective data augmentation method, to provide more data to train the model. To evaluate the proposed method, we used standard and self-made datasets with peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) as criteria. The results show that the proposed model exhibits a state-of-the-art performance with a PSNR of 25.12 dB and an SSIM of 0.64, and makes field-emission scanning electron microscopy more practical and promising in the field of optical device optimization.

An Adversarial Learning Approach for Super-Resolution Enhancement Based on AgCl@Ag Nanoparticles in Scanning Electron Microscopy Images

“…These methods can be divided into the following categories: (1) Model-based super-resolution (SR) [18,19]: These methods model the degradation process of images based on a prior model and regularize the reconstruction images according to the features of the projections. However, these algorithms can obtain an ideal image quality only on the premise that the model-based priors are valid; (2) Learning-based super-resolution [20][21][22]: These methods can be trained with a dataset made up of low-resolution (LR) and highresolution (HR) images pairs to learn a nonlinear mapping, then restore the missing high-frequency information and greatly enhance the image quality. Moreover, as long as the model is trained, SR images can be achieved simply through feed-forward propagation, which saves a lot of time and decreases the computational expense.…”