Developing and Evaluating Deep Neural Network-Based Denoising for Nanoparticle TEM Images with Ultra-Low Signal-to-Noise

Vincent, Joshua; Manzorro, Ramón; Mohan, S.; Tang, Binh; Sheth, Dev Yashpal; Simoncelli, Eero P.; Matteson, David S.; Fernandez‐Granda, Carlos; Crozier, Peter A.

doi:10.1017/s1431927621012678

Cited by 30 publications

(30 citation statements)

References 46 publications

(44 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Ground-truth images are not available in TEM, because measuring at high SNR is often impossible. Prior works have addressed this by using simulated training data [38,59], whereas others have trained CNNs directly on noisy real data [51].…”

Section: Application To Electron Microscopymentioning

confidence: 99%

“…Dataset. We use the training set of 5583 simulated images and the test set of 40 real TEM images from [38,59]. The data correspond to a catalytic platinum nanoparticle on a CeO 2 support (Section B).…”

Section: Application To Electron Microscopymentioning

confidence: 99%

“…[38]. The training set is augmented via downsampling, random flips, and random rotations of patches in these images [38,59].…”

Section: C4 Electron Microscope Datamentioning

confidence: 99%

See 2 more Smart Citations

Adaptive Denoising via GainTuning

Mohan,

Vincent,

Manzorro

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Deep convolutional neural networks (CNNs) for image denoising are usually trained on large datasets. These models achieve the current state of the art, but they have difficulties generalizing when applied to data that deviate from the training distribution. Recent work has shown that it is possible to train denoisers on a single noisy image. These models adapt to the features of the test image, but their performance is limited by the small amount of information used to train them. Here we propose "GainTuning", in which CNN models pre-trained on large datasets are adaptively and selectively adjusted for individual test images. To avoid overfitting, GainTuning optimizes a single multiplicative scaling parameter (the "Gain") of each channel in the convolutional layers of the CNN. We show that GainTuning improves state-of-the-art CNNs on standard image-denoising benchmarks, boosting their denoising performance on nearly every image in a held-out test set. These adaptive improvements are even more substantial for test images differing systematically from the training data, either in noise level or image type. We illustrate the potential of adaptive denoising in a scientific application, in which a CNN is trained on synthetic data, and tested on real transmission-electronmicroscope images. In contrast to the existing methodology, GainTuning is able to faithfully reconstruct the structure of catalytic nanoparticles from these data at extremely low signal-to-noise ratios.Preprint. Under review.

show abstract

Section: Application To Electron Microscopymentioning

confidence: 99%

Section: Application To Electron Microscopymentioning

confidence: 99%

See 1 more Smart Citation

Adaptive Denoising via GainTuning

Mohan,

Vincent,

Manzorro

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Thus, in situ observation requires a low-dose electron observation technique. Denoising methods based on sparse coding ( Anada et al, 2019 ) and a deep neural network ( Katsuno et al, 2021 ; Vincent et al, 2021 ) have been developed in addition to ordinary methods, and these methods are expected to enable observations with a low dose of electrons in the future. However, there are a few issues specific to in situ observation.…”

Section: Introductionmentioning

confidence: 99%

Early Detection of Nucleation Events From Solution in LC-TEM by Machine Learning

et al. 2022

View full text Add to dashboard Cite

To support the detection, recording, and analysis of nucleation events during in situ observations, we developed an early detection system for nucleation events observed using a liquid-cell transmission electron microscope. Detectability was achieved using the machine learning equivalent of detection by humans watching a video numerous times. The detection system was applied to the nucleation of sodium chloride crystals from a saturated acetone solution of sodium chlorate. Nanoparticles with a radius of more greater than 150 nm were detected in a viewing area of 12 μm × 12 μm by the detection system. The analysis of the change in the size of the growing particles as a function of time suggested that the crystal phase of the particles with a radius smaller than 400 nm differed from that of the crystals larger than 400 nm. Moreover, the use of machine learning enabled the detection of numerous nanometer sized nuclei. The nucleation rate estimated from the machine-learning-based detection was of the same order as that estimated from the detection using manual procedures.

show abstract

“…In fact, machine learning is a nascent tool in electron microscopy that is envisioned to have a large potential for quantitative image analysis [1,2]. In electron microscopy, applications of machine learning have up to now included segmentation of medical images [3], grain and phase identification [4,5,6], noise filtering [7,8] and in-plane location of atoms [9,10,11]. Moreover, Ede et al…”

Section: Introductionmentioning

confidence: 99%

Reconstructing the exit wave in high-resolution transmission electron microscopy using machine learning

Larsen,

Dahl,

Hansen

et al. 2021

Preprint

View full text Add to dashboard Cite

Reconstruction of the exit wave function is an important route to interpreting high-resolution transmission electron microscopy (HRTEM) images. Here we demonstrate that convolutional neural networks can be used to reconstruct the exit wave from a focal series of three HRTEM images, with a fidelity comparable to conventional exit wave reconstruction. We use a fully convolutional neural network based on the U-Net architecture, and demonstrate that we can train it on simulated exit waves and simulated HRTEM images of graphene-supported molybdenum disulphide (an industrial desulfurization catalyst). We then apply the trained network to analyse experimentally obtained images from similar samples, and obtain exit waves that clearly show the atomically resolved structure of both the MoS 2 nanoparticles and the graphene support. We also show that it is possible to successfully train the neural networks to reconstruct exit waves for more than 3800 different two-dimensional materials taken from the Computational 2D Materials Database of known and proposed two-dimensional materials.

show abstract

Developing and Evaluating Deep Neural Network-Based Denoising for Nanoparticle TEM Images with Ultra-Low Signal-to-Noise

Cited by 30 publications

References 46 publications

Adaptive Denoising via GainTuning

Adaptive Denoising via GainTuning

Early Detection of Nucleation Events From Solution in LC-TEM by Machine Learning

Reconstructing the exit wave in high-resolution transmission electron microscopy using machine learning

Contact Info

Product

Resources

About