Deep Denoising For Scientific Discovery: A Case Study In Electron Microscopy

Mohan, S.; Manzorro, Ramón; Vincent, Joshua; Tang, Binh; Sheth, Dev Yashpal; Simoncelli, Eero P.; Matteson, David S.; Crozier, Peter A.; Fernandez‐Granda, Carlos

doi:10.48550/arxiv.2010.12970

Cited by 4 publications

(11 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Ground-truth images are not available in TEM, because measuring at high SNR is often impossible. Prior works have addressed this by using simulated training data [38,59], whereas others have trained CNNs directly on noisy real data [51].…”

Section: Application To Electron Microscopymentioning

confidence: 99%

“…Dataset. We use the training set of 5583 simulated images and the test set of 40 real TEM images from [38,59]. The data correspond to a catalytic platinum nanoparticle on a CeO 2 support (Section B).…”

Section: Application To Electron Microscopymentioning

confidence: 99%

“…We use the simulated TEM image dataset introduced in Ref. [38] for pre-training CNNs. The simulated dataset contains 1024 x 1024 images, which are binned to match the approximate pixel size of the experimentally acquired real image series (described below).…”

Section: Simulated Transmission Electron Microscopy Datamentioning

confidence: 99%

“…The real data consist of a series of images of the Pt/CeO2 catalyst. The images were acquired in a N2 gas atmosphere using an aberration-corrected FEI Titan transmission electron microscope (TEM), operated at 300 kV and coupled with a Gatan K2 IS direct electron detector [38]. The detector was operated in electron counting mode with a time resolution of 0.025 sec/frame and an incident electron dose rate of 5,000 e − /Å 2 /s.…”

Section: Simulated Transmission Electron Microscopy Datamentioning

confidence: 99%

“…Our experiments are carried out on 400 × 400 patches extracted from about 5000 simulated TEM introduced in Ref. [38]. The training set is augmented via downsampling, random flips, and random rotations of patches in these images [38,59].…”

Section: C4 Electron Microscope Datamentioning

confidence: 99%

See 4 more Smart Citations

Adaptive Denoising via GainTuning

Mohan,

Vincent,

Manzorro

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Deep convolutional neural networks (CNNs) for image denoising are usually trained on large datasets. These models achieve the current state of the art, but they have difficulties generalizing when applied to data that deviate from the training distribution. Recent work has shown that it is possible to train denoisers on a single noisy image. These models adapt to the features of the test image, but their performance is limited by the small amount of information used to train them. Here we propose "GainTuning", in which CNN models pre-trained on large datasets are adaptively and selectively adjusted for individual test images. To avoid overfitting, GainTuning optimizes a single multiplicative scaling parameter (the "Gain") of each channel in the convolutional layers of the CNN. We show that GainTuning improves state-of-the-art CNNs on standard image-denoising benchmarks, boosting their denoising performance on nearly every image in a held-out test set. These adaptive improvements are even more substantial for test images differing systematically from the training data, either in noise level or image type. We illustrate the potential of adaptive denoising in a scientific application, in which a CNN is trained on synthetic data, and tested on real transmission-electronmicroscope images. In contrast to the existing methodology, GainTuning is able to faithfully reconstruct the structure of catalytic nanoparticles from these data at extremely low signal-to-noise ratios.Preprint. Under review.

show abstract

Section: Application To Electron Microscopymentioning

confidence: 99%

Section: Application To Electron Microscopymentioning

confidence: 99%

Section: Simulated Transmission Electron Microscopy Datamentioning

confidence: 99%

Section: Simulated Transmission Electron Microscopy Datamentioning

confidence: 99%

Section: C4 Electron Microscope Datamentioning

confidence: 99%

See 3 more Smart Citations

Adaptive Denoising via GainTuning

Mohan,

Vincent,

Manzorro

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

Developing Deep Neural Network-based Denoising Techniques for Time-Resolved In Situ TEM of Catalyst Nanoparticles

et al. 2021

Self Cite

View full text Add to dashboard Cite

A deep learning-based convolutional neural network has been developed to denoise atomicresolution in situ TEM image datasets of catalyst nanoparticles acquired on high speed, direct electron counting detectors, where the signal is severely limited by shot noise. The network was applied to a model catalyst of CeO2-supported Pt nanoparticles. We leverage multislice simulation to generate a large and flexible dataset for training and testing the network. The proposed network outperforms state-of-the-art denoising methods by a significant margin on both simulated and experimental test data. Factors contributing to the performance are identified, including most importantly (a) the geometry of the images used during training and (b) the size of the network's receptive field. Through a gradient-based analysis, we investigate the mechanisms used by the network to denoise experimental images. This shows the network exploits information on the surrounding structure and that it adapts its filtering approach when it encounters atomic-level defects at the catalyst surface. Extensive analysis has been done to characterize the network's ability to correctly predict the exact atomic structure at the catalyst surface. Finally, we develop an approach based on the log-likelihood ratio test that provides a quantitative measure of uncertainty regarding the atomic-level structure in the network-denoised image.

show abstract

Defect Detection in Atomic Resolution Transmission Electron Microscopy Images Using Machine Learning

et al. 2021

View full text Add to dashboard Cite

Point defects play a fundamental role in the discovery of new materials due to their strong influence on material properties and behavior. At present, imaging techniques based on transmission electron microscopy (TEM) are widely employed for characterizing point defects in materials. However, current methods for defect detection predominantly involve visual inspection of TEM images, which is laborious and poses difficulties in materials where defect related contrast is weak or ambiguous. Recent efforts to develop machine learning methods for the detection of point defects in TEM images have focused on supervised methods that require labeled training data that is generated via simulation. Motivated by a desire for machine learning methods that can be trained on experimental data, we propose two self-supervised machine learning algorithms that are trained solely on images that are defect-free. Our proposed methods use principal components analysis (PCA) and convolutional neural networks (CNN) to analyze a TEM image and predict the location of a defect. Using simulated TEM images, we show that PCA can be used to accurately locate point defects in the case where there is no imaging noise. In the case where there is imaging noise, we show that incorporating a CNN dramatically improves model performance. Our models rely on a novel approach that uses the residual between a TEM image and its PCA reconstruction.

show abstract

Deep Denoising For Scientific Discovery: A Case Study In Electron Microscopy

Cited by 4 publications

References 48 publications

Adaptive Denoising via GainTuning

Adaptive Denoising via GainTuning

Developing Deep Neural Network-based Denoising Techniques for Time-Resolved In Situ TEM of Catalyst Nanoparticles

Defect Detection in Atomic Resolution Transmission Electron Microscopy Images Using Machine Learning

Contact Info

Product

Resources

About