Advancing electron microscopy using deep learning

Abstract

Electron microscopy, a sub-field of microanalysis, is a victim of its own success. The widespread use of electron microscopy for imaging molecules and materials has had an enormous impact on our understanding of countless systems and has accelerated impacts in drug discovery and materials design, for electronic, energy, environment and health applications. With this success a bottleneck has emerged, as the rate at which we can collect data has significantly exceeded the rate at which we can analyse it. Fortunately, this has coincided with the rise of advanced computational methods, including data science and machine learning. Deep learning, a sub-field of machine learning capable of learning from large quantities of data such as images, is ideally suited to overcome some of the challenges of electron microscopy at scale. There are a variety of different deep learning approaches relevant to the field, with unique advantages and disadvantages. In this review, we describe some well-established methods, with some recent examples, and introduce some new methods currently emerging in computer science. Our summary of deep learning is designed to guide electron microscopists to choose the right deep learning algorithm for their research and prepare for their digital future.

Simultaneous 2D Projection and 3D Topographic Imaging of Gas-Dependent Dynamics of Catalytic Nanoparticles

“…We believe that the types of observation shown here offer exciting opportunities if combined with quantification at a statistical level to explore the dynamics of a large population of supported nanoparticles obtained from combined 2D and 3D imaging. Approaches such as deep learning − would sample a larger range of individual particle behaviors and yield more quantitative statistical conclusions. Future studies could calculate in advance the optimal area to image in STEM to balance dose, image quality and number of particles acquired, and minimize dose and acquisition time with advanced scan techniques yielding quantitative results beyond the more qualitative outcomes reported here.…”