167-PFlops Deep Learning for Electron Microscopy: From Learning Physics to Atomic Manipulation

Deep materials informatics: Applications of deep learning in materials science

“…Although there are some general guidelines for choosing an architecture for a given problem based on prior successful designs, there are no formal methods to identify the optimal architecture for a given task, and it is an open research problem. [34] • Model interpretability: Deep learning based models are generally viewed as black-box models due to being highly complex. Although researchers have tried with some success to systematically study the workings of the neural network, in general they are not as readily interpretable as some of the traditional statistical models like linear regression.…”

“…Patton et al [34] recently presented a 167-petaflop projected (152.5-petaflop measured; petaflop is a unit of computing speed, equaling 10 15 floating point operations per second) deep learning system called MENNDL to automate raw electron microscopy image based atomic defect identification and analysis on a supercomputer using 4200 nodes (with six GPUs per node). It intelligently generates and evaluates millions of deep neural networks with varying architectures and hyperparameters using a scalable, parallel, asynchronous, genetic algorithm augmented with a support vector machine to automatically find the best performing network, all in a matter of hours, which is much faster than a human expert can do.…”

“…In this paper, we discuss some of the recent advances in deep materials informatics for exploring PSPP linkages in materials, after a brief introduction to the basics of deep learning, and its challenges and opportunities. Illustrative examples of deep materials informatics that we review in this paper include learning the chemistry of materials using only elemental composition, [24] structure-aware property prediction, [25,26] crystal structure prediction, [27] learning multiscale homogenization [28,29] and localization [30] linkages in high-contrast composites, structure characterization [31,32] and quantification, [33,34] and microstructure reconstruction [35] and design. [36] We also discuss the future outlook and envisioned impact of deep learning in materials science before summarizing and concluding the paper.…”

Exascale Deep Learning to Accelerate Cancer Research

“…Utilizing MENNDL and a GPU-based HPC system, this process is automated and can take on the order of hours, while typically achieving performance better than the network hand-tuned by a domain expert. In a previous work, we introduced the scalability of the original MENNDL code, demonstrating a peak performance of 167 sustained petaflops on designing a deep neural network for use on scanning transmission electron microscopy data [24]. The scalability of MENNDL relies on its asynchronous, master-worker genetic algorithm implementation, which is used to keep as many GPUs as are available busy evaluating candidate network designs over the course of the evolution.…”

“…MENNDL is wrapped around a deep learning implementation framework (e.g., PyTorch, TensorFlow), which is used to evaluate each candidate network topology and hyperparameter set by training a network's weights. Previous versions of the MENNDL software were written in C++ and utilized the Caffe framework as the deep learning backend [24]. This work uses a new version of the MENNDL software, which is written in Python and utilizes PyTorch as the deep learning backend.…”