Design of a Graphical User Interface for Few-Shot Machine Learning Classification of Electron Microscopy Data

Abstract

The recent growth in data volumes produced by modern electron microscopes requires rapid, scalable, and flexible approaches to image segmentation and analysis.Few-shot machine learning, which can richly classify images from a handful of userprovided examples, is a promising route to high-throughput analysis. However, current command-line implementations of such approaches can be slow and unintuitive to use, lacking the real-time feedback necessary to perform effective classification.Here we report on the development of a Python-based graphical user interface that enables end users to easily conduct and visualize the output of few-shot learning models. This interface is lightweight and can be hosted locally or on the web, providing the opportunity to reproducibly conduct, share, and crowd-source few-shot analyses.

An Automated Scanning Transmission Electron Microscope Guided by Sparse Data Analytics

“…The application for few-shot ML analysis has been described elsewhere. 60 In brief, the application integrates Python, D3, JavaScript, HTML/CSS, and Vega-lite with Flask, a Python web framework. The front-end interactive visualization was created with JavaScript Olszta et al…”

“…WizEM is a few-shot ML application featuring a web-based Python Flask GUI. 60 It is used to classify and record the quantity and coordinates of user-defined features in images. The results of the analysis can be displayed to the user at the end of an open-loop acquisition or used as the basis for closed-loop decision making, as described in Section III B.…”

“…As data is acquired via Open-loop Control, it is passed to the WizEM application for feature classification shown in Figure 2b. The support set and model parameters for an analysis can dynamically adjusted in an interactive GUI, as described in Doty et al, 60 or be initialized from the cloud. In the typical application of the former process, the user selects one of the first few acquired frames containing microstructural features of interest.…”

“…68 For a diffraction rotation calibration, small, electron transparent platelets of varying dimension (100s nm to µm) are evaporated onto a carbon film TEM grid. 60 As shown in Figure 2a, the user first acquires a pre-defined search grid within the HubEM application. This search grid is collected with specific image overlap parameters and knowledge of the stage movements to facilitate post-acquisition stitching, as will be discussed in Section III C. The observed distribution and orientation of the particles includes individual platelets lying both parallel and perpendicular to the primary beam (termed "rod" and "plate," respectively), as well as plate clusters.…”

“…Few-shot has seen minimal usage within the materials science community, primarily in the analysis of electron backscatter diffraction (EBSD) patterns, 58 but it has great potential to inform triaging and classification tasks in novel scenarios. We have recently demonstrated the efficacy and flexibility of the few-shot approach for segmentation of electron microscope images; 59 using just 2-10 user-provided examples in an intuitive graphical user interface (GUI), 60 it is possible to quickly classify microstructural features in both atomic-resolution and lower-magnification images. The output of the few-shot approach is essentially a feature map and statistics on their relative abundance.…”