Microstructure Cluster Analysis with Transfer Learning and Unsupervised Learning

Abstract

We apply computer vision and machine learning methods to analyze two datasets of microstructural images. A transfer learning pipeline utilizes the fully connected layer of a pre-trained convolutional neural network as the image representation. An unsupervised learning method uses the image representations to discover visually distinct clusters of images within two datasets. A minimally supervised clustering approach classifies micrographs into visually similar groups. This approach successfully classifies images both in a dataset of surface defects in steel, where the image classes are visually distinct and in a dataset of fracture surfaces that humans have difficulty classifying. We find that the unsupervised, transfer learning method gives results comparable to fully supervised, custom-built approaches.

Overview: Computer Vision and Machine Learning for Microstructural Characterization and Analysis

“…In fact, a linear support vector machine (SVM) was trained to classify the primary microconstituent in each image in the UHCS database (supervised machine learning), achieving 99 ± 1 pct accuracy (defined as fraction of correct classifications). [5] Other studies have used unsupervised and supervised ML to detect and classify defects, [93][94][95] microstructural constituents, [96] atomic structures, [97] and damage. [98] Thus, image classification contributes to a wide variety of image processing tasks including image analysis, keyword identification, and quality control.…”

“…For instance, to make the processing/structure connection, CV/ML studies have examined micrograph databases to correlate microstructure with annealing history. [34,122] Likewise, CV/ML systems have been applied to relate microstructure with outcome properties including stress hot spot [123,124] and damage formation, [98] fatigue failure initiation, [125] fracture energy, [93] ionic conductivity, [100] and fatigue strength. [122] Using the ability of CNNs to generate structures, several recent studies have also made strides toward the inverse problem of designing microstructures with target properties.…”

A transfer learning approach for improved classification of carbon nanomaterials from TEM images

“…[8][9][10][11][12][13][14][15][16] By applying computer vision techniques to materials science image datasets, surface properties, phase distributions, and structural features of materials can be captured, measured with high precision, and quantied to investigate the underlying structure-properties relationship. 17 These machine learning and computer vision methods oen used supervised learning techniques such as Support Vector Machine (SVM) 9,10,13,14 and Convolutional Neural Network (CNN), 10,11 and unsupervised dimensionality reduction techniques such as t-distributed Stochastic Neighbor Embedding (t-SNE). [9][10][11] Machine learning algorithms make predictions on unseen data using the statistical models that were trained on previously collected data, by assuming that the distributions of the two data sets are the same.…”

“…17 These machine learning and computer vision methods oen used supervised learning techniques such as Support Vector Machine (SVM) 9,10,13,14 and Convolutional Neural Network (CNN), 10,11 and unsupervised dimensionality reduction techniques such as t-distributed Stochastic Neighbor Embedding (t-SNE). [9][10][11] Machine learning algorithms make predictions on unseen data using the statistical models that were trained on previously collected data, by assuming that the distributions of the two data sets are the same. 18 Since the rst introduction of the ImageNet Large-Scale Visual Recognition Challenge (ILSVRC) 19 and AlexNet 20 with a CNN framework, [21][22][23][24] numerous attempts have been made to construct high-performance architectures for image classication, single-object localization, and object detection.…”

Predicting structure zone diagrams for thin film synthesis by generative machine learning

“…If physical models do not exist, instead of applying atomistic calculations, machine learning can provide surrogate models bridging the gap between process parameters and resulting microstructure. Machine learning evolved as a new category for microstructure cluster analysis 41,42 , microstructure recognition [43][44][45] , defect analysis 46 , materials design 47 , and materials optimization 48 . Generative deep learning models are able to produce new data based on hidden information in training data 49 .…”