A study of real-world micrograph data quality and machine learning model robustness

Abstract

Machine-learning (ML) techniques hold the potential of enabling efficient quantitative micrograph analysis, but the robustness of ML models with respect to real-world micrograph quality variations has not been carefully evaluated. We collected thousands of scanning electron microscopy (SEM) micrographs for molecular solid materials, in which image pixel intensities vary due to both the microstructure content and microscope instrument conditions. We then built ML models to predict the ultimate compressive strength (UCS) of consolidated molecular solids, by encoding micrographs with different image feature descriptors and training a random forest regressor, and by training an end-to-end deep-learning (DL) model. Results show that instrument-induced pixel intensity signals can affect ML model predictions in a consistently negative way. As a remedy, we explored intensity normalization techniques. It is seen that intensity normalization helps to improve micrograph data quality and ML model robustness, but microscope-induced intensity variations can be difficult to eliminate.

Explainable machine learning in materials science

“…5 is a random forest and the image featurization is binarized statistical image features (BSIF). Other machine learning models and image normalization methods show similar results 61 . These results signify the limitation of ML model robustness.…”

“…Ideally, ML model predictions should only depend on the microstructure content, not the microscope settings. However, results show that darker images are consistently predicted to have bigger ultimate compressive strength (UCS) values, even with image normalization 61 . The x-axis shows experiment id.…”

“…This problem is known as distribution shift and is very common in all kinds of realworld ML problems. For example, Zhong et al 61 recently showed that instrument-induced intensity variations within scanning electron microscopy (SEM) images can deviate ML predictions of material ultimate compressive strength (UCS). The authors evaluated five popular ML models and four image intensity normalization methods on hundreds of well-controlled experimental SEM images of a molecular solid material.…”

Leveraging EBSD data by deep learning for bainite, ferrite and martensite segmentation

“…As revealed by table 5, the model does not seem to be sensitive to the acquisition conditions, provided that they are sufficiently good to evidence a contrast between the different phases on the model input. Machine learning model robustness over different acquisition conditions was scarcely addressed in the literature for microstructures segmentation [11,14,27]. Such approach was however not compatible with the U-Net segmentation model used in this study, as it is mostly composed of convolutional layers.…”

“…The ability of the model to generalize over different acquisition conditions (acquisition step, diffraction pattern acquisition parameters) was also addressed. Indeed, it is necessary to ensure that the developed model will work consistently whatever the EBSD/SEM setup for optimized data acquisition [11,14,27].…”