Microstructure Instance Segmentation from Aluminum Alloy Metallographic Image Using Different Loss Functions

Abstract

Automatic segmentation of metallographic image is very important for the implementation of an automatic metallographic analysis system. In this paper, a novel instance segmentation framework of a metallographic image was implemented, which can assign each pixel to a physical instance of a microstructure. In this framework, we used the Mask R-CNN as the basic network to complete the learning and recognition of the latent feature of an aluminum alloy microstructure. Meanwhile, we implemented five different loss functions based on this framework and compared the influence of these loss functions on metallographic image segmentation performance. We carried out several experiments to verify the effectiveness of the proposed framework. In these experiments, we compared and analyzed six different evaluation metrics and provided constructive suggestions for the performance evaluation of metallographic image segmentation method. A large number of experimental results have shown that the proposed method can achieve the instance segmentation of an aluminum alloy metallographic image and the segmentation results are satisfactory.

The Adoption of Image-Driven Machine Learning for Microstructure Characterization and Materials Design: A Perspective

“…Paper Functional goals DeCost2015 [10] Classify microstructure images into one of seven image classes, and develop a bag of visual features Chowdhury2016 [11] Binary classification of microstructural features (dendrites) and feature orientations Zhang2021 [14] Generate isotropic and anisotropic synthetic 3D porous structures using 2D slices as input Gobert2018 [15] In situ defect detection detection using supervised machine learning, for powder bed fusion (PBF) additive manufacturing Scime2019 [16] In-situ defect detection in additive manufacturing melt pools Stan2020 [17] Semantic segmentation of computed tomography and serial sectioning images Strohmann2019 [18] Semantic segmentation of 3D microstructure of Al-Si Evsevleev2020 [19] Deep-learning based semantic segmentation of individual phases from synchrotron x-ray computed tomography images Tsopanidis2019 [20] Semantic segmentation of fracture images of MgAl2O4, using a deep-cnn Azimi2018 [21] Pixel-wise segmentation of steel microstructure datasets Campbell2018 [22] Automated extraction of quantitative data from material microstructures using advanced image processing technique Agbozo2019 [23] Quantitative metallographic analysis through object segmentation of SEM images Forster2020 [24] Classification of HRTEM images of Carbon Nanotubes into its appropriate chirality Ziatdinov2017 [25] Semantic segmentation of defects and subsequent defect structure identification from atomic scale STEM images Roberts2019 [26] Semantic segmentation of crystallographic defects from electron micrographs Yao2020 [27] Nanoparticle segmentation from liquid-phase TEM images by U-Net Chan2020 [29] Automated quantitative analysis of microstructure using unsupervised algorithms Baskaran2020 [30] Contextual segmentation of morphological features in titanium alloys using a two-stage machine learning pipeline Aguiar2019 [32] Classification of TEM data, atomic resolution images and diffraction data into crystal structures at family level and genera level Zhu2017 [69] Particle recognition from cryo-EM datasets Chen2020 [70] Instance semantic segmentation of Al alloy metallographic images DeCost2018 [71] Semantic segmentation of ultrahigh carbon steel microstructures through deep learning techniques Furat2019 [72] Semantic segmentation of computed tomography data of Al-Cu specimens Vuola2019…”

“…A fast emerging technique in this category is semantic segmentation, which can be summarized as pixel classification performed in the context of the whole image. It is noted that there also exists models that perform classification at a scale that is in between the two extremes stated above, such as the prediction of bounding boxes using a Region Proposal Network (RPN) in [70]. Figure 3 shows characteristic examples of application of the different types of classification discussed above to characterization analysis.…”

Quantitative Analysis of Metallographic Image Using Attention-Aware Deep Neural Networks

“…With regard to the analysis of multi-phase metallographic images, only a few works have been found to transfer classical object detector to recognize different constitutions [ 25 ]. Chen etc.…”

Rapid and Flexible Semantic Segmentation of Electron Microscopy Data Using Few-Shot Machine Learning

“…), [26][27][28][29] or to assign a label to each pixel in the image so that they are classified into discrete categories. 25,[30][31][32] The latter classification type is segmentation of an image to identify local features (e.g. line defects, phases, crystal structures), referred to as semantic segmentation.…”