Efficient reconstruction of prior austenite grains in steel from etched light optical micrographs using deep learning and annotations from correlative microscopy

Abstract

The high-temperature austenite phase is the initial state of practically all technologically relevant hot forming and heat treatment operations in steel processing. The phenomena occurring in austenite, such as recrystallization or grain growth, can have a decisive influence on the subsequent properties of the material. After the hot forming or heat treatment process, however, the austenite transforms into other microstructural constituents and information on the prior austenite morphology are no longer directly accessible. There are established methods available for reconstructing former austenite grain boundaries via metallographic etching or electron backscatter diffraction (EBSD) which both exhibit shortcomings. While etching is often difficult to reproduce and strongly depend on the investigated steel’s alloying concept, EBSD acquisition and reconstruction is rather time-consuming. But in fact, though, light optical micrographs of steels contrasted with conventional Nital etchant also contain information about the former austenite grains. However, relevant features are not directly apparent or accessible with conventional segmentation approaches. This work presents a deep learning (DL) segmentation of prior austenite grains (PAG) from Nital etched light optical micrographs. The basis for successful segmentation is a correlative characterization from EBSD, light and scanning electron microscopy to specify the ground truth required for supervised learning. The DL model shows good and robust segmentation results. While the intersection over union of 70% does not fully reflect the model performance due to the inherent uncertainty in PAG estimation, a mean error of 6.1% in mean grain size derived from the segmentation clearly shows the high quality of the result.

Multiscale Attention-Based Instance Segmentation for Measuring Crystals With Large Size Variation

“…In a post-processing step, all pixels located within the same boundary are assigned to the same label, resulting in an enumeration of separated instances, an instance label map. To perform the boundary segmentation task, the works in this field are using neural networks with U-Net inspired architectures, e.g., [1], [7]- [9]. The boundary method has a significant drawback, since neighboring crystals are fused together if the segmented boundary mistakenly has gaps.…”

“…[1] train a model with centerline dice loss [13] as part of their loss function. Bachmann et al [7] propose a weighted cross-entropy and Jaccard loss to tackle the imbalanced foreground/background classes. Liu et al [14] predict an adaptive boundary-weighted map based on the original U-Net distance transform [15].…”

“…This, in turn, leads to incorrect measurement of the number and size of crystals. Therefore, post-processing strategies have been proposed to close fragmented segmentation boundaries, e.g., rule-based morphological operations like dilation and erosion [10], [11], applying the watershed algorithm to complete partially segmented crystal boundaries [7], boundary skeletonization to automatically delete or extend discontinued boundaries [12] or GAN-based approaches to automatically close gaps in areas where the boundary is occluded by impurities [8]. However, these post-processing strategies are not capable of closing all boundary gaps and may also incorrectly connect boundary lines.…”

Overview: Machine Learning for Segmentation and Classification of Complex Steel Microstructures

“…In the meantime, ML methods have been applied to a wide range of microstructure analysis tasks, and it is difficult to summarize them concisely. Without claiming to be comprehensive, it can be stated that mainly metallic materials are dealt with, and there is still a clear focus on steel microstructures [1,25,30,46,47]. However, there are also case studies on non-ferrous metals such as copper [48], titanium [49][50][51] or magnetic compounds [52].…”

“…Hot-forming or any type of temperature treatment during steel production generally occurs in the high-temperature phase, that is, austenite. The development of both the austenite grain size during these processes and the resulting austenite grain size are highly significant for the final steel product because they influence the type and properties of the final microstructure, e.g., phase transformation behavior such as bainite or martensite formation or final grain size [46]. Additionally, knowledge about austenite grain evolution is important for understanding and optimizing associated process like thermomechanical controlled processing or alloy designs [91].…”