Comparing End-to-End Machine Learning Methods for Spectra Classification

Abstract

In scientific research, spectroscopy and diffraction experimental techniques are widely used and produce huge amounts of spectral data. Learning patterns from spectra is critical during these experiments. This provides immediate feedback on the actual status of the experiment (e.g., time-resolved status of the sample), which helps guide the experiment. The two major spectral changes what we aim to capture are either the change in intensity distribution (e.g., drop or appearance) of peaks at certain locations, or the shift of those on the spectrum. This study aims to develop deep learning (DL) classification frameworks for one-dimensional (1D) spectral time series. In this work, we deal with the spectra classification problem from two different perspectives, one is a general two-dimensional (2D) space segmentation problem, and the other is a common 1D time series classification problem. We focused on the two proposed classification models under these two settings, the namely the end-to-end binned Fully Connected Neural Network (FCNN) with the automatically capturing weighting factors model and the convolutional SCT attention model. Under the setting of 1D time series classification, several other end-to-end structures based on FCNN, Convolutional Neural Network (CNN), ResNets, Long Short-Term Memory (LSTM), and Transformer were explored. Finally, we evaluated and compared the performance of these classification models based on the High Energy Density (HED) spectra dataset from multiple perspectives, and further performed the feature importance analysis to explore their interpretability. The results show that all the applied models can achieve 100% classification confidence, but the models applied under the 1D time series classification setting are superior. Among them, Transformer-based methods consume the least training time (0.449 s). Our proposed convolutional Spatial-Channel-Temporal (SCT) attention model uses 1.269 s, but its self-attention mechanism performed across spatial, channel, and temporal dimensions can suppress indistinguishable features better than others, and selectively focus on obvious features with high separability.

Self-Supervised Approaches to the Classification of Spectra: Application to Phase Transitions in X-ray Diffraction Data

“…The shared feature extraction backbone model applied in this approach is the Conv SC attention model from Ref. 27 , but without feed-forward network, as shown in Fig. S1 of the supplementary material.…”

“…These may require a long training time, and rely on experts' knowledge in terms of data annotation. Current popular methods are based on deep neural networks (DL), of which the most commonly established are convolutional neural network (CNN) 22-,24 , recurrent neural networks (RNNs) 25,26 , attention-based neural networks 27,28 , and hybrid models 26,27,29 . However, it should be noted that the strength of supervised ML methods, that is the possibility of introducing domainknowledge through annotation, is often problem-speci c and time-consuming, which again hinders automation.…”

“…The backbone encoders applied in all three models were adapted from the ConvSC attention model in Ref. 27 , which was speci cally designed for 1D spectral classi cation. We furthermore demonstrate the validity and performances of these three frameworks targeting the identi cation of a phase transition as seen by x-ray diffraction.…”

RIIAnet: A Real-Time Segmentation Network Integrated with Multi-Type Features of Different Depths for Pavement Cracks

“…The idea underlying convolution is to form global information from the combination of features in the local sensory field to make target judgments, and the computing power is allocated more to the channel dimension [54]. However, it was found in [39] that as the depth of the network increases, the relevance of the convolutional kernels becomes stronger, the diversification decreases, and the redundant information between channels becomes prominent.…”

Water chemical oxygen demand prediction model based on the CNN and ultraviolet-visible spectroscopy

“…With further research, more innovative data pre-processing algorithms have been proposed to improve the validity of the data, and a variety of efficient water quality prediction models have been proposed to improve the model's accuracy. These research studies have provided a theoretical foundation for the efficient and accurate detection of the water quality (Guang et al, 2019;Passos and Saraiva, 2019;Sun et al, 2021).…”