Developing Fully Automated Quality Control Methods for Preprocessing Raman Spectra of Biomedical and Biological Samples

Abstract

Spectral preprocessing is frequently required to render Raman spectra useful for further processing and analyses. The various preprocessing steps, individually and sequentially, are increasingly being automated to cope with large volumes of data from, for example, hyperspectral imaging studies. Full automation of preprocessing is especially desirable when it produces consistent results and requires minimal user input. It is therefore essential to evaluate the "quality" of such preprocessed spectra. However, relatively few methods exist to evaluate preprocessing quality, and fully automated methods for doing so are virtually non-existent. Here we provide a brief overview of fully automated spectral preprocessing and fully automated quality assessment of preprocessed spectra. We follow this with the introduction of fully automated methods to establish figures-of-merit that encapsulate preprocessing quality. By way of illustration, these quantitative methods are applied to simulated and real Raman spectra. Quality factor and quality parameter figures-of-merit resulting from individual preprocessing step quality tests, as well as overall figures-of-merit, were found to be consistent with the quality of preprocessed spectra.

EXPRESS: Smoothing Raman Spectra with Contiguous Single-Channel Fitting of Voigt Distributions: An Automated, High Quality Procedure

“…We used a stringent QT that simultaneously incorporates two tests for normality (the chi-squared goodness-of-fit test for normally distributed noise and the runs test for non-randomness) and two tests for detecting outliers, i.e., clipped peaks and filled valleys (the Anderson-Darling test and a threshold based on the LOD given normally distributed noise). 26 We found that all 20 0.5% noise spectra smoothed with Modification 2 passed the QT while only one spline-smoothed spectrum did and none that were smoothed with the iSG method passed. For the 5% case, all 20 Modification 2 smoothed spectra passed the QT and the results improved for the other methods with 19 and 17, respectively, passing the test.…”

Augmented Two-Dimensional Correlation Spectroscopy for the Joint Analysis of Correlated Changes in Spectroscopic and Disparate Sources

“…S3 for repeated pseudospectra. Repeated pseudospectra were obtained from pseudospectra 23 of the measured spectra to further improve the spectral resolution. Of the two, node-narrowing produced results similar to using unenhanced spectra while using repeated pseudospectra produced worse results.…”

“…Overlapping peaks that change at different rates would complicate joint analyses. We attempted to enhance spectrometric resolution with the use of pseudospectra 23 and node narrowing 24 to mitigate such effects. A pseudospectrum is generated from the first derivative of a “parent” spectrum with the absolute values of the negative features shifted to coincide maximally with the positive features, summed with the positive features, and the sum smoothed with a moving average filter of spectral resolution size.…”

Machine Learning-Assisted Sampling of Surfance-Enhanced Raman Scattering (SERS) Substrates Improve Data Collection Efficiency

“…In a typical SERS data acquisition process, whether it is a clinical diagnostic platform or a characterization of an unknown chemical entity, hundreds to thousands of spectra are typically collected, preprocessed, and subjected to downstream analyses, e.g., principal component analysis (PCA), hierarchical clustering, or other types of classification routines. Much literature has been devoted to the preprocessing considerations, 2–4 including de-noising, smoothing, baseline correction algorithms, background subtraction methods, and cosmic ray removal.…”