Practical Methods for Noise Removal: Applications to Spikes, Nonstationary Quasi-Periodic Noise, and Baseline Drift

Abstract

A new approach to signal processing of analytical time-domain data is presented. It consists in identifying the types of noise, characterizing them, and subsequently subtracting them from the otherwise unprocessed data set. The algorithms have been successfully applied to three classes of noise commonly found in analytical signals: spikes, ripples, and baseline drift. Traditional filters have been used as an intermediary step to detect and remove spikes in the signal with 96.8% success. Adaptive ensemble average subtraction has been developed to remove nonstationary ripples that have similar time scales as the signal of interest. This method increased the signal-to-noise ratio by up to 250% and led to minimal distortion of the signal, unlike conventional Fourier filters. Finally the removal of baseline drift has been achieved by subtraction of a mathematical model for the baseline. These three methods are generic, computationally fast, and applicable to a wide range of analytical techniques. Full Matlab codes and examples are included as Supporting Information.

Clinical instrumentation and applications of Raman spectroscopy

“…These include the median filter, the moving average window filter, the Gaussian filter whose full width at half maximum is typically set equal to half the spectral resolution of the system, and the Savitzky–Golay filter of various orders. 59,78–81 Other methods include using PCA, genetic algorithms, and other multivariate statistical approaches to remove the higher order components and effectively removing noise. 82,83 In using any of these or other methods of noise smoothing, care should be taken to retain the integrity of the spectral lineshape especially when dealing with samples with multiple peaks that are close to each other.…”

Raman and coherent anti-Stokes Raman scattering microscopy studies of changes in lipid content and composition in hormone-treated breast and prostate cancer cells

“…The acquired Raman spectra contain not only the desired signal from the sample, but also a background signal containing cosmic rays and the signals from the glass coverslip and PBS solution. In order to remove the various background signals, we performed the following data processing steps: (1) cosmic rays removal, 25 (2) data smoothing using a 5-point moving average filter, (3) background removal by subtracting an average of several point Raman spectra acquired off of the cell sample (glass and PBS only), (4) baseline correction using bioinformatics tool routines (msbackadj regression method with linear interpolation), (5) data normalization by the area under the curve, and (6) wavenumber calibration using polystyrene reference spectra.…”

Tsunami arrival time detection system applicable to discontinuous time series data with outliers

“…Ehrentreich and Sümm-chen (2001) used a wavelet transform method to remove the spikes from the Raman spectra. Feuerstein et al (2009) developed a despiking algorithm based on filtering methods using clinical data. Goring and Nikora (2002) and Jesson et al (2013) presented a phase-space thresholding method that is applied to automated post-processing software to remove spikes from acoustic Doppler velocimeter data (Jesson et al, 2015).…”