Comparison of spectral and spatial denoising techniques in the context of High Definition FT-IR imaging hyperspectral data

Abstract

The recent emergence of High Definition (HD) FT-IR and Quantum Cascade Laser (QCL) Microscopes elevated the IR imaging field very close to clinical timescales. However, the speed of acquisition and data quality are still the critical factors in reaching the clinic. Denoising offers aide in both aspects if performed properly. However, there is a lack of a direct comparison of the efficiency of denoising techniques in IR imaging in general. To achieve such comparison within a rigorous framework and obtaining the critical information about signal loss, a simulated dataset strongly bound by experimental parameters was created. Using experimental structural and spectral information and experimental noise levels data as an input for the simulation, a direct comparison of spatial (Fourier transform, Mean Filter, Weighted Mean Filter, Gauss Filter, Median Filter, spatial Wavelets and Deep Neural Networks) and spectral (Savitzky-Golay, Fourier transform, Principal Component Analysis, Minimum Noise Fraction and spectral Wavelets) denoising schemes was enabled. All of these techniques were compared on the simulated dataset, taking into account SNR gain, signal distortion and sensitivity to tuning parameters as comparison metrics. Later, the best techniques were applied to experimental data for validation. The results presented here clearly show the benefit of using hyperspectral denoising schemes such as PCA and MNF which outperform other methods.

Noise-free simulation of an FT-IR imaging hyperspectral dataset of pancreatic biopsy core bound by experiment

“…Since preprocessing applied to more realistic noisy data might affect them in an unexpected way, therefore it is useful to compare introduced changes with the initial signal. This was exactly the case in the study investigating the influence of different denoising techniques on FT-IR data 8 . To numerically estimate level of deformation done by denoising, a Signal Distortion (SD) metrics was defined and it consists of the following steps.…”

“…Moreover, every experimental signal will have a noise component even after long acquisitions. In order to have a true insight into signal distortion we decided to create simulated datasets with raw and noisy data 8,9 .…”

“…This kind of analysis was never done before, especially on that scale. We recently published results, where among others we presented optimization process of five spectral denoising techniques 8 . By making this data available to other researchers we aim to encourage them to perform similar optimizations, which is possible not only for denoising but might be applied to diversity of preprocessing techniques.…”

Quality-preserving low-cost probabilistic 3D denoising with applications to Computed Tomography

“…All deployed common denoising and image quality assessment tools are available in the MATLAB functions from the "Image Processing", "Deep Learning", "Machine Learning" and "Wavelets" toolboxes of MathWorks. We used denoising methods based on local window filtering of the data (3D Gaussian filtering with the MATLAB-function imgaussf ilt3(), 3D local median filtering with the MATLABfunction medf ilt3() and bilateral filtering with the MATLAB-function imbilatf ilt()) [16,17,18,14], spectral denoising methods (the 3D wavelets denoising with the MATLAB-function wavedec3()) [19,20,21,22,15] and a deep learning denoising method based on pre-trained feed-forward denoising convolutional neural networks (DnCNNs, with the MATLAB-functions denoiseImage() and denoisingN etwork()) [25,26,13,27]. For each of the considered images, the standard deviation of the local Gaussian smoothing kernel σ was changed in the range σ = [0.2, 0.4, 0.6, .…”

“…The unsupervised approaches search for a hidden pattern without prior learning, whereas the supervised techniques aim to identify features previously learned from the training data. Unsupervised methods do not require previous training, allow high-speed computations, and belong to the most frequentlyused image denoising instruments [14,15]. They include methods based on local averaging of the data (like Gaussian, weighted Gaussian, bilateral and mean average filtering) [16,17,18,14] and spectral methods (like Fourier-, wavelet-and PCA-denoising) [19,20,21,22,15].…”