Composing hybrid wavelets for optimum and near‐optimum representation and accelerated evaluation of N‐way data sets

Abstract

In hyphenated measurement devices, temporal, spatial, and spectral resolutions continue to increase. While this is advantageous from a chemical sensing perspective, the amount of data grows exponentially; this imposes challenges on Chemometric algorithms resulting in long computation times. In online sensing, however, time resolution is of vital importance and time delays introduced by lengthy computations become unacceptable. Further, in many applications, data need to be documented and a continuous stream of large data sets presents another technical burden regarding archiving space. This work builds on the previous wavelet studies that have already been published. It was found that there is no reason to use the same wavelet for all dimensions of a data set. 'Hybrid wavelets' were introduced which combine different wavelets; this facilitates fine-tuning of the compression. The challenge in using hybrid wavelets lies in the very large number of possible wavelet combinations. A method is presented that automatically determines the optimum and near-optimum wavelet combinations for a given data set. These wavelet combinations are found by evaluating each dimension of the data set separately. This procedure enables an optimization of computation speed, compressed data set size and accuracy of the Chemometric model.Two data cubes acquired from two different experiments are used to show the selection algorithm's capabilities. These examples demonstrate that this algorithm selects hybrid wavelets that are superior to randomly selected wavelet combinations regarding data approximation, compressed data set size, and acceleration of computations.

Accelerating kernel principal component analysis (KPCA) by utilizing two‐dimensional wavelet compression: applications to spectroscopic imaging

“…In order to ensure a meaningful wavelet compression in the X-dimension [22][23][24][25], the same wavelet coefficients in the X-direction have to be removed from all rows of K. If this is not ensured, the subsequent WTs in the Y-dimension would incorporate wavelet coefficients belonging to different positions of the X-dimension WTs. This would cause the final result to be meaningless.…”

“…New approaches are required that enable diagonalization of such large matrices on personal computers within reasonable time. Already available wavelet-based compression methods [22][23][24][25] cannot be used since they load the full dataset and compress it while holding the entire dataset in memory. Here, this must be avoided at all times simply because of datasets sizes.…”

“…This translates the KPCA model from the compressed feature space into the uncompressed feature space (24), (25). Since wavelet Figure 1.…”

“…K is built using measurement spectra that have previously been mapped from the 'input space' X into the 'feature space' H (see Equations (24), (25) and related discussion). As outlined in the appendix, K has dimensions M Â M with M being the number of (here) spectra.…”

“…In order to handle large amounts of data in a timely fashion, multi-dimensional wavelet transformations (WTs) [13][14][15][16][17][18][19][20][21] have been employed as an effective method of data compression prior to chemometric computations [22][23][24][25] (and references therein). These techniques facilitate reductions in memory requirements and computation times.…”