Cluster Analysis for IR and NIR Spectroscopy: Current Practices to Future Perspectives

Abstract

Supervised machine learning techniques have become well established in the study of spectroscopy data. However, the unsupervised learning technique of cluster analysis hasn't reached the same level maturity in chemometric analysis. This paper surveys recent studies which apply cluster analysis to NIR and IR spectroscopy data. In addition, we summarize the current practices in cluster analysis of spectroscopy and contrast these with cluster analysis literature from the machine learning and pattern recognition domain. This includes practices in data pre-processing, feature extraction, clustering distance metrics, clustering algorithms and validation techniques. Special consideration is given to the specific characteristics of IR and NIR spectroscopy data which typically includes high dimensionality and relatively low sample size. The findings highlighted a lack of quantitative analysis and evaluation in current practices for cluster analysis of IR and NIR spectroscopy data. With this in mind, we propose an analysis model or workflow with techniques specifically suited for cluster analysis of IR and NIR spectroscopy data along with a pragmatic application strategy.

An analysis framework for clustering algorithm selection with applications to spectroscopy

“…Cluster analysis of spectroscopy (as seen in the public spectroscopy datasets we evaluate) is often applied in controlled laboratory setting for food, agriculture and biomedical studies [ 4 ]. We now consider the characteristics that are important for this application and the associated user’s needs.…”

“…Considering the purpose of the analysis and the user’s needs, the multi-modal (hierarchical) cluster analysis capability is important as it enables the results to be related back to the hierarchical nature of the samples (e.g. biological species, genus and family) [ 4 ]. However, the number of parameters and the deterministic characteristics of the algorithms are typically not as important as they are unlikely to be performed as part of an automated process.…”

“…A second observation of note was that while k -means clustering algorithm was the second-best performing algorithm for each set of spectroscopy data, it was not recommended by the analysis framework. This is primarily due to its lack of a hierarchical output, and hence, does not match the typical user’s needs where the public spectroscopy datasets commonly analyse biological materials that contain a taxonomical hierarchy [ 4 ].…”

“…The publicly available spectroscopy datasets utilized in our study include mid-infrared (MIR), near-infrared (NIR), and Fourier transform infrared (FTIR) spectroscopy and are described as follows: The Coffee datasets [ 29 , 30 ] from two different species, the Fruit dataset [ 30 , 31 ] from adulterated and non-adulterated strawberry purees, the Liver datasets [ 32 ] annotated according to the majority presence of collagen, glycogen, lipids, or DNA in the cell, the Mangos datasets [ 33 , 34 ] of four different mango cultivars, the Marzipan datasets [ 35 , 36 ] of nine marzipan types, the Meats dataset [ 30 , 37 ] from chicken, pork and turkey mince, the Olive Oil dataset [ 30 , 38 ] of olive oils from four geographic regions, and the Wine (FTIR spectroscopy) dataset [ 36 , 39 ] of wine from four geographic regions. These represent the typical spectroscopy datasets used in laboratory studies where the context is to demonstrate a spectroscopic testing technique can differentiate different materials or families of materials [ 4 ].…”

“…To make these trade-offs in selecting appropriate clustering algorithms, there needs to be understanding and consideration of the intended application for the analysis. In our previous works, which surveyed cluster analysis in spectroscopy, we found little consistency, evaluation or even justification in the selection of clustering algorithms [ 4 ]. This highlighted the need for better practices.…”