How frequently do clusters occur in hierarchical clustering analysis? A graph theoretical approach to studying ties in proximity

Abstract

BackgroundHierarchical cluster analysis (HCA) is a widely used classificatory technique in many areas of scientific knowledge. Applications usually yield a dendrogram from an HCA run over a given data set, using a grouping algorithm and a similarity measure. However, even when such parameters are fixed, ties in proximity (i.e. two equidistant clusters from a third one) may produce several different dendrograms, having different possible clustering patterns (different classifications). This situation is usually disregarded and conclusions are based on a single result, leading to questions concerning the permanence of clusters in all the resulting dendrograms; this happens, for example, when using HCA for grouping molecular descriptors to select that less similar ones in QSAR studies.ResultsRepresenting dendrograms in graph theoretical terms allowed us to introduce four measures of cluster frequency in a canonical way, and use them to calculate cluster frequencies over the set of all possible dendrograms, taking all ties in proximity into account. A toy example of well separated clusters was used, as well as a set of 1666 molecular descriptors calculated for a group of molecules having hepatotoxic activity to show how our functions may be used for studying the effect of ties in HCA analysis. Such functions were not restricted to the tie case; the possibility of using them to derive cluster stability measurements on arbitrary sets of dendrograms having the same leaves is discussed, e.g. dendrograms from variations of HCA parameters. It was found that ties occurred frequently, some yielding tens of thousands of dendrograms, even for small data sets.ConclusionsOur approach was able to detect trends in clustering patterns by offering a simple way of measuring their frequency, which is often very low. This would imply, that inferences and models based on descriptor classifications (e.g. QSAR) are likely to be biased, thereby requiring an assessment of their reliability. Moreover, any classification of molecular descriptors is likely to be far from unique. Our results highlight the need for evaluating the effect of ties on clustering patterns before classification results can be used accurately.Graphical abstractFour cluster contrast functions identifying statistically sound clusters within dendrograms considering ties in proximity

Nonunique UPGMA clusterings of microsatellite markers

“…Additional ties may also appear at any step of the hierarchical clustering process, and it is even possible to have ties due to the limited resolution (number of decimal digits) used to store the proximities matrix. Therefore, we conclude that the clustering of microsatellite markers is prone to generate tied distances, similarly to what happens in other cases [ 16 ].…”

Novel inhibitors to ADP ribose phosphatase of SARS-CoV-2 identified by structure-based high throughput virtual screening and molecular dynamics simulations

“…First, digital fingerprints of 3D ligand structures were generated using 3-point pharmacophore [ 16 ]. Next, hierarchical clustering with default parameters was performed to group similar compounds into different clusters using their fingerprints and a cluster ID was assigned to each compound [ 17 , 18 ].…”

Sequential extraction and risk assessment of pollutants from one major tributary of the Ganga

“…To yield different clusters of the data set, dendrogram can be fragmented at different levels. Each fragmented level provides a visual summary of the cluster through a picture of the groups and their proximity with a dramatic reduction in dimensionality of the original data (Leal et al 2016). In this study, the Ward's method is used evaluate distance between clusters (Bhardwaj & Parmar 2020).…”