Clusterability assessment for Gaussian mixture models

Nowakowska, Ewa; Koronacki, Jacek; Lipovetsky, Stan

doi:10.1016/j.amc.2014.12.038

Cited by 26 publications

(14 citation statements)

References 18 publications

(23 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Considerable effort has been invested to develop reliable methods for grouping sets of objects in multivariate space in such a way that objects falling in the same group are more similar among themselves than to objects in other groups, a technique known as “cluster analysis” (Hennig et al 2016) widely used in ecological applications. Clustering techniques often focus on the problem of optimally splitting a multivariate data set into k ≥ 2 clusters (Halkidi et al 2016), yet the problem of elucidating whether cohesive and well-differentiated groups of objects actually exist in the data, i.e., testing the null hypothesis that k = 1 against the alternative k ≥ 2, has been rarely investigated due to computational difficulties (the so-called “clusterability problem”; Ackerman and Ben-David 2009, Nowakowska et al 2015, Simovici and Hua 2019). Following recent technical developments, it has been advised that a prior study of clusterability should become an integral part of any cluster analysis (Adolfsson 2016, Adolfsson et al 2019, Simovici and Hua 2019).…”

Section: Discussionmentioning

confidence: 99%

“…Formal statistical analyses of “clusterability”, or whether a meaningful clustered structure can be objectively established in a multidimensional data set (Ackerman and Ben-David 2009, Nowakowska et al 2015, Simovici and Hua 2019), were conducted on the set of species coordinates in each of the three pollinator niche spaces considered (each defined by either proportion of visitors or proportion of visits). Dip multimodality tests (Hartigan and Hartigan 1985) were applied to distributions of pairwise distances between species, a method particularly robust to outliers, anomalous topologies and high dimensionality (Adolfsson et al 2019).…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Unclusterable, underdispersed arrangement of insect-pollinated plants in pollinator niche space

Herrera

2020

Preprint

View full text Add to dashboard Cite

Pollinators can mediate facilitative or competitive relationships between plant 11 species, but the comparative importance of these two conflicting phenomena in shaping 12 pollinator resource use in plant communities remains unexplored. This paper provides a case 13 example and proof of concept of the idea that the arrangement in pollinator niche space of large 14 species samples comprising complete or nearly complete plant communities can be instrumental 15 to evaluate the importance of facilitation and competition as drivers of pollinator resource at the 16 plant community level. Pollinator composition data for 221 plant species from the Sierra de 17Cazorla mountains (southeastern Spain), comprising 85% of families and ~95% of widely 18 distributed species of entomophilous plants, were used to address the following questions: (1) 19 Do objectively identifiable species clusters occur in pollinator niche space ? Three different 20 pollinator niche spaces were considered whose axes were defined by insect orders (N = 7), 21 families (N = 94) and genera (N = 346); and (2) If all plant species form a single, indivisible 22 cluster in pollinator niche space, Are species overdispersed or underdispersed relative to a 23 random arrangement ? "Clusterability" tests failed to reject the null hypothesis that there was 24 only one pollinator-defined species cluster in pollinator niche space, irrespective of the space 25 axes (insect orders, families or genera) or pollinator importance measurement (proportions of 26 pollinator individuals or flowers visited by each pollinator type) considered. Randomly 27 simulated species arrangements in each pollinator niche space showed that observed means of 28 pairwise interspecific distances in pollinator composition were smaller than values from 29 simulations, thus revealing significantly non-random, underdispersed arrangement of plant 30 species within the single cluster existing in pollinator niche space. Arrangement of plant species 31 in niche space in the montane plant community studied did not support a major role for 32 interspecific competition as a force shaping pollinator resource use by plants, but rather a 33 Herrera -3 situation closer to the facilitation-dominated extreme in a hypothetical competition-facilitation 34 gradient. Results also illustrate the potential of investigations on complete or nearly complete 35 entomophilous plant communities for addressing novel hypotheses and contributing insights to 36 the understanding of the ecology and evolution of plant-pollinator systems. 37 38 39 composition; pollinator niche space; pollinator syndromes. 40 41 42The maturation of a full-fledged niche theory was a significant landmark in the trajectory of 43 twentieth-century ecology (Vandermeer 1972, Chase and Leibold 2003, Ricklefs 2012. 44Originally aimed at investigating how many and how similar coexisting species could be in a 45 community, niche theory was predominantly concerned with animal communities and rested on 46 the overarching assumption that interspecific c...

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Unclusterable, underdispersed arrangement of insect-pollinated plants in pollinator niche space

Herrera

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…This approach uses the curve centre as the tree location, with the apex indicating the overall tree height and the area under the curve representing the circular crown area. A component overlap analysis of the mixed, normal data distributions identifies changes in the curve location, height and crown area between the overlapping parabolas (Nowakowska, Koronacki et al 2015). A Gaussian overlap models where a single tree, identified and described in both datasets, can be aligned to a potential match in the opposing dataset and any similarities in the biophysical properties compared and quantified.…”

Section: Gaussian Overlapping and The Jaccard Similarity Coefficientmentioning

confidence: 99%

ARBOR: A new framework for assessing the accuracy of individual tree crown delineation from remotely-sensed data

Murray

Gullick

Blackburn

et al. 2019

Remote Sensing of Environment

View full text Add to dashboard Cite

To assess the accuracy of individual tree crown (ITC) delineation techniques the same tree needs to be identified in two different datasets, for example, ground reference (GR) data and crowns delineated from LiDAR. Many studies use arbitrary metrics or simple linear-distance thresholds to match trees in different datasets without quantifying the level of agreement. For example, successful match-pairing is often claimed where two data points, representing the same tree in different datasets, are located within 5m of one another. Such simple measures are inadequate for representing the multi-variate nature of ITC delineations and generate misleading measures of delineation accuracy. In this study, we develop a new framework for objectively quantifying the agreement between GR and remotely-sensed tree datasets: the Accuracy of Remotely-sensed Biophysical Observation and Retrieval (ARBOR) framework. Using common biophysical properties of ITC delineated trees (location, height and crown area), trees represented in different data sets were modelled as overlapping Gaussian curves to facilitate a more comprehensive assessment of the level of agreement. Extensive testing quantified the limitations of some frequently used match-pairing methods, in particular, the Hausdorff distance algorithm. We demonstrate that within the ARBOR framework, the Hungarian combinatorial optimisation algorithm improves the match between datasets, while the Jaccard similarity coefficient is effective for measuring the correspondence between the matched data populations. The ARBOR framework was applied to GR and remotely-sensed tree data from a woodland study site to demonstrate how ARBOR can identify the optimum ITC delineation technique, out of four different methods tested, based on two measures of statistical accuracy. Using ARBOR will limit further reliance on arbitrary thresholds as it provides an objective approach for quantifying accuracy in the development and application of ITC delineation algorithms.

show abstract

“…In the domain of computer science, the notion of structuredness somehow corresponds to the clusterability concept [4]. Intuitively, clusterability can be interpreted as a measure of an "intrinsic structure" of a dataset to be clustered [5]. Computer scientists have observed that a dataset of good clusterability can be clustered quite effectively (i.e., less impact from the NP-hard nature of the clustering problem).…”

Section: Introductionmentioning

confidence: 99%

“…Ackerman and Ben-David [4] have surveyed different definitions of clusterability and shown their incompatibility in pairwise comparisons. Nowakowska et al [5] argued that a clusterability measure should be partition-independent so that it does not depend on the clustering algorithms and the resulting solutions. Ackerman et al [7] proposed the use of the statistical distributions of pairwise distances between any two objects to evaluate clusterability.…”

Section: Introductionmentioning

confidence: 99%

Similarity Statistics for Clusterability Analysis with the Application of Cell Formation Problem

Zhu

2018

Journal of Probability and Statistics

View full text Add to dashboard Cite

This paper proposes the use of the statistics of similarity values to evaluate the clusterability or structuredness associated with a cell formation (CF) problem. Typically, the structuredness of a CF solution cannot be known until the CF problem is solved. In this context, this paper investigates the similarity statistics of machine pairs to estimate the potential structuredness of a given CF problem without solving it. One key observation is that a well-structured CF solution matrix has a relatively high percentage of high-similarity machine pairs. Then, histograms are used as a statistical tool to study the statistical distributions of similarity values. This study leads to the development of the U-shape criteria and the criterion based on the Kolmogorov-Smirnov test. Accordingly, a procedure is developed to classify whether an input CF problem can potentially lead to a well-structured or ill-structured CF matrix.In the numerical study, 20 matrices were initially used to determine the threshold values of the criteria, and 40 additional matrices were used to verify the results. Further, these matrix examples show that genetic algorithm cannot effectively improve the wellstructured CF solutions (of high grouping efficacy values) that are obtained by hierarchical clustering (as one type of heuristics). This result supports the relevance of similarity statistics to preexamine an input CF problem instance and suggest a proper solution approach for problem solving.

show abstract

Clusterability assessment for Gaussian mixture models

Cited by 26 publications

References 18 publications

Unclusterable, underdispersed arrangement of insect-pollinated plants in pollinator niche space

Unclusterable, underdispersed arrangement of insect-pollinated plants in pollinator niche space

ARBOR: A new framework for assessing the accuracy of individual tree crown delineation from remotely-sensed data

Similarity Statistics for Clusterability Analysis with the Application of Cell Formation Problem

Contact Info

Product

Resources

About