Clustering Evaluation in High-Dimensional Data

Tomašev, Nenad; Radovanović, Miloš

doi:10.1007/978-3-319-24211-8_4

Cited by 22 publications

(20 citation statements)

References 95 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Arbelaitz et al (2013) compare 30 measures of clustering, some of which are close variants of each other. A similar study, on synthetic data and a smaller number of measures, was undertaken by Tomašev and Radovanović (2016). The experiments by Arbelaitz et al include results on 20 real, labelled datasets of up to 166 features and up to 2310 items-much smaller in both dimensions than is the case for document collections, but in the absence of larger-scale evaluations this work provides the best independent reference of which we are aware.…”

Section: Background On Measurement Of Cluster Qualitymentioning

confidence: 90%

Measurement of clustering effectiveness for document collections

2022

View full text Add to dashboard Cite

Clustering of the contents of a document corpus is used to create sub-corpora with the intention that they are expected to consist of documents that are related to each other. However, while clustering is used in a variety of ways in document applications such as information retrieval, and a range of methods have been applied to the task, there has been relatively little exploration of how well it works in practice. Indeed, given the high dimensionality of the data it is possible that clustering may not always produce meaningful outcomes. In this paper we use a well-known clustering method to explore a variety of techniques, existing and novel, to measure clustering effectiveness. Results with our new, extrinsic techniques based on relevance judgements or retrieved documents demonstrate that retrieval-based information can be used to assess the quality of clustering, and also show that clustering can succeed to some extent at gathering together similar material. Further, they show that intrinsic clustering techniques that have been shown to be informative in other domains do not work for information retrieval. Whether clustering is sufficiently effective to have a significant impact on practical retrieval is unclear, but as the results show our measurement techniques can effectively distinguish between clustering methods.

show abstract

Section: Background On Measurement Of Cluster Qualitymentioning

confidence: 90%

Measurement of clustering effectiveness for document collections

2022

View full text Add to dashboard Cite

show abstract

“…Choice of Dimensionality Reduction Technique Various DR methods are analyzed as to whether they improve the clustering behavior of DBSCAN compared to applying clustering on the original dimensions.Ḡ + , a measure of the discordance between pairs of point distances and is robust w.r.t. differences in dimensionality [25], is used as a metric. It indicates whether members of the same cluster are closer together than members of different clusters.…”

Section: Discussionmentioning

confidence: 99%

Prototype Discovery Using Quality-Diversity

Hagg

Asteroth

Bäck

2018

Parallel Problem Solving From Nature – PPSN XV

View full text Add to dashboard Cite

An iterative computer-aided ideation procedure is introduced, building on recent quality-diversity algorithms, which search for diverse as well as high-performing solutions. Dimensionality reduction is used to define a similarity space, in which solutions are clustered into classes. These classes are represented by prototypes, which are presented to the user for selection. In the next iteration, quality-diversity focuses on searching within the selected class. A quantitative analysis is performed on a 2D airfoil, and a more complex 3D side view mirror domain shows how computer-aided ideation can help to enhance engineers' intuition while allowing their design decisions to influence the design process.

show abstract

“…In particular in high-dimensional space, performance evaluation is a major challenge and, therefore, needs to be optimised iteratively for determining the optimal number of clusters as well as the clustering techniques to be applied. This is done until a cluster split can be found that provides the best performance evaluation values (Tomašev and Radovanović 2016). To this end, external metrics are used in addition to selected internal evaluation metrics from the optimisation of k to assess the formed clusters, to be able to guarantee a holistic perspective and to provide the clearest assessment of the cluster quality.…”

Section: Performance Evaluationmentioning

confidence: 99%

Analytical Comparison of Clustering Techniques for the Recognition of Communication Patterns

Kaya

Schoop

2021

Group Decis Negot

View full text Add to dashboard Cite

The systematic processing of unstructured communication data as well as the milestone of pattern recognition in order to determine communication groups in negotiations bears many challenges in Machine Learning. In particular, the so-called curse of dimensionality makes the pattern recognition process demanding and requires further research in the negotiation environment. In this paper, various selected renowned clustering approaches are evaluated with regard to their pattern recognition potential based on high-dimensional negotiation communication data. A research approach is presented to evaluate the application potential of selected methods via a holistic framework including three main evaluation milestones: the determination of optimal number of clusters, the main clustering application, and the performance evaluation. Hence, quantified Term Document Matrices are initially pre-processed and afterwards used as underlying databases to investigate the pattern recognition potential of clustering techniques by considering the information regarding the optimal number of clusters and by measuring the respective internal as well as external performances. The overall research results show that certain cluster separations are recommended by internal and external performance measures by means of a holistic evaluation approach, whereas three of the clustering separations are eliminated based on the evaluation results.

show abstract

Clustering Evaluation in High-Dimensional Data

Cited by 22 publications

References 95 publications

Measurement of clustering effectiveness for document collections

Measurement of clustering effectiveness for document collections

Prototype Discovery Using Quality-Diversity

Analytical Comparison of Clustering Techniques for the Recognition of Communication Patterns

Contact Info

Product

Resources

About