An Internal Validity Index Based on Density-Involved Distance

Hu, Lianyu; Zhong, Chongquan

doi:10.1109/access.2019.2906949

Cited by 25 publications

(15 citation statements)

References 65 publications

(66 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…That capability, however, is difficult to describe clearly. Some CVIs' definitions show them to be categorized into center/non-center representative 21 or density-representative. Similarly, the DSI is a separability-representative CVI.…”

Section: Discussionmentioning

confidence: 99%

“…Various CVIs have been created for the clustering of many types of datasets. 20 By methods of calculation, 21 the internal CVIs are based on two categories of representatives: center and non-center. Center-based internal CVIs use descriptors of clusters.…”

Section: Related Workmentioning

confidence: 99%

“…The Calinski-Harabasz index (CH) 24 and Silhouette coefficient (Sil) 25 are two traditional internal CVIs. In recently developed internal CVIs, the I index, 26 WB index, 27 Clustering Validation index based on Nearest Neighbors (CVNN), 18 and Cluster Validity index based on Density-involved Distance (CVDD) 21 are selected. Eight typical internal CVIs, which range from early studies (Dunn, 1974) to the most recent studies (CVDD, 2019), are selected to compare with our proposed CVI.…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

A Distance-based Separability Measure for Internal Cluster Validation

Guan

Loew

2021

Preprint

View full text Add to dashboard Cite

To evaluate clustering results is a significant part of cluster analysis. Since there are no true class labels for clustering in typical unsupervised learning, many internal cluster validity indices (CVIs), which use predicted labels and data, have been created. Without true labels, to design an effective CVI is as difficult as to create a clustering method. And it is crucial to have more CVIs because there are no universal CVIs that can be used to measure all datasets and no specific methods of selecting a proper CVI for clusters without true labels. Therefore, to apply a variety of CVIs to evaluate clustering results is necessary. In this paper, we propose a novel internal CVI -the Distancebased Separability Index (DSI), based on a data separability measure. We compared the DSI with eight internal CVIs including studies from early Dunn (1974) to most recent CVDD (2019) and an external CVI as ground truth, by using clustering results of five clustering algorithms on 12 real and 97 synthetic datasets. Results show DSI is an effective, unique, and competitive CVI to other compared CVIs. We also summarized the general process to evaluate CVIs and created the rank-difference metric for comparison of CVIs' results. The code can be found in author's website linked up with the ORCID:

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

A Distance-based Separability Measure for Internal Cluster Validation

Guan

Loew

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Finally, we have studied in depth the closest recent papers that deal with internal CVIs that use the concept of density and analyzed the main differences with our proposal. Hu and Zhong 32 design a new robust density-involved distance measure. They then define an internal validity index based on this separation measure using minimum spanning trees.…”

Section: Related Workmentioning

confidence: 99%

New internal clustering validation measure for contiguous arbitrary‐shape clusters

Rojas-Thomas

Santos

2021

Int J Intell Syst

View full text Add to dashboard Cite

In this study a new internal clustering validation index is proposed. It is based on a measure of the uniformity of the data in clusters. It uses the local density of each cluster, in particular, the normalized variability of the density within the clusters to find the ideal partition. The new validity measure allows it to capture the spatial pattern of the data and obtain the right number of clusters in an automatic way. This new approach, unlike the traditional one that usually identifies wellseparated compact clouds, works with arbitrary-shape clusters that may be contiguous or even overlapped.The proposed clustering measure has been evaluated on nine artificial data sets, with different cluster distributions and an increasing number of classes, on three highly nonlinear data sets, and on 17 real data sets. It has been compared with nine well-known clustering validation indices with very satisfactory results. This proves that including density in the definition of clustering validation indices may be useful to identify the right partition of arbitrary-shape and different-size clusters.

show abstract

“…Cluster analysis, with which one can find the hidden structures inside the investigated datasets, playing an important role in the domain of data mining [1], [2]. Clustering algorithms and cluster validity indices are two most important tasks in cluster analysis [3], [4]. Determining the optimal number of clusters is usually completed based on one cluster validity index or several [5], [6].…”

Section: Introductionmentioning

confidence: 99%

Volume and Surface Area-Based Cluster Validity Index

Yue

Ding

2020

IEEE Access

View full text Add to dashboard Cite

Cluster validity index plays an important role in assessing the quality of clustering results. However, most of the existing validity indices take a trial-and-error strategy, and their correctness depend on not only the measurements of intra-and inter-cluster distances but also the specific clustering algorithms and data structures. Consequently, the applications of these indices are limited in practice. In this paper, we firstly define the total surface area and volume of all clusters in a 2-dimensinal data space, thereby recovering their natural interrelation among various numbers of clusters. On this basis, a novel validity index is proposed to directly assess the clustering results of any dataset, which does not require any trail-and-error process, clustering algorithms, data structures, or the measurements of intra-and inter-cluster distances. In the case of a high-dimensional data space, all clusters are transformed into spherical clusters of normalized size in a 2-dimensinal data space through a multidimensional scaling transformation. Two groups of typical synthetic datasets and real datasets with various characteristics are used to validate the novel validity index.INDEX TERMS Cluster validity index, multidimensional scaling transformation, volume and surface area.

show abstract

An Internal Validity Index Based on Density-Involved Distance

Cited by 25 publications

References 65 publications

A Distance-based Separability Measure for Internal Cluster Validation

A Distance-based Separability Measure for Internal Cluster Validation

New internal clustering validation measure for contiguous arbitrary‐shape clusters

Volume and Surface Area-Based Cluster Validity Index

Contact Info

Product

Resources

About