A reweighting approach to robust clustering

Dotto, Francesco; Farcomeni, Alessio; García-Escudero, Luis Ángel; Mayo-Íscar, Agustín

doi:10.1007/s11222-017-9742-x

Cited by 25 publications

(19 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As an open point for further research, an automatic procedure for selecting reasonable values for the labelled and unlabelled trimming levels, along the lines of Dotto et al (2018), is under study. Additionally, a robust wrapper variable selection for dealing with high-dimensional problems could be useful for further enhancing the discriminating power of the proposed methodology.…”

Section: Discussionmentioning

confidence: 99%

“…Nevertheless, exploratory tools such as Density-Based Silhouette plot (Menardi, 2011) and trimmed likelihood curves (García-Escudero et al, 2011) could be employed to validate and assess the choice of α l and αu. A more automatic approach, like the one introduced in Dotto et al (2018), could also be adapted to our framework. This, however, goes beyond the scope of the present manuscript, it will nonetheless be addressed in the future.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

A robust approach to model-based classification based on trimming and constraints

Cappozzo

Greselin

Murphy

2019

Adv Data Anal Classif

View full text Add to dashboard Cite

In a standard classification framework a set of trustworthy learning data are employed to build a decision rule, with the final aim of classifying unlabelled units belonging to the test set. Therefore, unreliable labelled observations, namely outliers and data with incorrect labels, can strongly undermine the classifier performance, especially if the training size is small. The present work introduces a robust modification to the Model-Based Classification framework, employing impartial trimming and constraints on the ratio between the maximum and the minimum eigenvalue of the group scatter matrices. The proposed method effectively handles noise presence in both response and exploratory variables, providing reliable classification even when dealing with contaminated datasets. A robust information criterion is proposed for model selection. Experiments on real and simulated data, artificially adulterated, are provided to underline the benefits of the proposed method.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

A robust approach to model-based classification based on trimming and constraints

Cappozzo

Greselin

Murphy

2019

Adv Data Anal Classif

View full text Add to dashboard Cite

show abstract

“…From the distance function aspect, metric learning is used to learn a robust metric to measure the similarity between two points by taking the outliers into account [5], [20]; L 1 norm models the outliers as the sparse constraint for cluster analysis [6], [21]. From the data aspect, the outliers are assigned few weights during clustering process [22]; low-rank representation treats the data as the clean part and outliers, and constrains the clean part with the lowest rank [7]. From the model fusion aspect, ensemble clustering integrates different partitions into a consensus one to deliver a robust result [23], [24].…”

Section: Robust Clusteringmentioning

confidence: 99%

Clustering With Outlier Removal

Liu

et al. 2021

IEEE Trans. Knowl. Data Eng.

View full text Add to dashboard Cite

Cluster analysis and outlier detection are strongly coupled tasks in data mining area. Cluster structure can be easily destroyed by few outliers; on the contrary, outliers are defined by the concept of cluster, which are recognized as the points belonging to none of the clusters. Unfortunately, most existing studies do not notice the coupled relationship between these two task and handle them separately. In light of this, we consider the joint cluster analysis and outlier detection problem, and propose the Clustering with Outlier Removal (COR) algorithm. Generally speaking, the original space is transformed into the binary space via generating basic partitions in order to define clusters. Then an objective function based Holoentropy is designed to enhance the compactness of each cluster with a few outliers removed. With further analyses on the objective function, only partial of the problem can be handled by K-means optimization. To provide an integrated solution, an auxiliary binary matrix is nontrivally introduced so that COR completely and efficiently solves the challenging problem via a unified K-means--with theoretical supports. Extensive experimental results on numerous data sets in various domains demonstrate the effectiveness and efficiency of COR significantly over state-of-the-art methods in terms of cluster validity and outlier detection. Some key factors in COR are further analyzed for practical use. Finally, an application on flight trajectory is provided to demonstrate the effectiveness of COR in the real-world scenario.

show abstract

“…We note that by considering a crude rule like , we are likely not to consider all units belonging to a particular group. Therefore, a successive reweighting step (which, in the spirit of the FS, can be performed adaptively; see also Dotto, Farcomeni, García‐Escudero, & Mayo‐Iscar, ) is necessary for refining the tentative groups that have been found. A preliminary proposal, rooted in an exploratory framework, is described in Atkinson et al (, p. 369).…”

Section: Wild Adaptive Trimming For Robust Cluster Analysismentioning

confidence: 99%

Wild adaptive trimming for robust estimation and cluster analysis

Cerioli

Farcomeni

Riani

2018

Scandinavian J Statistics

Self Cite

View full text Add to dashboard Cite

Trimming principles play an important role in robust statistics. However, their use for clustering typically requires some preliminary information about the contamination rate and the number of groups. We suggest a fresh approach to trimming that does not rely on this knowledge and that proves to be particularly suited for solving problems in robust cluster analysis. Our approach replaces the original K‐population (robust) estimation problem with K distinct one‐population steps, which take advantage of the good breakdown properties of trimmed estimators when the trimming level exceeds the usual bound of 0.5. In this setting, we prove that exact affine equivariance is lost on one hand but, on the other hand, an arbitrarily high breakdown point can be achieved by “anchoring” the robust estimator. We also support the use of adaptive trimming schemes, in order to infer the contamination rate from the data. A further bonus of our methodology is its ability to provide a reliable choice of the usually unknown number of groups.

show abstract

A reweighting approach to robust clustering

Cited by 25 publications

References 34 publications

A robust approach to model-based classification based on trimming and constraints

A robust approach to model-based classification based on trimming and constraints

Clustering With Outlier Removal

Wild adaptive trimming for robust estimation and cluster analysis

Contact Info

Product

Resources

About