Hostility measure for multi-level study of data complexity

Abstract

Complexity measures aim to characterize the underlying complexity of supervised data. These measures tackle factors hindering the performance of Machine Learning (ML) classifiers like overlap, density, linearity, etc. The state-of-the-art has mainly focused on the dataset perspective of complexity, i.e., offering an estimation of the complexity of the whole dataset. Recently, the instance perspective has also been addressed. In this paper, the hostility measure, a complexity measure offering a multi-level (instance, class, and dataset) perspective of data complexity is proposed. The proposal is built by estimating the novel notion of hostility: the difficulty of correctly classifying a point, a class, or a whole dataset given their corresponding neighborhoods. The proposed measure is estimated at the instance level by applying the k-means algorithm in a recursive and hierarchical way, which allows to analyze how points from different classes are naturally grouped together across partitions. The instance information is aggregated to provide complexity knowledge at the class and the dataset levels. The validity of the proposal is evaluated through a variety of experiments dealing with the three perspectives and the corresponding comparative with the state-of-the-art measures. Throughout the experiments, the hostility measure has shown promising results and to be competitive, stable, and robust.

Dynamic Disagreeing Neighbors: A complexity measure for instance selection

“…In addition, some of the original complexity measures have been adapted to the instance level [3]. Recently, a multi-level analysis of data complexity has been addressed, covering the instance, the class, and the dataset level with a new proposed complexity measure called hostility measure [17].…”

“…However, the instance perspective of data complexity has fostered their use in tasks related to IS like noise filter or data sampling. For example, in [17], they filter a 10% and a 50% of the most complex points, reducing the error. In [34], data complexity is employed for curriculum learning.…”

“…Consequently, the k would have to be increased to achieve a smoother measure, thus moving away from the recommended parameters. Indeed, the lack of smoothness resulting from the value of k is one of the criticisms of the kDN [17].…”

Effectiveness of SMOTE-ENN to Reduce Complexity in Classification Model

“…Hance, improving the dataset characteristics is crucial to enhance the performance of classification. These characteristics can include overlapping classes, linearity of bound decisions, and imbalance ratio in the dataset (Lancho et al 2023). Ho and Basu (2002) introduced a measurement to assess the dataset characteristics by examining the geometrical distribution of data.…”