The impact of data difficulty factors on classification of imbalanced and concept drifting data streams

Abstract

Class imbalance introduces additional challenges when learning classifiers from concept drifting data streams. Most existing work focuses on designing new algorithms for dealing with the global imbalance ratio and does not consider other data complexities. Independent research on static imbalanced data has highlighted the influential role of local data difficulty factors such as minority class decomposition and presence of unsafe types of examples. Despite often being present in real-world data, the interactions between concept drifts and local data difficulty factors have not been investigated in concept drifting data streams yet. We thoroughly study the impact of such interactions on drifting imbalanced streams. For this purpose, we put forward a new categorization of concept drifts for class imbalanced problems. Through comprehensive experiments with synthetic and real data streams, we study the influence of concept drifts, global class imbalance, local data difficulty factors, and their combinations, on predictions of representative online classifiers. Experimental results reveal the high influence of new considered factors and their local drifts, as well as differences in existing classifiers’ reactions to such factors. Combinations of multiple factors are the most challenging for classifiers. Although existing classifiers are partially capable of coping with global class imbalance, new approaches are needed to address challenges posed by imbalanced data streams.

A survey on learning from imbalanced data streams: taxonomy, challenges, empirical study, and reproducible experimental framework

“…To evaluate the classifiers in specific and controlled scenarios, we prepared data streams generators under different imbalanced and drifting settings. Nine generators in MOA (Bifet et al, 2010b) plus one generator proposed by Brzeziński (Brzeziński et al, 2021) were used. Those generators are presented in Table 3, with their number of attributes, classes, and whether they can generate internal concept drifts.…”

“…Goal of the experiment. This experiment addresses RQ3 and evaluates the robustness of the classifiers to instance-level difficulties (Brzeziński et al, 2021). We evaluated the Brzeziński generator with borderline or rare instances, and combining both at the same time.…”

“…Check for instance-level difficulties. Instance-level difficulties in data streams pose significant difficulties to most of the classifiers (Brzeziński et al, 2021). It is crucial to analyze your stream to understand if such factors are present.…”

“…An experimental analysis of oversampling in drifting streams was presented by Bernardo and Della Valle (2022), focusing on C-SMOTE and how it can benefit selected classifiers. In Brzeziński et al (2021) authors examined the interactions between the global class imbalance ratio, local data difficulty factors, and concept drift, using five classifiers. While these works are an important first step, there is a need for a unified and holistic study of learning from imbalanced data streams that could be used as a template for researchers to evaluate their newly proposed algorithms.…”

Hypergraph-based data modeling for binary classification

“…There are already many papers dealing with the challenge of class imbalance for the classification task [10,19,20]. Many difficulty factors for this problem have been investigated [10,20] and various solutions proposed [21]. However, none of these solutions is based on hypergraph modeling of categorical features and focused on optimizing the random undersampling.…”