Learning from Imbalanced Data Streams

Fernández, Alberto; García, Salvador; Galar, Mikel; Prati, Ronaldo C.; Krawczyk, Bartosz; Herrera, Francisco

doi:10.1007/978-3-319-98074-4_11

Cited by 48 publications

(46 citation statements)

References 58 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For binary outcomes, the Brier score is defined as

Brier score = \frac{1}{N} {140%true \sum}_{t = 1}^{N} {(f_{t} - o_{t})}^{2}

where f t is the predicted probability for example t , o t is the actual outcome of example t and N is the total number of examples in the sample. Because the Brier score measures the mean squared difference between the predicted probability of a certain outcome for a particular instance and the actual outcome, lower Brier scores indicate better performance (Fernández et al, 2018). However, for imbalanced datasets, the Brier score may appear very promising overall but poor for the rare class (cases; Wallace & Dahabreh, 2012).…”

Section: Methodsmentioning

confidence: 99%

Assessing the predictive ability of the Suicide Crisis Inventory for near‐term suicidal behavior using machine learning approaches

Parghi

Chennapragada

Barzilay

et al. 2020

Int J Methods Psych Res

View full text Add to dashboard Cite

Objective: This study explores the prediction of near-term suicidal behavior using machine learning (ML) analyses of the Suicide Crisis Inventory (SCI), which measures the Suicide Crisis Syndrome, a presuicidal mental state. Methods: SCI data were collected from high-risk psychiatric inpatients (N = 591) grouped based on their short-term suicidal behavior, that is, those who attempted suicide between intake and 1-month follow-up dates (N = 20) and those who did not (N = 571). Data were analyzed using three predictive algorithms (logistic regression, random forest, and gradient boosting) and three sampling approaches (split sample, Synthetic minority oversampling technique, and enhanced bootstrap). Results: The enhanced bootstrap approach considerably outperformed the other sampling approaches, with random forest (98.0% precision; 33.9% recall; 71.0% Area under the precision-recall curve [AUPRC]; and 87.8% Area under the receiver operating characteristic [AUROC]) and gradient boosting (94.0% precision; 48.9% recall; 70.5% AUPRC; and 89.4% AUROC) algorithms performing best in predicting positive cases of near-term suicidal behavior using this dataset. Conclusions: ML can be useful in analyzing data from psychometric scales, such as the SCI, and for predicting near-term suicidal behavior. However, in cases such as the current analysis where the data are highly imbalanced, the optimal method of measuring performance must be carefully considered and selected.

show abstract

“…For binary outcomes, the Brier score is defined as

Brier score = \frac{1}{N} {140%true \sum}_{t = 1}^{N} {(f_{t} - o_{t})}^{2}

Section: Methodsmentioning

confidence: 99%

Assessing the predictive ability of the Suicide Crisis Inventory for near‐term suicidal behavior using machine learning approaches

Parghi

Chennapragada

Barzilay

et al. 2020

Int J Methods Psych Res

View full text Add to dashboard Cite

show abstract

“…Over and under sampling strategies are very popular and effective approaches to deal with the class imbalance problem [21,25,33,50]. To compensate the class imbalance by biasing the process of discrimination, the ROS algorithm randomly replicates samples from the minority classes while the RUS technique randomly eliminates samples from the majority classes, until achieving a relative classes balance [23,60].…”

Section: Sampling Class Imbalance Approachesmentioning

confidence: 99%

“…Moreover, they have been combined with editing methods, such as Tomek's Links (TL) [26], Editing Nearest Neighbor (ENN) [27], Condensed Nearest Neighbor rule (CNN) [28] and others [22,[29][30][31][32][33]63]; i.e., hybrid methods, such as SMOTE+TL, SMOTE+CNN, SMOTE+OSS and SMOTE+ENN [22,64,65]. They have been successfully applied to deal the class imbalance problem [21].…”

Section: Sampling Class Imbalance Approachesmentioning

confidence: 99%

“…Data sampling methods have important drawbacks, such as longer training times and over-fitting, which are more likely to occur when minority samples are replicated. Additionally, the loss of information is common when many samples are removed from majority classes, potentially dismissing useful information for the classifier [19][20][21]. Then, more "intelligent" sampling, methods including a heuristic mechanism, have been developed [17,22,23].…”

Section: Introductionmentioning

confidence: 99%

“…It has seen that big data class imbalance approaches have been addressed by adaptation of traditional techniques, mainly sampling methods [21,44]. However, recent studies show that some conclusions from machine learning are not applicable to the big data context; for example, in machine learning is common that SMOTE performs better than ROS [24], but in big data some results do not show this trend [1,48].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Data Sampling Methods to Deal With the Big Data Multi-Class Imbalance Problem

et al. 2020

View full text Add to dashboard Cite

The class imbalance problem has been a hot topic in the machine learning community in recent years. Nowadays, in the time of big data and deep learning, this problem remains in force. Much work has been performed to deal to the class imbalance problem, the random sampling methods (over and under sampling) being the most widely employed approaches. Moreover, sophisticated sampling methods have been developed, including the Synthetic Minority Over-sampling Technique (SMOTE), and also they have been combined with cleaning techniques such as Editing Nearest Neighbor or Tomek's Links (SMOTE+ENN and SMOTE+TL, respectively). In the big data context, it is noticeable that the class imbalance problem has been addressed by adaptation of traditional techniques, relatively ignoring intelligent approaches. Thus, the capabilities and possibilities of heuristic sampling methods on deep learning neural networks in big data domain are analyzed in this work, and the cleaning strategies are particularly analyzed. This study is developed on big data, multi-class imbalanced datasets obtained from hyper-spectral remote sensing images. The effectiveness of a hybrid approach on these datasets is analyzed, in which the dataset is cleaned by SMOTE followed by the training of an Artificial Neural Network (ANN) with those data, while the neural network output noise is processed with ENN to eliminate output noise; after that, the ANN is trained again with the resultant dataset. Obtained results suggest that best classification outcome is achieved when the cleaning strategies are applied on an ANN output instead of input feature space only. Consequently, the need to consider the classifier's nature when the classical class imbalance approaches are adapted in deep learning and big data scenarios is clear.

show abstract