Probability Density Machine: A New Solution of Class Imbalance Learning

Abstract

Class imbalance learning (CIL) is an important branch of machine learning as, in general, it is difficult for classification models to learn from imbalanced data; meanwhile, skewed data distribution frequently exists in various real-world applications. In this paper, we introduce a novel solution of CIL called Probability Density Machine (PDM). First, in the context of Gaussian Naive Bayes (GNB) predictive model, we analyze the reason why imbalanced data distribution makes the performance of predictive model decline in theory and draw a conclusion regarding the impact of class imbalance that is only associated with the prior probability, but does not relate to the conditional probability of training data. Then, in such context, we show the rationality of several traditional CIL techniques. Furthermore, we indicate the drawback of combining GNB with these traditional CIL techniques. Next, profiting from the idea of K-nearest neighbors probability density estimation (KNN-PDE), we propose the PDM which is an improved GNB-based CIL algorithm. Finally, we conduct experiments on lots of class imbalance data sets, and the proposed PDM algorithm shows the promising results.

Skewed Evolving Data Streams Classification with Actionable Knowledge Extraction using Data Approximation and Adaptive Classification Framework

“…By examining changes in measure values, distributions, and gradients with diverging class proportions, the authors of [29] have provided measure dynamics. In [30], a novel solution to class imbalance learning called Probability Density Machine (PDM) had proposed. First, it analyzed why imbalanced data distribution makes the performance of the predictive model decline in theory.…”

“…In the above section, we have reviewed the various recently proposed streaming data classification methods under different categories. The streaming data classification to address the class imbalance problem had studied in [21][22][23][24][25][26][27][28][29][30]. The streaming data classification with addressing the concept drift had been proposed in studies in [31][32][33][34][35].…”

“…• Although idea drift and class imbalance have received considerable attention [21][22][23][24][25][26][27][28][29][30][31][32][33][34][35], the combined solutions received less attention. Recently explored joint solutions in [36][37][38][39] are insufficient for the skewed dynamic data stream classification problem.…”

“…• Major solutions proposed for stream data learning and classification were based on a sliding windowbased mechanism which leads to poor stream data mining performances with higher costs due to duplicate chunks caused by moving sliding windows. • Data redundancy is a significant challenge for the existing data stream classification solutions [21][22][23][24][25][26][27][28][29][30][31][32][33][34][35][36][37][38][39].…”

“…• Existing solutions failed to bridge the trade-off between class imbalance, concept drift, data redundancy, and performance parameters. Stream data processing solutions in [21][22][23][24][25][26][27][28][29][30] have focused on the class imbalance problem but limit the other challenges and vice-versa using [31][32][33][34][35]. Methods in [36][37][38][39] have addressed concept drift and class imbalance, but data approximation and performance improvement (computational and classification) remain unresolved.…”