Bayesian optimization for predicting rare internal failures in manufacturing processes

A survey on addressing high-class imbalance in big data

“…Maurya [13] presents the ImbalancedBayesOpt (IBO) algorithm as a unique approach based on the optimization of Matthew's Correlation Coefficient (MCC), which can be used as a measure of class imbalance in binary datasets. The value for MCC is computed from the TP, TN, FP, and FN numbers provided in a classifier's confusion matrix, and its equation is shown in Fig.…”

“…The author also proposes the ImbalancedGridOpt (IGO) algorithm, which optimizes the MCC metric on an imbalanced dataset using a uniform grid search over the parameter, w, i.e., weight of the positive (minority) instances. The case study presented is based on a dataset representing measurements of manufactured parts as they move through the production line, with the binary class representing whether or not a part will fail quality control, where a value of 1 indicates failure (minority class) [13]. The severely imbalanced dataset, represented by a class ratio of about 171:1, consisted of approximately 1.18 million instances characterized by 6747 features.…”

“…The value of w for the best classification performance for IBO was 0.87 (Table 4, Appendix A2). Our general analysis of Maurya [13] is as follows: selection of GBM as the classifier over other more commonly used learners is not advisable, because a few studies show it as a better performer and classifier performances depend on multiple factors including dataset characteristics and empirical settings; the original big data is not completely utilized in the experiments-only a small fraction is used as the training dataset, making the empirical work more comparable to works on class imbalance in traditional data; proposed algorithms, IBO and IGO, are compared to only one other learner (i.e., GBM), limiting the results' generalization in the data mining community; and, only under-sampling is considered (for part of the experiments), instead of also including other data-sampling methods, such as EUS, ROS, and SMOTE.…”

“…Maurya [13] ImbalancedBayesOpt is a Bayesian optimization algorithm that maximizes MCC by learning optimal weights for the positive and negative classes Veeramachaneni et al [60] AI 2 combines artificial intelligence (matrix decomposition, neural networks, joint probability methods) with analyst intuition Galpert et al [14] This method combines Apache Hadoop (MapReduce) ROS with Apache Spark SVM (ROS + SVM-BD)…”

“…For the cost-sensitive techniques, our discussion includes a fuzzy rule-based classification approach [9,10] and an online learner scheme [11,12]. For the hybrid/ensemble techniques, our discussion includes a Bayesian Optimization Algorithm that maximizes Matthew's Correlation Coefficient by learning optimal weights for the positive and negative classes [13], and an approach that combines Random Over-Sampling (ROS) and Support Vector Machines (SVMs) [14]. One of the primary problems we encountered during our surveyed works was that the MapReduce big data framework was observed to be quite sensitive to high-class imbalance [15], primarily due to the adverse effects of creating multiple partitions within the already very small minority class space.…”