Optimal subset selection for causal inference using machine learning ensembles and particle swarm optimization

Abstract

We suggest and evaluate a method for optimal construction of synthetic treatment and control samples for the purpose of drawing causal inference. The balance optimization subset selection problem, which formulates minimization of aggregate imbalance in covariate distributions to reduce bias in data, is a new area of study in operations research. We investigate a novel metric, cross-validated area under the receiver operating characteristic curve (AUC) as a measure of balance between treatment and control groups. The proposed approach provides direct and automatic balancing of covariate distributions. In addition, the AUC-based approach is able to detect subtler distributional differences than existing measures, such as simple empirical mean/variance and count-based metrics. Thus, optimizing AUCs achieves a greater balance than the existing methods. Using 5 widely used real data sets and 7 synthetic data sets, we show that optimization of samples using existing methods (Chi-square, mean variance differences, Kolmogorov–Smirnov, and Mahalanobis) results in samples containing imbalance that is detectable using machine learning ensembles. We minimize covariate imbalance by minimizing the absolute value of the distance of the maximum cross-validated AUC on $$ M $$ M folds from 0.50, using evolutionary optimization. We demonstrate that particle swarm optimization (PSO) outperforms modified cuckoo swarm (MCS) for a gradient-free, non-linear noisy cost function. To compute AUCs, we use supervised binary classification approaches from the machine learning and credit scoring literature. Using superscore ensembles adds to the classifier-based two-sample testing literature. If the mean cross-validated AUC based on machine learning is 0.50, the two groups are indistinguishable and suitable for causal inference.

High dimensional feature data reduction of multichannel sEMG for gesture recognition based on double phases PSO

“…The proposed method requires only a small amount of feature extraction computation and can still reach a significant accuracy for multichannel sEMG. In the future, we will further study the sEMG based applications in the human-machine interaction, such as the real control of the robot, or some rehabilitation treatments based on sEMG, and seek for better pattern recognition method [44,45].…”

Credit risk assessment mechanism of personal auto loan based on PSO-XGBoost Model

“…Then, we apply two machine learning methods, i.e., the Logistic Regression (LR) model and the Random Forest (RF) model as the classifiers for training and prediction. The effect of the models is compared using relevant evaluation indexes such as F1-score, G-means, MCC, and AUC [45][46][47].…”

“…The F1-measure is the harmonic mean of the recall rate R and precision rate P, which can evaluate the overall classification of unbalanced data sets [45][46][47]. The larger the value of F1, the better is the classification effect of the classifier, with.…”

“…In general, the greater the MCC, the better is the classification effect of the model. The MCC is expressed as [45][46][47]:…”

“…The AUC is the area under the ROC (Receiver Operating Characteristic) curve, and is a common index to measure the overall classification performance of the classifier [45][46][47].…”