Deep interpretable ensembles

Abstract

Ensembles improve prediction performance and allow uncertainty quantification by aggregating predictions from multiple models. In deep ensembling, the individual models are usually black box neural networks, or recently, partially interpretable semi-structured deep transformation models. However, interpretability of the ensemble members is generally lost upon aggregation. This is a crucial drawback of deep ensembles in high-stake decision fields, in which interpretable models are desired. We propose a novel transformation ensemble which aggregates probabilistic predictions with the guarantee to preserve interpretability and yield uniformly better predictions than the ensemble members on average. Transformation ensembles are tailored towards interpretable deep transformation models but are applicable to a wider range of probabilistic neural networks. In experiments on several publicly available data sets, we demonstrate that transformation ensembles perform on par with classical deep ensembles in terms of prediction performance, discrimination, and calibration. In addition, we demonstrate how transformation ensembles quantify both aleatoric and epistemic uncertainty, and produce minimax optimal predictions under certain conditions.

Deep Learning Versus Neurologists: Functional Outcome Prediction in LVO Stroke Patients Undergoing Mechanical Thrombectomy

“…In each fold, we trained 5 randomly initialized versions of the model to consider uncertainty in model parameters and potentially improve prediction performance. 21 The predictions of the 5 folds were then averaged to 1 final prediction. We imputed missing values of clinical variables (Table 1) using missForest.…”

Enhancing the Internet of Medical Things (IoMT) Security with Meta-Learning: A Performance-Driven Approach for Ensemble Intrusion Detection Systems

“…In the context of IDS, ensemble learning plays a crucial role in improving detection accuracy by combining the outputs of multiple classifiers [ 50 ]. However, the interpretability of individual ensemble members may be lost during the aggregation process, highlighting a trade-off between accuracy and explainability [ 51 ]. Moreover, the selection of appropriate ensemble learning models is essential to address the specific requirements of IDS, considering factors such as feature selection and model performance [ 52 ].…”