Integrating Data and Model Space in Ensemble Learning by Visual Analytics

Abstract

Ensembles of classifier models typically deliver superior performance and can outperform single classifier models given a dataset and classification task at hand. However, the gain in performance comes together with the lack of comprehensibility, posing a challenge to understand how each model affects the classification outputs and from where the errors come. We propose a tight visual integration of the data and the model space for exploring and combining classifier models. We introduce an interactive workflow that builds upon the visual integration and enables the effective exploration of classification outputs and models. The involvement of the user is key to our approach. Therefore, we elaborate on the role of the human and connect our approach to theoretical frameworks on human-centered machine learning. We showcase the usefulness of our approach and the integration of the user via binary and multiclass classification problems. Based on ensembles automatically selected by a standard ensemble selection algorithm, the user can manipulate models and alternative combinations.

A Review on Human–AI Interaction in Machine Learning and Insights for Medical Applications

“…Different papers in the literature have presented different approaches in this area. For example, in Schneider [ 84 ], human experts could determine an area in a visualization system related to the classifier outputs and modify the outputs via indirect parameter tuning. In Fails and Oslen [ 13 ], humans determined some regions in images for image classification and gave a permit to the model to classify features based on the information they provided.…”

“…So, users could select the best classifiers combination with better performance to improve the accuracy of the ensemble classifier. While “EnsembleMatrix” focused on selecting the best combination of classifiers among different combinations, in Schneider et al [ 84 ], despite considering the combination of classifiers, the performance of each classifier was accessible and steerable by a visualization system for human experts and both data space (classification results) and model space (ensemble of classifiers results) were integrated with ensemble learning via visual analytics to help human experts for getting their desired results. In this paper, users could select classification output regions, observe how different classifiers classified the area, add and remove classifiers in an ensemble and track the performance of the different ensemble models.…”

Ablate, Variate, and Contemplate: Visual Analytics for Discovering Neural Architectures

“…Mühlbacher and Piringer support analyzing and comparing regression models based on visualization of feature dependencies and model residuals [42]. Schneider et al demonstrate how the visual integration of the data and the model space can help users select relevant classifiers to form an ensemble [50]. Snowcat is a visual analytics tool that enables model selection from a set of black box models returned from a automated machine learning backend by visually comparing their predictions in the context of the data source [6].…”

NAS-Navigator: Visual Steering for Explainable One-Shot Deep Neural Network Synthesis

“…There has been significant research in the visualization community to make the process of neural network model selection and search more effective. Techniques exist to evaluate network architectures where the models are already known and they are required to be compared on the same validation dataset [11,44,56,71]. On the other hand, visual analytics frameworks have been proposed to get the human in the loop for effectively applying machine learning to different scenarios.…”