RfX: A Design Study for the Interactive Exploration of a Random Forest to Enhance Testing Procedures for Electrical Engines

Eirich, Joscha; Munch, Mélanie; Jäckle, Dominik; Sedlmair, Michael; Bonart, J. A. E.; Schreck, Tobias

doi:10.1111/cgf.14452

Cited by 3 publications

(4 citation statements)

References 60 publications

(107 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In conclusion, none of the above works have experimented with the fusion of bagged and boosted decision trees, and in particular, with visualizing both tree types in a joint decisions space to observe their dissimilarity, which can result in unique and undiscovered decisions. RfX 33 supports the comparison of several decision trees originating from a RF model with a dissimilarity projection and icicle plots, allowing electrical engineers to browse a single decision tree by using a node-link diagram. In contrast, VisRuler does not concentrate on a specific domain and gives attention to unique decision paths instead of trees with more scalable visual representations.…”

Section: Related Workmentioning

confidence: 95%

“…As in VisRuler, relevant works that utilize bagging methods use the RF algorithm to produce decision trees. [31][32][33][34][35] iForest 31 provides users with tree-related information and an overview of the involved decision paths for case-based reasoning, with the goal of revealing the model's working internals. However, iForest can be used only for binary classification, while VisRuler can be used with multi-class data sets, see Section System Overview and Use Case.…”

Section: Interpretation Of Bagged Decision Treesmentioning

confidence: 99%

See 1 more Smart Citation

VisRuler: Visual analytics for extracting decision rules from bagged and boosted decision trees

Chatzimparmpas

Martins

Kerren

2023

Information Visualization

View full text Add to dashboard Cite

Bagging and boosting are two popular ensemble methods in machine learning (ML) that produce many individual decision trees. Due to the inherent ensemble characteristic of these methods, they typically outperform single decision trees or other ML models in predictive performance. However, numerous decision paths are generated for each decision tree, increasing the overall complexity of the model and hindering its use in domains that require trustworthy and explainable decisions, such as finance, social care, and health care. Thus, the interpretability of bagging and boosting algorithms—such as random forest and adaptive boosting—reduces as the number of decisions rises. In this paper, we propose a visual analytics tool that aims to assist users in extracting decisions from such ML models via a thorough visual inspection workflow that includes selecting a set of robust and diverse models (originating from different ensemble learning algorithms), choosing important features according to their global contribution, and deciding which decisions are essential for global explanation (or locally, for specific cases). The outcome is a final decision based on the class agreement of several models and the explored manual decisions exported by users. We evaluated the applicability and effectiveness of VisRuler via a use case, a usage scenario, and a user study. The evaluation revealed that most users managed to successfully use our system to explore decision rules visually, performing the proposed tasks and answering the given questions in a satisfying way.

show abstract

Section: Related Workmentioning

confidence: 95%

Section: Interpretation Of Bagged Decision Treesmentioning

confidence: 99%

VisRuler: Visual analytics for extracting decision rules from bagged and boosted decision trees

Chatzimparmpas

Martins

Kerren

2023

Information Visualization

View full text Add to dashboard Cite

show abstract

“…In conclusion, none of the above works have experimented with the fusion of bagged and boosted decision trees, and in particular, with visualizing both tree types in a joint decisions space to observe their dissimilarity, which can result in unique and undiscovered decisions. RfX [170] supports the comparison of several decision trees originating from a RF model with a dissimilarity projection and icicle plots, allowing electrical engineers to browse a single decision tree by using a node-link diagram. In contrast, VisRuler does not concentrate on a specific domain and gives attention to unique decision paths instead of trees with more scalable visual representations.…”

Section: Interpretation Of Bagged Decision Treesmentioning

confidence: 95%

“…As in VisRuler, relevant works that utilize bagging methods use the RF algorithm to produce decision trees [170,479,480,487,745]. iForest [745] provides users with tree-related information and an overview of the involved decision paths for case-based reasoning, with the goal of revealing the model's working internals.…”

Section: Interpretation Of Bagged Decision Treesmentioning

confidence: 99%

Visual Analytics for Explainable and Trustworthy Machine Learning

Chatzimparmpas¹

View full text Add to dashboard Cite

The deployment of artificial intelligence solutions and machine learning research has exploded in popularity in recent years, with numerous types of models proposed to interpret and predict patterns and trends in data from diverse disciplines. However, as the complexity of these models grows, it becomes increasingly difficult for users to evaluate and rely on the model results, since their inner workings are mostly hidden in black boxes, which are difficult to trust in critical decision-making scenarios. While automated methods can partly handle these problems, recent research findings suggest that their combination with innovative methods developed within information visualization and visual analytics can lead to further insights gained from models and, consequently, improve their predictive ability and enhance trustworthiness in the entire process. Visual analytics is the area of research that studies the analysis of vast and intricate information spaces by combining statistical and machine learning models with interactive visual interfaces. By following this methodology, human experts can better understand such spaces and apply their domain expertise in the process of building and improving the underlying models. The primary goals of this dissertation are twofold, focusing on (1) methodological aspects, by conducting qualitative and quantitative meta-analyses to support the visualization research community in making sense of its literature and to highlight unsolved challenges, as well as (2) technical solutions, by developing visual analytics approaches for various machine learning models, such as dimensionality reduction and ensemble learning methods. Regarding the first goal, we define, categorize, and examine in depth the means for visual coverage of the different trust levels at each stage of a typical machine learning pipeline and establish a design space for novel visualizations in the area. Regarding the second goal, we discuss multiple visual analytics tools and systems implemented by us to facilitate the underlying research on the various stages of the machine learning pipeline, i.e., data processing, feature engineering, hyperparameter tuning, understanding, debugging, refining, and comparing models. Our approaches are data-agnostic, but mainly target tabular data with meaningful attributes in diverse domains, such as health care and finance. The applicability and effectiveness of this work were validated with case studies, usage scenarios, expert interviews, user studies, and critical discussions of limitations and alternative designs. The results of this dissertation provide new avenues for visual analytics research in explainable and trustworthy machine learning. Keywords: visualization, interaction, visual analytics, explainable machine learning, XAI, trustworthy machine learning, ensemble learning, dimensionality reduction, supervised learning, unsupervised learning, ML, AI, tabular data

show abstract

DeforestVis: Behaviour Analysis of Machine Learning Models with Surrogate Decision Stumps

Chatzimparmpas,

Martins,

Telea

et al. 2024

Computer Graphics Forum

View full text Add to dashboard Cite

As the complexity of machine learning (ML) models increases and their application in different (and critical) domains grows, there is a strong demand for more interpretable and trustworthy ML. A direct, model‐agnostic, way to interpret such models is to train surrogate models—such as rule sets and decision trees—that sufficiently approximate the original ones while being simpler and easier‐to‐explain. Yet, rule sets can become very lengthy, with many if–else statements, and decision tree depth grows rapidly when accurately emulating complex ML models. In such cases, both approaches can fail to meet their core goal—providing users with model interpretability. To tackle this, we propose DeforestVis, a visual analytics tool that offers summarization of the behaviour of complex ML models by providing surrogate decision stumps (one‐level decision trees) generated with the Adaptive Boosting (AdaBoost) technique. DeforestVis helps users to explore the complexity versus fidelity trade‐off by incrementally generating more stumps, creating attribute‐based explanations with weighted stumps to justify decision making, and analysing the impact of rule overriding on training instance allocation between one or more stumps. An independent test set allows users to monitor the effectiveness of manual rule changes and form hypotheses based on case‐by‐case analyses. We show the applicability and usefulness of DeforestVis with two use cases and expert interviews with data analysts and model developers.

show abstract

RfX: A Design Study for the Interactive Exploration of a Random Forest to Enhance Testing Procedures for Electrical Engines

Cited by 3 publications

References 60 publications

VisRuler: Visual analytics for extracting decision rules from bagged and boosted decision trees

VisRuler: Visual analytics for extracting decision rules from bagged and boosted decision trees

Visual Analytics for Explainable and Trustworthy Machine Learning

DeforestVis: Behaviour Analysis of Machine Learning Models with Surrogate Decision Stumps

Contact Info

Product

Resources

About