Visual Predictive Analytics using iFuseML

Sehgal, Gunjan; Rawat, Mrinal; Gupta, Bindu; Gupta, Garima; Sharma, Geetika; Shroff, Gautam

doi:10.2312/eurova.20181106

Cited by 4 publications

(7 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Section: Buckets Of Modelsmentioning

confidence: 99%

“…There are also a few works that focus specifically on regression problems [41,52,69]. For instance, the more recent tool iFuseML [52] operates with prediction errors in order to present ensemble models with more accurate predictions to the users. The comparison of models is very different in our approach: we use preliminary results from performance metrics in order to select the appropriate models that will boost the final stack performance.…”

Section: Buckets Of Modelsmentioning

confidence: 99%

See 1 more Smart Citation

StackGenVis: Alignment of Data, Algorithms, and Models for Stacking Ensemble Learning Using Performance Metrics

Chatzimparmpas,

Martins,

Kucher

et al. 2020

Preprint

View full text Add to dashboard Cite

In machine learning (ML), ensemble methods-such as bagging, boosting, and stacking-are widely-established approaches that regularly achieve top-notch predictive performance. Stacking (also called "stacked generalization") is an ensemble method that combines heterogeneous base models, arranged in at least one layer, and then employs another metamodel to summarize the predictions of those models. Although it may be a highly-effective approach for increasing the predictive performance of ML, generating a stack of models from scratch can be a cumbersome trial-and-error process. This challenge stems from the enormous space of available solutions, with different sets of data instances and features that could be used for training, several algorithms to choose from, and instantiations of these algorithms (i.e., models) that perform differently according to diverse metrics. In this work, we present a knowledge generation model, which supports ensemble learning with the use of visualization, and a visual analytics system for stacked generalization. Our system, StackGenVis, assists users in dynamically managing data instances, selecting the most important features for a given data set, and choosing a set of top-performant and diverse algorithms. In consequence, our proposed tool helps users to decide between distinct models and to reduce the complexity of the resulting stack by removing overpromising and underperforming models. The applicability and effectiveness of StackGenVis are demonstrated with two use cases: a real-world healthcare data set and a collection of data related to sentiment/stance detection in texts. Finally, the tool has been evaluated through interviews with three ML experts.

show abstract

Section: Buckets Of Modelsmentioning

confidence: 99%

Section: Buckets Of Modelsmentioning

confidence: 99%

StackGenVis: Alignment of Data, Algorithms, and Models for Stacking Ensemble Learning Using Performance Metrics

Chatzimparmpas,

Martins,

Kucher

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Visualizing and understanding the decision paths of random forest algorithms, as well as how their performance was reached, serves as a foundation for assessing the trust in bagging ensemble learning. Other, more general examples can be found in the works by Schneider et al [564] and Sehgal et al [571].…”

Section: Machine Learning Methodsmentioning

confidence: 92%

“…There is also a body of literature devoted to regression problems [144,471,571,742]. For instance, BEAMES [144] contains four ML algorithms and a model sampling method, with the help of which the system generates an ordered list of models that aids the user in selecting a performant model.…”

Section: And Section 84)mentioning

confidence: 99%

See 1 more Smart Citation

Visual Analytics for Explainable and Trustworthy Machine Learning

Chatzimparmpas¹

View full text Add to dashboard Cite

The deployment of artificial intelligence solutions and machine learning research has exploded in popularity in recent years, with numerous types of models proposed to interpret and predict patterns and trends in data from diverse disciplines. However, as the complexity of these models grows, it becomes increasingly difficult for users to evaluate and rely on the model results, since their inner workings are mostly hidden in black boxes, which are difficult to trust in critical decision-making scenarios. While automated methods can partly handle these problems, recent research findings suggest that their combination with innovative methods developed within information visualization and visual analytics can lead to further insights gained from models and, consequently, improve their predictive ability and enhance trustworthiness in the entire process. Visual analytics is the area of research that studies the analysis of vast and intricate information spaces by combining statistical and machine learning models with interactive visual interfaces. By following this methodology, human experts can better understand such spaces and apply their domain expertise in the process of building and improving the underlying models. The primary goals of this dissertation are twofold, focusing on (1) methodological aspects, by conducting qualitative and quantitative meta-analyses to support the visualization research community in making sense of its literature and to highlight unsolved challenges, as well as (2) technical solutions, by developing visual analytics approaches for various machine learning models, such as dimensionality reduction and ensemble learning methods. Regarding the first goal, we define, categorize, and examine in depth the means for visual coverage of the different trust levels at each stage of a typical machine learning pipeline and establish a design space for novel visualizations in the area. Regarding the second goal, we discuss multiple visual analytics tools and systems implemented by us to facilitate the underlying research on the various stages of the machine learning pipeline, i.e., data processing, feature engineering, hyperparameter tuning, understanding, debugging, refining, and comparing models. Our approaches are data-agnostic, but mainly target tabular data with meaningful attributes in diverse domains, such as health care and finance. The applicability and effectiveness of this work were validated with case studies, usage scenarios, expert interviews, user studies, and critical discussions of limitations and alternative designs. The results of this dissertation provide new avenues for visual analytics research in explainable and trustworthy machine learning. Keywords: visualization, interaction, visual analytics, explainable machine learning, XAI, trustworthy machine learning, ensemble learning, dimensionality reduction, supervised learning, unsupervised learning, ML, AI, tabular data

show abstract