A survey of surveys on the use of visualization for interpreting machine learning models

Abstract

Research in machine learning has become very popular in recent years, with many types of models proposed to comprehend and predict patterns and trends in data originating from different domains. As these models get more and more complex, it also becomes harder for users to assess and trust their results, since their internal operations are mostly hidden in black boxes. The interpretation of machine learning models is currently a hot topic in the information visualization community, with results showing that insights from machine learning models can lead to better predictions and improve the trustworthiness of the results. Due to this, multiple (and extensive) survey articles have been published recently trying to summarize the high number of original research papers published on the topic. But there is not always a clear definition of what these surveys cover, what is the overlap between them, which types of machine learning models they deal with, or what exactly is the scenario that the readers will find in each of them. In this article, we present a meta-analysis (i.e. a “survey of surveys”) of manually collected survey papers that refer to the visual interpretation of machine learning models, including the papers discussed in the selected surveys. The aim of our article is to serve both as a detailed summary and as a guide through this survey ecosystem by acquiring, cataloging, and presenting fundamental knowledge of the state of the art and research opportunities in the area. Our results confirm the increasing trend of interpreting machine learning with visualizations in the past years, and that visualization can assist in, for example, online training processes of deep learning models and enhancing trust into machine learning. However, the question of exactly how this assistance should take place is still considered as an open challenge of the visualization community.

Scenario-Based Requirements Elicitation for User-Centric Explainable AI

“…Aiming a user-centric view into decision-making for fraud detection, the domain of visual analytics has also been providing contributions through visualization tools and capabilities for fraud detection [21]. Indeed, some researchers have acknowledged the importance of the human-computer interaction or human-in-the-loop perspectives contributing to research in XAI, and the need to investigate new human-AI interfaces for Explainable AI [22][23][24][25][26]. Therefore, a user-centric perspective is essential for reviewing fraud cases, whether in XAI or visual analytics research, as every wrong decision made causes financial harm for customers and organizations.…”

A bird’s-eye view of deep learning in bioimage analysis

“…Nevertheless, the call for more explainability and interpretability of deep learning methods is legitimate and receiving growing attention in many areas of AI research [307] , [308] , [264] including computer vision [309] , [310] , [311] . A host of visual analytics tools have been developed to dissect DNNs and uncover what they have actually learned [265] , [312] , [313] . Such tools have not yet found widespread application in bioimage analysis but could help practitioners better understand the predictions made by network models.…”

Human-Centered Artificial Intelligence: Three Fresh Ideas

“…Many applications involving machine and deep learning algorithms provide post-hoc explanations of why a decision refused mortgage or parole requests. However, exploratory user interfaces using interactive visual designs offer a more likely path to successful customer adoption and acceptance (Chatzimparmpas et al, 2020;Hohman et al, 2018;Nourashrafeddin et al, 2018;Yang et al, 2020). Well-designed interactive visual interfaces will improve the work of machine learning algorithm developers and facilitate comprehension by various stakeholders.…”