DRAVA: Aligning Human Concepts with Machine Learning Latent Dimensions for the Visual Exploration of Small Multiples

Abstract

This paper proposes Drava, a novel method that utilizes Disentangled Representation learning as A Visual Analytics approach for concept-driven exploration of small multiples. While latent vectors extracted by machine learning models are widely used to organize and explore data (e.g., layout data items based on their latent vectors in a 2D space using t-SNE), they usually suffer from a lack of interpretability. Disentangled representation learning (DRL) alleviates this issue by learning vectors that encode concepts in separated dimensions and are therefore more interpretable. Even though promising, it can be challenging to apply these disentangled vectors due to the imperfections of algorithms and our limited understanding of how to support DRL-based visual exploration. To address this problem, we propose a three-step workflow where users 1) understand ML-learned concepts, 2) refine and align ML concepts with human concepts, and 3) generate new knowledge about the analyzed data through concept-driven exploration. We support this three-step workflow by providing a set of interactions based on visual piles and enhancing the DRL model with a concept adaptor that fine-tunes concept quality based on human feedback. Interactive visual piles enable users to effectively organize, summarize, and compare groups of items based on human-readable concepts. The concept adaptor assists users in modifying potential imperfect semantic vectors and better facilitates concept-driven exploration. We demonstrate the usability of Drava using four usage scenarios with different datasets. Our experiments show that the semantic latent vectors can capture human-readable concepts and the concept adaptor can further improve these semantic concepts using human feedback.

What are People Talking about in High-Dimension Data Visualization? LLM-supported Analysis of Domain Literature

“…Visualization has been widely employed for HD data analysis [21,27,40], including visualizing multidimensional data (e.g., parallel coordinates, scatterplot matrices) and visualizing more interpretable lowdimensional data derived from the original HD data (e.g., embedding visualization [11,50]).…”

“…Throughout the year, tasks related to HD data have surfaced in surveys [21,27,35,40,41], system design goals [13,22,50], evaluation metrics [6,15], and expert interviews [8]. For example, Huang et al [21] reviewed 122 visualization papers and grouped embedding visualizations based on their visual analytics tasks, such as model debugging, model explanation, and result presentation.…”

Concept Lens: Visually Analyzing the Consistency of Semantic Manipulation in GANs

“…GANravel [10], and Drava [33] introduce interactive systems for exploring concepts in latent space by leveraging human annotation and feedback. These methods are primarily focused on obtaining a single concept direction, rather than summarizing and exploring a large collection of concepts.…”

“…whether a concept is identified as a global direction [11,30] or a vector field in the latent space [37], and is domain-agnostic, e.g., we do not rely on a bank of external concepts for analysis. Further, unlike recent works that analyze a small set of latent dimensions [33], Concept Lens aims to relate a large (e.g. hundreds) set of concepts in order to provide a more complete picture of a GAN's latent space.…”

Arizona State University Orange Mall Green Infrastructure Project

“…Visual analytics systems play a crucial role in augmenting various human-AI tasks, including data augmentation [81], labeling [29,32], data exploration and analysis [25,35,55,67,70,79], bias inspection and mitigation [2,23,36,37,73], model explanation [31,47,66,69], and model refinement [30,72].…”