Towards Visual Explainable Active Learning for Zero-Shot Classification

Jia, Shichao; Li, Zeyu; Zhang, Jiawan

doi:10.1109/tvcg.2021.3114793

Cited by 22 publications

(11 citation statements)

References 59 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Users can label instances recommended by an active learning algorithm or select informative instances to label with the help of visualization, which are used to further improve the underlying model. Such an integration is also substantiated by other work [25], [26], [27], [28], [29], [30], [31].…”

Section: Visualization For Annotation Quality Improvementsupporting

confidence: 76%

Towards Better Caption Supervision for Object Detection

Chen¹,

Wu²,

Wang³

et al. 2022

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

show abstract

Section: Visualization For Annotation Quality Improvementsupporting

confidence: 76%

Towards Better Caption Supervision for Object Detection

Chen¹,

Wu²,

Wang³

et al. 2022

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

show abstract

“…RetainVis [ 31 ] leverages an attention module where users can directly edit attention weights to update the model. Jia et al [ 25 ] designed Semantic Navigator, which guides users to steer the model by editing a class-attribute matrix in a zero-shot learning process.…”

Section: Related Workmentioning

confidence: 99%

Polyphony: an Interactive Transfer Learning Framework for Single-Cell Data Analysis

Cheng

Keller

et al. 2023

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

Fig. 1. The interface of Polyphony contains three views: the comparison view (A), the anchor set view (B), and the marker view (C).The comparison view provides an overview of the joint embedding space, and offers users interactions to inspect (A1), delete (A2), and add (A3) anchors. The anchor set view orders the anchors in a table, supporting inspection and comparison of different anchors (B1-2). The marker view shows the significant genes (C1) for the query and reference cells from a focal anchor.

show abstract

“…Apart from showing latent vectors, previous studies have combined interactive visual analytics with interactive or explainable ML to introduce interpretability into the analysis of latent vectors [18,23,68]. Several studies [18,20,59] used DRL to extract semantic dimensions and associate model performance with human concepts (e.g., brightness of images, location of objects).…”

Section: Related Workmentioning

confidence: 99%

“…The semantic dimensions learned by DRL are directly used without refinement, mostly because they are low-level concepts that can be easily extracted by ML. Jia et al [23] proposed a visual explainable active learning approach that asks users questions and uses their answers to learn explainable attributes that can be used to classify images from unseen classes. Zhao et al [68] proposed a visualization tool where users can explore and label image patches with a certain concept.…”

Section: Related Workmentioning

confidence: 99%

DRAVA: Aligning Human Concepts with Machine Learning Latent Dimensions for the Visual Exploration of Small Multiples

Wang¹,

L’Yi²,

Gehlenborg³

2022

Preprint

View full text Add to dashboard Cite

This paper proposes Drava, a novel method that utilizes Disentangled Representation learning as A Visual Analytics approach for concept-driven exploration of small multiples. While latent vectors extracted by machine learning models are widely used to organize and explore data (e.g., layout data items based on their latent vectors in a 2D space using t-SNE), they usually suffer from a lack of interpretability. Disentangled representation learning (DRL) alleviates this issue by learning vectors that encode concepts in separated dimensions and are therefore more interpretable. Even though promising, it can be challenging to apply these disentangled vectors due to the imperfections of algorithms and our limited understanding of how to support DRL-based visual exploration. To address this problem, we propose a three-step workflow where users 1) understand ML-learned concepts, 2) refine and align ML concepts with human concepts, and 3) generate new knowledge about the analyzed data through concept-driven exploration. We support this three-step workflow by providing a set of interactions based on visual piles and enhancing the DRL model with a concept adaptor that fine-tunes concept quality based on human feedback. Interactive visual piles enable users to effectively organize, summarize, and compare groups of items based on human-readable concepts. The concept adaptor assists users in modifying potential imperfect semantic vectors and better facilitates concept-driven exploration. We demonstrate the usability of Drava using four usage scenarios with different datasets. Our experiments show that the semantic latent vectors can capture human-readable concepts and the concept adaptor can further improve these semantic concepts using human feedback.

show abstract

Towards Visual Explainable Active Learning for Zero-Shot Classification

Cited by 22 publications

References 59 publications

Towards Better Caption Supervision for Object Detection

Towards Better Caption Supervision for Object Detection

Polyphony: an Interactive Transfer Learning Framework for Single-Cell Data Analysis

DRAVA: Aligning Human Concepts with Machine Learning Latent Dimensions for the Visual Exploration of Small Multiples

Contact Info

Product

Resources

About