Guide Me in Analysis: A Framework for Guidance Designers

Ceneda, Davide; Andrienko, Natalia; Andrienko, Gennady; Gschwandtner, Theresia; Miksch, Silvia; Piccolotto, Nikolaus; Schreck, Tobias; Streit, Marc; Suschnigg, Josef; Tominski, Christian

doi:10.1111/cgf.14017

Cited by 23 publications

(38 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the current setting, where domain experts look at large spreadsheets of correlation values, there is no clear entry point and no systematic process in place for analyzing potentially problematic bias. Providing such clear entry points and a streamlined process towards discovering such biased labels is key in a visualization approach for domain experts [13,14]. Additionally, progressing through the thousands of labels, essentially dividing the label space into problematic and non-problematic after an analysis session is something that visualizations need to support.…”

Section: Technical Challengesmentioning

confidence: 99%

“…G1 Visual Guidance. Any visual analytics approach should provide systematic guidance for its users [13,14]. This guideline just gets more important when dealing with large data, such as in detecting problematic bias in large label spaces.…”

Section: Design Guidelinesmentioning

confidence: 99%

“…datasets [13]. Thus, we implemented both the removal of annotations that are viewed as non-critical and the flagging of annotations that expose problematic bias.…”

Section: Guidance Visualizationmentioning

confidence: 99%

See 2 more Smart Citations

Visual Identification of Problematic Bias in Large Label Spaces

Bäuerle¹,

Turker²,

Burke³

et al. 2022

Preprint

View full text Add to dashboard Cite

While the need for well-trained, fair ML systems is increasing ever more, measuring fairness for modern models and datasets is becoming increasingly difficult as they grow at an unprecedented pace. One key challenge in scaling common fairness metrics to such models and datasets is the requirement of exhaustive ground truth labeling, which cannot always be done. Indeed, this often rules out the application of traditional analysis metrics and systems. At the same time, ML-fairness assessments cannot be made algorithmically, as fairness is a highly subjective matter. Thus, domain experts need to be able to extract and reason about bias throughout models and datasets to make informed decisions. While visual analysis tools are of great help when investigating potential bias in DL models, none of the existing approaches have been designed for the specific tasks and challenges that arise in large label spaces. Addressing the lack of visualization work in this area, we propose guidelines for designing visualizations for such large label spaces, considering both technical and ethical issues. Our proposed visualization approach can be integrated into classical model and data pipelines, and we provide an implementation of our techniques open-sourced as a TensorBoard plug-in. With our approach, different models and datasets for large label spaces can be systematically and visually analyzed and compared to make informed fairness assessments tackling problematic bias.

show abstract

Section: Technical Challengesmentioning

confidence: 99%

Section: Design Guidelinesmentioning

confidence: 99%

See 1 more Smart Citation

Visual Identification of Problematic Bias in Large Label Spaces

Bäuerle¹,

Turker²,

Burke³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Rind et al give some typical approaches for the integration of knowledge in visualization systems [40] and discuss an example for knowledge assistance with direct knowledge access [48]. Ceneda et al developed a framework to guide the development of knowledge-assisted visualization systems [9].…”

Section: Implementation Steps For Kava Processes Withmentioning

confidence: 99%

Knowledge Rocks:Adding Knowledge Assistance to Visualization Systems

Lohfink,

Anton,

Leitte

et al. 2021

Preprint

View full text Add to dashboard Cite

We present Knowledge Rocks, an implementation strategy and guideline for augmenting visualization systems to knowledgeassisted visualization systems, as defined by the KAVA model. Visualization systems become more and more sophisticated. Hence, it is increasingly important to support users with an integrated knowledge base in making constructive choices and drawing the right conclusions. We support the effective reactivation of visualization software resources by augmenting them with knowledge-assistance. To provide a general and yet supportive implementation strategy, we propose an implementation process that bases on an applicationagnostic architecture. This architecture is derived from existing knowledge-assisted visualization systems and the KAVA model. Its centerpiece is an ontology that is able to automatically analyze and classify input data, linked to a database to store classified instances. We discuss design decisions and advantages of the KR framework and illustrate its broad area of application in diverse integration possibilities of this architecture into an existing visualization system. In addition, we provide a detailed case study by augmenting an it-security system with knowledge-assistance facilities.

show abstract

“…Analysts do not know which lags to select and are generally aware of this knowledge gap. Since they are in an interactive VA session, we used the guidance design framework [61] to design appropriate guidance [52]. As discussed in Section 3.3, the analysis goal is to obtain a new/interesting result.…”

Section: Parameter Selectionmentioning

confidence: 99%

TBSSvis: Visual Analytics for Temporal Blind Source Separation

Piccolotto¹,

Bögl²,

Gschwandtner³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Temporal Blind Source Separation (TBSS) is used to obtain the true, underlying processes from noisy temporal multivariate data, such as electrocardiograms. While these algorithms are widely used, the involved tasks are not well supported in current visualization tools, which offer only textbased interactions and static images. Analysts are limited in analyzing and comparing obtained results, which consist of diverse data such as matrices and ensembles of time series. Additionally, parameters have a big impact on separation performance, but as a consequence of improper tooling analysts currently do not consider the whole parameter space. We propose to solve these problems by applying visual analytics (VA) principles. To this end, we developed a task abstraction and visualization design in a user-centered design process. We present TBSSvis, an interactive webbased VA prototype, which we evaluated in two qualitative user studies. Feedback and observations from these studies show that TBSSvis supports the actual workflow and combination of interactive visualizations that facilitate the tasks involved in analyzing TBBS results. It also provides guidance to facilitate informed parameter selection and the analysis of the data at hand.

show abstract

Guide Me in Analysis: A Framework for Guidance Designers

Cited by 23 publications

References 35 publications

Visual Identification of Problematic Bias in Large Label Spaces

Visual Identification of Problematic Bias in Large Label Spaces

Knowledge Rocks:Adding Knowledge Assistance to Visualization Systems

TBSSvis: Visual Analytics for Temporal Blind Source Separation

Contact Info

Product

Resources

About