Mediators: Conversational Agents Explaining NLP Model Behavior

Feldhus, Nils; Ravichandran, Ajay Madhavan; Möller, Sebastian

doi:10.48550/arxiv.2206.06029

Cited by 3 publications

(3 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In parallel, a dialog manager tracks the dialogue state and guides the conversation towards the user goals. Finally, natural language generation techniques (often rule-based) are used to respond to the user (Feldhus et al, 2022). The NLP methods heavily draw on and contribute to the CA's task knowledge base.…”

Section: Introductionmentioning

confidence: 99%

Knowledge Transfer between Humans and Conversational Agents: A Review, Organizing Framework, and Future Directions

Sukhwal,

Cui,

Kankanhalli

2023

Proceedings of the Annual Hawaii International Conference on System Sciences

View full text Add to dashboard Cite

Conversational agents (CAs) that use natural language to interact with humans are becoming ubiquitous in our daily lives. For CAs to perform effectively, knowledge transfer between human users and CAs is vital to complete tasks and to build common understanding with humans. While such knowledge transfer is important, relatively less research attention has been paid to it. Overall, we lack a systematic overview of how knowledge transfer can be facilitated between humans and CAs. Motivated thus, this article presents a literature review of empirical IS, HCI and Communications studies on the knowledge transfer between humans and CAs. We analyzed papers on this topic, synthesized the studies based on the antecedents, directions, processes, and outcomes of knowledge transfer. We contribute by providing a systematic understanding of research on knowledge transfer in human-CA interactions, proposing an organizing framework, identifying gaps in prior work, and outlining key future research directions.

show abstract

Section: Introductionmentioning

confidence: 99%

Knowledge Transfer between Humans and Conversational Agents: A Review, Organizing Framework, and Future Directions

Sukhwal,

Cui,

Kankanhalli

2023

Proceedings of the Annual Hawaii International Conference on System Sciences

View full text Add to dashboard Cite

show abstract

“…In the literature, researchers have suggested some prototype designs for generating explanations using natural language. However, these initial designs address specific explanations and model classes, limiting their applicability in general conversational explainability settings [62,19].…”

Section: Introductionmentioning

confidence: 99%

TalkToModel: Explaining Machine Learning Models with Interactive Natural Language Conversations

Slack

Krishna²,

Lakkaraju

et al. 2022

Preprint

View full text Add to dashboard Cite

Practitioners increasingly use machine learning (ML) models, yet they have become more complex and harder to understand. To address this issue, researchers have proposed techniques to explain model predictions. However, practitioners struggle to use explainability methods because they do not know which to choose and how to interpret the results. We address these challenges by introducing TalkToModel: an interactive dialogue system that enables users to explain ML models through natural language conversations. TalkToModel comprises three components: 1) an adaptive dialogue engine that interprets natural language and generates meaningful responses, 2) an execution component, which constructs the explanations used in the conversation, 3) a conversational interface. In real-world evaluations, 73% of healthcare workers agreed they would use TalkToModel over existing systems for understanding a disease prediction model, and 85% of ML professionals agreed TalkToModel was easier to use, demonstrating that TalkToModel is highly effective for model explainability.

show abstract

“…Many model-agnostic visualizations [2,8,17,30,69,71] focus on input-output model behavior and are generally applicable to different models. For example, What-If Tool [71] allows users to understand model behavior concerning feature importance, different inputs, and other hypothetical situations.…”

Section: Visualization For Understanding Nlp Modelsmentioning

confidence: 99%

Commonsensevis : visualizing and understanding commonsense reasoning capabilities of natural language models

Huang¹

View full text Add to dashboard Cite

Model editing Model understanding Model prediction Behavior diagnosis Commonsense knowledge extraction Multilevel visualization where do adults use glue sticks? A. classroom B. o ce C. desk drawer adult glue stick o ce work use atLocation commonsense Do NLP models have commonsense?Users NLP model Fig. 1: Since commonsense knowledge is not explicitly stated, it is challenging to conduct a scalable analysis of what commonsense knowledge NLP models do (not) learn. We employ a knowledge graph to derive implicit commonsense in the model input as context information. Then, we use it to align model behavior with human reasoning through multi-level interactive visualizations. Thereafter, users can understand, diagnose, and edit specific knowledge areas where models do not perform well.Abstract-Recently, large pretrained language models have achieved compelling performance on commonsense benchmarks. Nevertheless, it is unclear what commonsense knowledge the models learn and whether they solely exploit spurious patterns. Feature attributions are popular explainability techniques that identify important input concepts for model outputs. However, commonsense knowledge tends to be implicit and rarely explicitly presented in inputs. These methods cannot infer models' implicit reasoning over mentioned concepts. We present CommonsenseVIS, a visual explanatory system that utilizes external commonsense knowledge bases to contextualize model behavior for commonsense question-answering. Specifically, we extract relevant commonsense knowledge in inputs as references to align model behavior with human knowledge. Our system features multi-level visualization and interactive model probing and editing for different concepts and their underlying relations. Through a user study, we show that CommonsenseVIS helps NLP experts conduct a systematic and scalable visual analysis of models' relational reasoning over concepts in different situations.

show abstract

Mediators: Conversational Agents Explaining NLP Model Behavior

Cited by 3 publications

References 0 publications

Knowledge Transfer between Humans and Conversational Agents: A Review, Organizing Framework, and Future Directions

Knowledge Transfer between Humans and Conversational Agents: A Review, Organizing Framework, and Future Directions

TalkToModel: Explaining Machine Learning Models with Interactive Natural Language Conversations

Commonsensevis : visualizing and understanding commonsense reasoning capabilities of natural language models

Contact Info

Product

Resources

About