Guidelines for Human-AI Interaction

Amershi, Saleema; Weld, Dan; Vorvoreanu, Mihaela; Fourney, Adam; Nushi, Besmira; Collisson, Penny; Suh, Jina; Iqbal, Shamsi T.; Bennett, Paul; Inkpen, Kori; Teevan, Jaime; Kikin-Gil, Ruth; Horvitz, Eric

doi:10.1145/3290605.3300233

Cited by 862 publications

(562 citation statements)

References 28 publications

Supporting

Mentioning

547

Contrasting

Unclassified

Order By: Relevance

“…Other guidelines (Horvitz, 1999;Dudley & Kristensson, 2018) paved the way for Microsoft's Guidelines for AI-Human Interaction (https://www.microsoft.com/enus/research/project/guidelines-for-human-ai-interaction/), which has 18 guidelines for initial use, normal use, coping with problems, and changes over time (Amershi et al, 2019). These guidelines emphasize user understanding and control, while addressing ways for the system to "make clear why the system did what it did" and "learn from user behavior.…”

Section: Prometheus Principles and Examplesmentioning

confidence: 99%

Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy

Shneiderman

2020

International Journal of Human–Computer Interaction

450

196

View full text Add to dashboard Cite

Well-designed technologies that offer high levels of human control and high levels of computer automation can increase human performance, leading to wider adoption. The Human-Centered Artificial Intelligence (HCAI) framework clarifies how to (1) design for high levels of human control and high levels of computer automation so as to increase human performance, (2) understand the situations in which full human control or full computer control are necessary, and (3) avoid the dangers of excessive human control or excessive computer control. The methods of HCAI are more likely to produce designs that are Reliable, Safe & Trustworthy (RST). Achieving these goals will dramatically increase human performance, while supporting human self-efficacy, mastery, creativity, and responsibility.

show abstract

Section: Prometheus Principles and Examplesmentioning

confidence: 99%

Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy

Shneiderman

2020

International Journal of Human–Computer Interaction

450

196

View full text Add to dashboard Cite

show abstract

“…These empirical findings guided the designers and system builders to propose human-in-the-loop data science design principles [1,2,11,27,28,59]. Gil et al surveyed papers about building machine learning systems and developed a set of design guidelines for building human-centered machine learning systems [11].…”

Section: Human-in-the-loop Machine Learningmentioning

confidence: 99%

“…Gil et al surveyed papers about building machine learning systems and developed a set of design guidelines for building human-centered machine learning systems [11]. Amershi et al in parallel reviewed a broader spectrum of AI applications and proposed a set of design suggestions for AI system in general [2], some of which are overlapping with the ones in [11].…”

Section: Human-in-the-loop Machine Learningmentioning

confidence: 99%

Trust in AutoML

Drozdal

Weisz²,

Wang³

et al. 2020

Proceedings of the 25th International Conference on Intelligent User Interfaces

View full text Add to dashboard Cite

We explore trust in a relatively new area of data science: Automated Machine Learning (AutoML). In AutoML, AI methods are used to generate and optimize machine learning models by automatically engineering features, selecting models, and optimizing hyperparameters. In this paper, we seek to understand what kinds of information influence data scientists' trust in the models produced by AutoML? We operationalize trust as a willingness to deploy a model produced using automated methods. We report results from three studies -qualitative interviews, a controlled experiment, and a card-sorting task -to understand the information needs of data scientists for establishing trust in AutoML systems. We find that including transparency features in an AutoML tool increased user trust and understandability in the tool; and out of all proposed features, model performance metrics and visualizations are the most important information to data scientists when establishing their trust with an AutoML tool. CCS CONCEPTS• Human-centered computing → User studies; Empirical studies in HCI ; • Computing methodologies → Artificial intelligence.

show abstract

“…The computing research community may need to cross disciplinary boundaries and draw on the collective expertise of social scientists and disabilities scholars in order to better understand how humans interact with AI systems and provide the appropriate guidelines, e.g. [1], for developers to keep in mind when designing new AI-based systems.…”

Section: Human-ai Interaction Requires New Researchmentioning

confidence: 99%

Artificial intelligence fairness in the context of accessibility research on intelligent systems for people who are deaf or hard of hearing

Kafle

Glasser

Al-khazraji

et al. 2020

SIGACCESS Access. Comput.

View full text Add to dashboard Cite

We discuss issues of Artificial Intelligence (AI) fairness for people with disabilities, with examples drawn from our research on HCI for AI-based systems for people who are Deaf or Hard of Hearing (DHH). In particular, we discuss the need for inclusion of data from people with disabilities in training sets, the lack of interpretability of AI systems, ethical responsibilities of access technology researchers and companies, the need for appropriate evaluation metrics for AI-based access technologies (to determine if they are ready to be deployed and if they can be trusted by users), and the ways in which AI systems influence human behavior and influence the set of abilities needed by users to successfully interact with computing systems.

show abstract

Guidelines for Human-AI Interaction

Cited by 862 publications

References 28 publications

Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy

Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy

Trust in AutoML

Artificial intelligence fairness in the context of accessibility research on intelligent systems for people who are deaf or hard of hearing

Contact Info

Product

Resources

About