Intelligent Decision Assistance Versus Automated Decision-Making: Enhancing Knowledge Workers Through Explainable Artificial Intelligence

Abstract

While recent advances in AI-based automated decision-making have shown many benefits for businesses and society, they also come at a cost. It has for long been known that a high level of automation of decisions can lead to various drawbacks, such as automation bias and deskilling. In particular, the deskilling of knowledge workers is a major issue, as they are the same people who should also train, challenge and evolve AI. To address this issue, we conceptualize a new class of DSS, namely Intelligent Decision Assistance (IDA) based on a literature review of two different research streams-DSS and automation. IDA supports knowledge workers without influencing them through automated decision-making. Specifically, we propose to use techniques of Explainable AI (XAI) while withholding concrete AI recommendations. To test this conceptualization, we develop hypotheses on the impacts of IDA and provide first evidence for their validity based on empirical studies in the literature.

On the Interdependence of Reliance Behavior and Accuracy in AI-Assisted Decision-Making

“…In order to complement the AI system, the human would have to adhere to its recommendations if and only if these recommendations are correct and override them otherwise. Empirical studies have shown, however, that humans are often not able to achieve this type of appropriate reliance 2 [1,4,5,6]. Instead, we often observe that humans either over-or under-rely on AI recommendations, or simply cannot calibrate their reliance.…”

“…In the general case, we may think of under-reliance as a behavior where A < Acc AI , and overreliance as A > Acc AI . Note that there exists other work that has been thinking of these terms with respect to behavior at the level of individual decisions [4,13].…”

“…Measuring and calibrating the human reliance on AI recommendations has become a central pillar of research on AI-assisted decision-making [1,3,4]. This is especially important as both humans and AI systems are imperfect "decision-makers" with individual strengths and weaknesses [6,7,8].…”

“…Thus far, research has identified many scenarios in which underreliance or over-reliance results in reduced decision-making performance (e.g., [12,15]). In an emerging effort, more works are developed around achieving an appropriate level of reliance, which is a prerequisite for the human decision-maker to complement the AI system and ultimately improve the overall decision-making accuracy over the AI baseline [4]. In this work, we develop a framework that aims at improving our understanding of how human reliance behavior translates to decision-making accuracy.…”

Reflection Machines: Supporting Effective Human Oversight Over Medical Decision Support Systems

“…A core aspect is the identification of appropriate prompts to generate reasonable questions. This is similar to the problem of generating reasonable explanations in XAI 29 and often involves counterfactual reasoning. Counterfactuals feature prominently in human reasoning and communication about decisions.…”