Toward an Ecologically Valid Conceptual Framework for the Use of Artificial Intelligence in Clinical Settings: Need for Systems Thinking, Accountability, Decision-making, Trust, and Patient Safety Considerations in Safeguarding the Technology and Clinicians

Abstract

The health care management and the medical practitioner literature lack a descriptive conceptual framework for understanding the dynamic and complex interactions between clinicians and artificial intelligence (AI) systems. As most of the existing literature has been investigating AI’s performance and effectiveness from a statistical (analytical) standpoint, there is a lack of studies ensuring AI’s ecological validity. In this study, we derived a framework that focuses explicitly on the interaction between AI and clinicians. The proposed framework builds upon well-established human factors models such as the technology acceptance model and expectancy theory. The framework can be used to perform quantitative and qualitative analyses (mixed methods) to capture how clinician-AI interactions may vary based on human factors such as expectancy, workload, trust, cognitive variables related to absorptive capacity and bounded rationality, and concerns for patient safety. If leveraged, the proposed framework can help to identify factors influencing clinicians’ intention to use AI and, consequently, improve AI acceptance and address the lack of AI accountability while safeguarding the patients, clinicians, and AI technology. Overall, this paper discusses the concepts, propositions, and assumptions of the multidisciplinary decision-making literature, constituting a sociocognitive approach that extends the theories of distributed cognition and, thus, will account for the ecological validity of AI.

The Impact of the Implementation of Safety Measures on Frontline Workers’ Safety Accountability: A Saudi Arabian Case Study of a Well Intervention Business Model

“…By showing SCT s positive and significant effect on SA, the SEM analysis confirmed the acceptance of hypothesis H3. These findings are consistent with previous findings by [33,[41][42][43][44]. Among Aramco's frontline workers, SCT insights have a significant effect on the safety outcomes.…”

“…Similarly, the findings of [33]'s empirical investigation indicates that SP, ST and SC with feedback and SIs promote frontline workers' safety behaviors. From the medical point of view, the lack of accountability and management creates barriers to decision making [43]. The findings of [44]'s investigation posit that there is a direct link between safety risk management and flight safety performance.…”

“…From a time horizon point of view, the study is cross-sectional data or a short-term analysis involving data collection at a specific time. Moreover, various scholars, such as [9,33,[42][43][44]61], have applied the same strategy when investigating safety and environmental issues in the oil and gas fields.…”

Acceptance, initial trust formation, and human biases in artificial intelligence: Focus on clinicians

“…Research around healthcare AI can be categorized into two groups, one that promises multiple benefits of the AI ( 1 , 2 ) and the other that criticizes the technology ( 3 ). Although both the body of work is crucial and noteworthy, not much evidence exists that can confirm AIs' negative impact on patient safety or health outcome.…”

Factors influencing clinicians' willingness to use an AI-based clinical decision support system

“…Still, not much work has been invested in understanding how these technologies are perceived by clinicians, mainly existing AI users, and do they want to use them ? ( 5 ) Medical professionals often consider the potential of AI to be limited ( 6 ), and their perceptions can impact their intent to use or adopt AI in medicine. Therefore, it is important to understand the human factors that influence clinicians' intent to use AI; otherwise, AI would remain underused, keeping the healthcare industry benefiting from the technology.…”