Measuring and Understanding Trust Calibrations for Automated Systems: A Survey of the State-Of-The-Art and Future Directions

Abstract

Trust has been recognized as a central variable to explain the resistance to using automated systems (under-trust) and the overreliance on automated systems (over-trust). To achieve appropriate reliance, users’ trust should be calibrated to reflect a system’s capabilities. Studies from various disciplines have examined different interventions to attain such trust calibration. Based on a literature body of 1000+ papers, we identified 96 relevant publications which aimed to calibrate users’ trust in automated systems. To provide an in-depth overview of the state-of-the-art, we reviewed and summarized measurements of the trust calibration, interventions, and results of these efforts. For the numerous promising calibration interventions, we extract common design choices and structure these into four dimensions of trust calibration interventions to guide future studies. Our findings indicate that the measurement of the trust calibration often limits the interpretation of the effects of different interventions. We suggest future directions for this problem.

Will You Accept the AI Recommendation? Predicting Human Behavior in AI-Assisted Decision Making

“…Extensive research has examined the appropriateness of human reliance on AI systems (including broadly automated systems and robotics) [7,42,48,50,51,83,86]. In recent HCI studies, the focus has shifted from merely increasing trust in AI to facilitating appropriate trust and reliance [6,8,47,71,79,80,86,88,90].…”

“…Extensive research has examined the appropriateness of human reliance on AI systems (including broadly automated systems and robotics) [7,42,48,50,51,83,86]. In recent HCI studies, the focus has shifted from merely increasing trust in AI to facilitating appropriate trust and reliance [6,8,47,71,79,80,86,88,90]. Two widely recognized phenomena, automation bias [61] and algorithm aversion [16], highlight the challenge individuals face in aligning their trust and reliance with AI system capabilities.…”

“…In AI-assisted decision-making research, the focus has turned toward managing trust and reliance on a case-by-case basis [39,80,86]. Key concepts in this context are trust and reliance [39,80,86].…”

“…In AI-assisted decision-making research, the focus has turned toward managing trust and reliance on a case-by-case basis [39,80,86]. Key concepts in this context are trust and reliance [39,80,86]. Trust reflects subjective perceptions of AI, often assessed using self-report scales, while reliance pertains to objective behavior in response to AI systems [2,47,84,90,91].…”

“…The field of AI-assisted decision-making is gaining significant attention within HCI communities [6,47,49,71,73,79,80,86,88,90,94].…”