The level of measurement of trust in automation

Wei, Jiajun; Bolton, Matthew L.; Humphrey, Lisa

doi:10.1080/1463922x.2020.1766596

Cited by 12 publications

(11 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, additional care is still needed with regards to ensuring that concept definitions are operationalized accurately in empirical research. Existing survey measures are available to measure many of the concepts defined in our model (e.g., Chancey et al, 2017), as are discussions of how to measure these concepts (Kohn, De Visser, Wiese, Lee, & Shaw, 2021; Wei, Bolton, & Humphrey, 2020). However, researchers must ensure the content validity of any measures they use by checking that they correspond accurately and uniquely with the target construct.…”

Section: Discussion and Ways Forwardmentioning

confidence: 99%

A Conceptual Model of Trust, Perceived Risk, and Reliance on AI Decision Aids

Solberg

Kaarstad

Eitrheim

et al. 2022

Group & Organization Management

View full text Add to dashboard Cite

There is increasing interest in the use of artificial intelligence (AI) to improve organizational decision-making. However, research indicates that people’s trust in and choice to rely on “AI decision aids” can be tenuous. In the present paper, we connect research on trust in AI with Mayer, Davis, and Schoorman’s (1995) model of organizational trust to elaborate a conceptual model of trust, perceived risk, and reliance on AI decision aids at work. Drawing from the trust in technology, trust in automation, and decision support systems literatures, we redefine central concepts in Mayer et al.’s (1995) model, expand the model to include new, relevant constructs (like perceived control over an AI decision aid), and refine propositions about the relationships expected in this context. The conceptual model put forward presents a framework that can help researchers studying trust in and reliance on AI decision aids develop their research models, build systematically on each other’s research, and contribute to a more cohesive understanding of the phenomenon. Our paper concludes with five next steps to take research on the topic forward.

show abstract

Section: Discussion and Ways Forwardmentioning

confidence: 99%

A Conceptual Model of Trust, Perceived Risk, and Reliance on AI Decision Aids

Solberg

Kaarstad

Eitrheim

et al. 2022

Group & Organization Management

View full text Add to dashboard Cite

show abstract

“…In [44], Wei et al pointed out that, despite the availability of a large number of research in trust measurement, it is not clear what psychometric level of measurement is most appropriate for trust in automation. They discussed the various levels of psychometric scales (nominal, ordinal, interval, and ratio) and investigated what could be a step toward having a standardized level of measurement for trust.…”

Section: B Trust Measurement-scales and Approachesmentioning

confidence: 99%

“…Self-Report Measurements: These approaches measure trust by collecting responses in the form of surveys and questionnaires from participants. There is currently no standard for assessing human-machine trust [44], [48]. At present, it is measured by researchers using custom scales or validated measurements.…”

Section: B Trust Measurement-scales and Approachesmentioning

confidence: 99%

A Review on Human–Machine Trust Evaluation: Human-Centric and Machine-Centric Perspectives

Gebru

Zeleke

Blankson

et al. 2022

IEEE Trans. Human-Mach. Syst.

View full text Add to dashboard Cite

As complex autonomous systems become increasingly ubiquitous, their deployment and integration into our daily lives will become a significant endeavor. Human-machine trust relationship is now acknowledged as one of the primary aspects that characterize a successful integration. In the context of human-machine interaction (HMI), proper use of machines and autonomous systems depends both on the human and machine counterparts. On one hand, it depends on how well the human relies on the machine regarding the situation or task at hand based on willingness and experience. On the other hand, it depends on how well the machine carries out the task and how well it conveys important information on how the job is done. Furthermore, proper calibration of trust for effective HMI requires the factors affecting trust to be properly accounted for and their relative importance to be rightly quantified. In this article, the functional understanding of humanmachine trust is viewed from two perspectives-human-centric and machine-centric. The human aspect of the discussion outlines factors, scales, and approaches, which are available to measure and calibrate human trust. The discussion on the machine aspect spans trustworthy artificial intelligence, built-in machine assurances, and ethical frameworks of trustworthy machines.

show abstract

“…To fill this gap, we developed a method [introduced by Humphrey (2019, 2020) and refined by Biltekoff, Bolton, and Humphrey (n.d.); Bolton, Biltekoff, and Humphrey (n.d.)] that enables researchers to empirically evaluate the level of measurement of subjective psychometric ratings scales. This method was used to assess multiple ratings scales that are commonly used in human factors engineering: trust (Wei et al, 2020); the three dimensions of the situation awareness rating technique (SART) along with situation awareness itself (Bolton et al, n.d.); and the six dimensions of the NASA task load index (NASA-TLX) (Biltekoff et al, n.d.).…”

Section: Introductionmentioning

confidence: 99%

On the Level of Measurement of Subjective Psychometric Ratings

Bolton

Biltekoff

Wei

et al. 2022

Proceedings of the Human Factors and Ergonomics Society Annual

Self Cite

View full text Add to dashboard Cite

Subjective psychometric ratings are critical tools in human factors engineering. Despite widespread use, there is controversy about whether such ratings can be treated at cardinal levels of measurement. Answering this question is important given that a measure’s level determines what mathematical and statistical operations can be meaningfully applied to them. This paper synthesizes results from an effort (published over several papers) that developed a method for assessing the level of measurement of subjective ratings and used it to evaluate common trust, situation awareness, and workload metrics. This paper synthesizes these results to determine that subjective measures should rarely, if ever, be treated as ratio. It also found that, in a population analysis, some measures (like all the ones considered) can be treated as interval. In all other situations, and especially in analyses or modeling efforts for individuals or small sample sizes, ordinal is the safest default.

show abstract

The level of measurement of trust in automation

Cited by 12 publications

References 51 publications

A Conceptual Model of Trust, Perceived Risk, and Reliance on AI Decision Aids

A Conceptual Model of Trust, Perceived Risk, and Reliance on AI Decision Aids

A Review on Human–Machine Trust Evaluation: Human-Centric and Machine-Centric Perspectives

On the Level of Measurement of Subjective Psychometric Ratings

Contact Info

Product

Resources

About