A Theoretical Model for Assessing Information Validity from Multiple Observers

Dorton, Stephen L.; Frommer, Ian; Garrison, Teena M.

doi:10.1109/cogsima.2019.8724242

Cited by 2 publications

(2 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The authors then individually coded the transcribed responses for the presence or absence of the identified factors (from the literature) and the themes (from the thematic analysis process). We regularly met to compare batches of coded cases, and used a simple two-step argumentation process to resolve any discrepancies in codes (Dorton, et al, 2019). This resulted in consensus being reached on all cases.…”

Section: Methodsmentioning

confidence: 99%

A Naturalistic Investigation of Trust, AI, and Intelligence Work

Dorton

Harper

2022

Journal of Cognitive Engineering and Decision Making

Self Cite

View full text Add to dashboard Cite

Artificial Intelligence (AI) is often viewed as the means by which the intelligence community will cope with increasing amounts of data. There are challenges in adoption, however, as outputs of such systems may be difficult to trust, for a variety of factors. We conducted a naturalistic study using the Critical Incident Technique (CIT) to identify which factors were present in incidents where trust in an AI technology used in intelligence work (i.e., the collection, processing, analysis, and dissemination of intelligence) was gained or lost. We found that explainability and performance of the AI were the most prominent factors in responses; however, several other factors affected the development of trust. Further, most incidents involved two or more trust factors, demonstrating that trust is a multifaceted phenomenon. We also conducted a broader thematic analysis to identify other trends in the data. We found that trust in AI is often affected by the interaction of other people with the AI (i.e., people who develop it or use its outputs), and that involving end users in the development of the AI also affects trust. We provide an overview of key findings, practical implications for design, and possible future areas for research.

show abstract

Section: Methodsmentioning

confidence: 99%

A Naturalistic Investigation of Trust, AI, and Intelligence Work

Dorton

Harper

2022

Journal of Cognitive Engineering and Decision Making

Self Cite

View full text Add to dashboard Cite

show abstract

“…We conducted semi-structured interviews using an interview prompt containing questions designed to understand the mental model of users, understanding how they view SATC2, identifying what their biggest challenges are, and what their vision for what an ideal SATC2 HMI would look like. We generated transcripts for each interview, then used a structured technique to conduct thematic analysis of responses [24]. This structured and repeatable method enabled us to glean numerous insights form interviews, while ensuring validity of the findings to the fullest extent possible [25].…”

Section: Rapid Prototypingmentioning

confidence: 99%

A Case for User-Centered Design in Satellite Command and Control

et al. 2021

Self Cite

View full text Add to dashboard Cite

The prevalence of unique, disparate satellite command and control (SATC2) systems in current satellite operations is problematic. As such, the United States Air Force aims to consolidate SATC2 systems into an enterprise solution that utilizes a common Human–Machine Interface (HMI). We employed a User-Centered Design (UCD) approach including a variety of methods from design thinking and human factors engineering to develop a solution that is effective, efficient, and meets operator needs. During a summative test event, we found that users had significantly higher situation awareness, lower workload, and higher subjective usability while using the HMI developed via UCD over the existing, or legacy, HMI. This case study serves as evidence to support the assertion that involving users early and often has positive and tangible effects on the development of aerospace systems.

show abstract

A Theoretical Model for Assessing Information Validity from Multiple Observers

Cited by 2 publications

References 16 publications

A Naturalistic Investigation of Trust, AI, and Intelligence Work

A Naturalistic Investigation of Trust, AI, and Intelligence Work

A Case for User-Centered Design in Satellite Command and Control

Contact Info

Product

Resources

About