Teamwork in clinical reasoning – cooperative or parallel play?

Olson, Andrew; Durning, Steven J.; Branson, Carolina Fernandez; Sick, Brian; Lane, Kathleen P; Rencic, Joseph

doi:10.1515/dx-2020-0020

Cited by 27 publications

(17 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In both cases, we can frame these protocols as structured ways to leverage the collective intelligence of an ensemble of image readers, that is to gain a higher decision accuracy than involving each reader individually (at the expense of some efficiency, but not necessarily so), avoiding some pitfalls of direct interaction where "the more" is not always "the better", mainly due to bandwagon effect, priming and truth bias [25,34,43]. In fact, in all the protocols described in Sect.…”

Section: Discussionmentioning

confidence: 99%

Studying human-AI collaboration protocols: the case of the Kasparov’s law in radiological double reading

Cabitza

Campagner

Sconfienza

2021

Health Inf Sci Syst

View full text Add to dashboard Cite

Purpose The integration of Artificial Intelligence into medical practices has recently been advocated for the promise to bring increased efficiency and effectiveness to these practices. Nonetheless, little research has so far been aimed at understanding the best human-AI interaction protocols in collaborative tasks, even in currently more viable settings, like independent double-reading screening tasks. Methods To this aim, we report about a retrospective case–control study, involving 12 board-certified radiologists, in the detection of knee lesions by means of Magnetic Resonance Imaging, in which we simulated the serial combination of two Deep Learning models with humans in eight double-reading protocols. Inspired by the so-called Kasparov’s Laws, we investigate whether the combination of humans and AI models could achieve better performance than AI models alone, and whether weak reader, when supported by fit-for-use interaction protocols, could out-perform stronger readers. Results We discuss two main findings: groups of humans who perform significantly worse than a state-of-the-art AI can significantly outperform it if their judgements are aggregated by majority voting (in concordance with the first part of the Kasparov’s law); small ensembles of significantly weaker readers can significantly outperform teams of stronger readers, supported by the same computational tool, when the judgments of the former ones are combined within “fit-for-use” protocols (in concordance with the second part of the Kasparov’s law). Conclusion Our study shows that good interaction protocols can guarantee improved decision performance that easily surpasses the performance of individual agents, even of realistic super-human AI systems. This finding highlights the importance of focusing on how to guarantee better co-operation within human-AI teams, so to enable safer and more human sustainable care practices.

show abstract

Section: Discussionmentioning

confidence: 99%

Studying human-AI collaboration protocols: the case of the Kasparov’s law in radiological double reading

Cabitza

Campagner

Sconfienza

2021

Health Inf Sci Syst

View full text Add to dashboard Cite

show abstract

“…20 Further, our model mostly addresses individual decision-making and improvement and does not take into account other important theories that are important for clinical reasoning and diagnosis, such as situated cognition or other social cognitive theories. 21 Similarly, we did not formally address the use of artificial or augmented intelligence, which is emerging as an important consideration and adjunct in modern clinical reasoning. Lastly, much of the success of this model relies on accurate and thoughtful clinical documentation; in the USA, much of the major driving force shaping clinical documentation is related to billing and coding, thus leaving actual clinical reasoning obfuscated.…”

Section: Limitationsmentioning

confidence: 99%

Improving diagnostic performance through feedback: the Diagnosis Learning Cycle

et al. 2021

Self Cite

View full text Add to dashboard Cite

BackgroundErrors in reasoning are a common cause of diagnostic error. However, it is difficult to improve performance partly because providers receive little feedback on diagnostic performance. Examining means of providing consistent feedback and enabling continuous improvement may provide novel insights for diagnostic performance.MethodsWe developed a model for improving diagnostic performance through feedback using a six-step qualitative research process, including a review of existing models from within and outside of medicine, a survey, semistructured interviews with individuals working in and outside of medicine, the development of the new model, an interdisciplinary consensus meeting, and a refinement of the model.ResultsWe applied theory and knowledge from other fields to help us conceptualise learning and comparison and translate that knowledge into an applied diagnostic context. This helped us develop a model, the Diagnosis Learning Cycle, which illustrates the need for clinicians to be given feedback about both their confidence and reasoning in a diagnosis and to be able to seamlessly compare diagnostic hypotheses and outcomes. This information would be stored in a repository to allow accessibility. Such a process would standardise diagnostic feedback and help providers learn from their practice and improve diagnostic performance. This model adds to existing models in diagnosis by including a detailed picture of diagnostic reasoning and the elements required to improve outcomes and calibration.ConclusionA consistent, standard programme of feedback that includes representations of clinicians’ confidence and reasoning is a common element in non-medical fields that could be applied to medicine. Adapting this approach to diagnosis in healthcare is a promising next step. This information must be stored reliably and accessed consistently. The next steps include testing the Diagnosis Learning Cycle in clinical settings.

show abstract

“…6,12 As a next step in this evolving science, we propose testing key measures of diagnostic process safety grounded in the context in which care is delivered (often referred to as situated cognition, that is, one's thinking depends upon the environment in which one is contemplating a diagnosis). 13 Developing such metrics of the diagnostic process and of the work conditions that enable or hinder performance on these metrics will allow for the development of standard systems for assessment of diagnostic processes and supportive contexts of care; linking these to diagnostic quality and errors could provide healthcare systems the means to measure diagnostic accuracy and revise workflows and support systems to improve diagnosis. The ultimate result will be a step forward in our ability to design systems of care that will allow us to get the diagnoses right, even when time is short.…”

Section: Next Steps In the Chain Of Evidencementioning

confidence: 99%