Differential Diagnosis Assessment in Ambulatory Care With an Automated Medical History–Taking Device: Pilot Randomized Controlled Trial

Abstract

Background Automated medical history–taking devices (AMHTDs) are emerging tools with the potential to increase the quality of medical consultations by providing physicians with an exhaustive, high-quality, standardized anamnesis and differential diagnosis. Objective This study aimed to assess the effectiveness of an AMHTD to obtain an accurate differential diagnosis in an outpatient service. Methods We conducted a pilot randomized controlled trial involving 59 patients presenting to an emergency outpatient unit and suffering from various conditions affecting the limbs, the back, and the chest wall. Resident physicians were randomized into 2 groups, one assisted by the AMHTD and one without access to the device. For each patient, physicians were asked to establish an exhaustive differential diagnosis based on the anamnesis and clinical examination. In the intervention group, residents read the AMHTD report before performing the anamnesis. In both the groups, a senior physician had to establish a differential diagnosis, considered as the gold standard, independent of the resident’s opinion and AMHTD report. Results A total of 29 patients were included in the intervention group and 30 in the control group. Differential diagnosis accuracy was higher in the intervention group (mean 75%, SD 26%) than in the control group (mean 59%, SD 31%; P=.01). Subgroup analysis showed a between-group difference of 3% (83% [17/21]-80% [14/17]) for low complexity cases (1-2 differential diagnoses possible) in favor of the AMHTD (P=.76), 31% (87% [13/15]-56% [18/33]) for intermediate complexity (3 differential diagnoses; P=.02), and 24% (63% [34/54]-39% [14/35]) for high complexity (4-5 differential diagnoses; P=.08). Physicians in the intervention group (mean 4.3, SD 2) had more years of clinical practice compared with the control group (mean 5.5, SD 2; P=.03). Differential diagnosis accuracy was negatively correlated to case complexity (r=0.41; P=.001) and the residents’ years of practice (r=0.04; P=.72). The AMHTD was able to determine 73% (SD 30%) of correct differential diagnoses. Patient satisfaction was good (4.3/5), and 26 of 29 patients (90%) considered that they were able to accurately describe their symptomatology. In 8 of 29 cases (28%), residents considered that the AMHTD helped to establish the differential diagnosis. Conclusions The AMHTD allowed physicians to make more accurate differential diagnoses, particularly in complex cases. This could be explained not only by the ability of the AMHTD to make the right diagnoses, but also by the exhaustive anamnesis provided.

Impact of a Commercial Artificial Intelligence–Driven Patient Self-Assessment Solution on Waiting Times at General Internal Medicine Outpatient Departments: Retrospective Study

“…Consequently, the implementation of automated medical history taking actually led to longer examination times. According to a previous study [ 18 ], physicians estimated that the use of an automated medical history–taking device has the potential to become time consuming in low-complexity cases, in which the medical history is easily taken. In the setting of the small- to medium-sized hospitals in Japan, case complexity is usually low for patients visiting general internal medicine outpatient departments without an appointment [ 26 , 27 ].…”

“…Its usability and acceptance by patients have been on the rise, and most patients (including older adults) can use automated medical history–taking devices without assistance [ 11 - 15 ]. Automated medical history taking is expected to assist physicians in developing differential diagnoses and to improve on accuracy of diagnoses, though this has not been the case previously [ 16 - 18 ]. Overall, computer-generated patient history recorded by an automated medical history–taking device was reported to be of higher quality, more comprehensive, better organized, and of greater relevance than patient information obtained through traditional methods of medical history taking [ 19 ].…”

Efficacy of Artificial-Intelligence-Driven Differential-Diagnosis List on the Diagnostic Accuracy of Physicians: An Open-Label Randomized Controlled Study

“…The quality of clinical documentation generated by an AI-driven AMHT system was reported to be as high as those of expert physicians [ 14 ]. AI-driven AMHT systems that also generate differential-diagnosis lists (so-called next-generation diagnosis-support systems [ 13 ]), were recently implemented in clinical practice [ 15 , 16 ]. A previous study reported that AI-driven AMHT systems with AI-driven differential-diagnosis lists could improve less-experienced physicians’ diagnostic accuracy in an ambulatory setting [ 16 ].…”

“…AI-driven AMHT systems that also generate differential-diagnosis lists (so-called next-generation diagnosis-support systems [ 13 ]), were recently implemented in clinical practice [ 15 , 16 ]. A previous study reported that AI-driven AMHT systems with AI-driven differential-diagnosis lists could improve less-experienced physicians’ diagnostic accuracy in an ambulatory setting [ 16 ]. Therefore, high-quality AI-driven AMHT systems that generate a differential-diagnosis list may be an option for improving physicians’ diagnostic accuracy.…”

“…The previous study, however, also suggested that the use of AI-driven AMHT systems may result in omission (when physicians reject a correct diagnosis suggested by AI) and commission (when physicians accept an incorrect diagnosis suggested by AI) errors [ 16 ], which are related to automation biases [ 17 , 18 , 19 ]. This means that AI-driven differential-diagnosis lists can sometimes negatively affect physicians’ diagnostic accuracy when using AI-driven AMHT systems.…”