Clinical Advice by Voice Assistants on Postpartum Depression: Cross-Sectional Investigation Using Apple Siri, Amazon Alexa, Google Assistant, and Microsoft Cortana (Preprint)

Abstract

BACKGROUND A voice assistant (VA) is an inanimate audio-interfaced software, augmented with artificial intelligence, capable of two-way dialogue, and increasingly used to access healthcare advice. Postpartum depression (PPD) is a prime candidate for a VA-based digital health intervention. OBJECTIVE To assess VA responses to PPD questions in terms of accuracy, advice given, and clinical appropriateness. METHODS This cross-sectional study examined four VAs installed on two mobile devices in early 2020. We posed 14 questions obtained from the American College of Obstetricians and Gynecologists’ (ACOG) patient-focused Frequently Asked Questions (FAQ) on PPD to each VA. We scored the VA responses according to accuracy of speech recognition, whether advice was given, and clinical agreement with ACOG FAQ. Two board-certified physicians assessed responses given by four major VAs (Apple Siri, Amazon Alexa, Google Assistant, and Microsoft Cortana). The three assessments for each VA response included accurate recognition of the query, presence of advice given, and clinical agreement with the ACOG FAQ. RESULTS Accurate recognition of the query ranged from 78.5% to 100%. Advice given ranged from 35.7% to 78.6%. Clinical agreement with the ACOG FAQ ranged from 14.3% to 28.6%. CONCLUSIONS While the best performing VA gave clinically appropriate advice to 28.6% of the PPD questions, all four VAs taken together achieved 64.3% clinically relevant advice. Technology companies and clinical organizations should partner to improve guidance, screen patients for mental health disorders, and educate patients on potential treatment.