Patient and Consumer Safety Risks When Using Conversational Assistants for Medical Information: An Observational Study of Siri, Alexa, and Google Assistant

Abstract

BackgroundConversational assistants, such as Siri, Alexa, and Google Assistant, are ubiquitous and are beginning to be used as portals for medical services. However, the potential safety issues of using conversational assistants for medical information by patients and consumers are not understood.ObjectiveTo determine the prevalence and nature of the harm that could result from patients or consumers using conversational assistants for medical information.MethodsParticipants were given medical problems to pose to Siri, Alexa, or Google Assistant, and asked to determine an action to take based on information from the system. Assignment of tasks and systems were randomized across participants, and participants queried the conversational assistants in their own words, making as many attempts as needed until they either reported an action to take or gave up. Participant-reported actions for each medical task were rated for patient harm using an Agency for Healthcare Research and Quality harm scale.ResultsFifty-four subjects completed the study with a mean age of 42 years (SD 18). Twenty-nine (54%) were female, 31 (57%) Caucasian, and 26 (50%) were college educated. Only 8 (15%) reported using a conversational assistant regularly, while 22 (41%) had never used one, and 24 (44%) had tried one “a few times.“ Forty-four (82%) used computers regularly. Subjects were only able to complete 168 (43%) of their 394 tasks. Of these, 49 (29%) reported actions that could have resulted in some degree of patient harm, including 27 (16%) that could have resulted in death.ConclusionsReliance on conversational assistants for actionable medical information represents a safety risk for patients and consumers. Patients should be cautioned to not use these technologies for answers to medical questions they intend to act on without further consultation from a health care provider.

At Your Service

“…They are more efficient than conventional screenbased interfaces, less error-prone [33], and can offer accessibility to users with disabilites [42]. However their unlimited range of possible inputs also facilitates a gulf between user expectations and actual capabilities [5,24].…”

“…The current landscape of commercial in-car voice assistants comprises Google's Assistant via Android Auto, Apple's Siri via CarPlay, Merceds-Benz' MBUX 4 , NIO NOMI 5 , BMW IPA 6 , and many more upcoming systems, e.g. by Ford, Toyota 7 , Honda, Hyundai, or Peugeot.…”

“…https://developer.amazon.com/alexa, last access: 2018-09-15 2 https://www.apple.com/siri/, last access: 2018-09-153 https://assistant.google.com, last access: 2018-09-154 https://www.mercedes-benz.com/a-class/com/en/mbux, 2018-09-155 https://www.nio.io/es8, last access: 2018-09-156 https://www.press.bmwgroup.com/global/article/detail/T0284429EN/ 7 https://www.toyota.com/concept-i, last access: 2018-09-158 https://soundhound.com/powered-by-houndify, 2018-[11][12][13][14][15][16][17][18][19][20][21][22][23][24][25][26] …”

Artificial intelligence in medical devices and clinical decision support systems

“…Many ML approaches and algorithms are in commercial use, and even more are in experimental trials. Recent examples that are well-publicized include self-driving cars (Google, Tesla, and Uber), voice-activated home assistance devices (Cortana, Alexa, Google Voice, and Siri), and autonomous robots that are being used in industrial, home, and healthcare settings (Bickmore, 2018).…”

A Context-Aware Conversational Agent in the Rehabilitation Domain

“…Finally, a comprehensive review on existing conversational agents for health-related purposes can be found in Reference [11]; the characteristics, current applications, and evaluation measures are laid bare regarding agents with unconstrained natural language-input capabilities. Nevertheless, certain concerns have been expressed over potential safety risks when patients are using virtual assistants as the sole source for medical information [12].…”