Investigating conversational agents in healthcare: Application of a technical-oriented taxonomy

Denecke, Kerstin; May, Richard

doi:10.1016/j.procs.2023.01.413

Cited by 5 publications

(5 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is in line with the research published on intelligent agents and systems with conversational user interfaces. Although many of the existing agents described are rule based [ 38 ], reports on LLM-based systems are emerging [ 39 ]. Other clinical tasks with which LLMs could help include triage and diagnostic tasks as well as risk assessment and treatment plan generation.…”

Section: Discussionmentioning

confidence: 99%

Potential of Large Language Models in Health Care: Delphi Study

Denecke,

May,

Rivera Romero

2024

J Med Internet Res

Self Cite

View full text Add to dashboard Cite

Background A large language model (LLM) is a machine learning model inferred from text data that captures subtle patterns of language use in context. Modern LLMs are based on neural network architectures that incorporate transformer methods. They allow the model to relate words together through attention to multiple words in a text sequence. LLMs have been shown to be highly effective for a range of tasks in natural language processing (NLP), including classification and information extraction tasks and generative applications. Objective The aim of this adapted Delphi study was to collect researchers’ opinions on how LLMs might influence health care and on the strengths, weaknesses, opportunities, and threats of LLM use in health care. Methods We invited researchers in the fields of health informatics, nursing informatics, and medical NLP to share their opinions on LLM use in health care. We started the first round with open questions based on our strengths, weaknesses, opportunities, and threats framework. In the second and third round, the participants scored these items. Results The first, second, and third rounds had 28, 23, and 21 participants, respectively. Almost all participants (26/28, 93% in round 1 and 20/21, 95% in round 3) were affiliated with academic institutions. Agreement was reached on 103 items related to use cases, benefits, risks, reliability, adoption aspects, and the future of LLMs in health care. Participants offered several use cases, including supporting clinical tasks, documentation tasks, and medical research and education, and agreed that LLM-based systems will act as health assistants for patient education. The agreed-upon benefits included increased efficiency in data handling and extraction, improved automation of processes, improved quality of health care services and overall health outcomes, provision of personalized care, accelerated diagnosis and treatment processes, and improved interaction between patients and health care professionals. In total, 5 risks to health care in general were identified: cybersecurity breaches, the potential for patient misinformation, ethical concerns, the likelihood of biased decision-making, and the risk associated with inaccurate communication. Overconfidence in LLM-based systems was recognized as a risk to the medical profession. The 6 agreed-upon privacy risks included the use of unregulated cloud services that compromise data security, exposure of sensitive patient data, breaches of confidentiality, fraudulent use of information, vulnerabilities in data storage and communication, and inappropriate access or use of patient data. Conclusions Future research related to LLMs should not only focus on testing their possibilities for NLP-related tasks but also consider the workflows the models could contribute to and the requirements regarding quality, integration, and regulations needed for successful implementation in practice.

show abstract

Section: Discussionmentioning

confidence: 99%

Potential of Large Language Models in Health Care: Delphi Study

Denecke,

May,

Rivera Romero

2024

J Med Internet Res

Self Cite

View full text Add to dashboard Cite

show abstract

“…and it also emphasizes the integration of emotion management in chatbots, highlighting how machine learning technologies can improve interaction and overall user experience [137], which could appropriately be examples of key studies. [9], [12], [17], [178]- [238], [311] 2023 71 [11], [10], [18], [239]- [306] In this case study, we have made a deliberate effort to introduce novelty to the existing literature review by incorporating references primarily from the years 2022 and 2023. This strategic decision was prompted by the shifting societal and research landscape following the unprecedented COVID-19 pandemic.…”

Section: Literature Search and Selectionmentioning

confidence: 99%

“…Chatbots have the ability to perform health diagnoses and provide basic information before patients contact doctors [9]. For example, if someone wants to quit smoking, they can interact with a bot [253], enabling virtual consultations with patients and providing therapies that conform to medical guidelines [255] although it still requires confirmation from a doctor.…”

Section: Chatbot For Healthcarementioning

confidence: 99%

See 1 more Smart Citation

Evolving Conversations: A Review of Chatbots and Implications in Natural Language Processing for Cultural Heritage Ecosystems

Suryanto,

Wibawa,

Hariyono

et al. 2023

IJRCS

View full text Add to dashboard Cite

Chatbot technology, a rapidly growing field, uses Natural Language Processing (NLP) methodologies to create conversational AI bots. Contextual understanding is essential for chatbots to provide meaningful interactions. Still, to date chatbots often struggle to accurately interpret user input due to the complexity of natural language and diverse fields, hence the need for a Systematic Literature Review (SLR) to investigate the motivation behind the creation of chatbots, their development procedures and methods, notable achievements, challenges and emerging trends. Through the application of the PRISMA method, this paper contributes to revealing the rapid and dynamic progress in chatbot technology with NLP learning models, enabling sophisticated and human-like interactions on the trends observed in chatbots over the past decade. The results, from various fields such as healthcare, organization and business, virtual personalities, to education, do not rule out the possibility of being developed in other fields such as chatbots for cultural preservation while suggesting the need for supervision in the aspects of language comprehension bias and ethics of chatbot users. In the end, the insights gained from SLR have the potential to contribute significantly to the advancement of chatbots on NLP as a comprehensive field.

show abstract

“…The result was a novel classification of health care CAs, consisting of 13 categories describing the CA appearance, communication modalities, and uses in health care settings. Denecke and May [44] recently published a technical-oriented taxonomy for health care CAs aimed at improving the reporting of the technical aspects of CA development. The authors included 18 categories grouped into 4 dimensions (agent appearance, setting, interaction, and data processing).…”

Section: Principal Findingsmentioning

confidence: 99%

Conversational Agents in Health Care: Expert Interviews to Inform the Definition, Classification, and Conceptual Framework

Martinengo,

Lin,

Jabir

et al. 2023

J Med Internet Res

View full text Add to dashboard Cite

Background Conversational agents (CAs), or chatbots, are computer programs that simulate conversations with humans. The use of CAs in health care settings is recent and rapidly increasing, which often translates to poor reporting of the CA development and evaluation processes and unreliable research findings. We developed and published a conceptual framework, designing, developing, evaluating, and implementing a smartphone-delivered, rule-based conversational agent (DISCOVER), consisting of 3 iterative stages of CA design, development, and evaluation and implementation, complemented by 2 cross-cutting themes (user-centered design and data privacy and security). Objective This study aims to perform in-depth, semistructured interviews with multidisciplinary experts in health care CAs to share their views on the definition and classification of health care CAs and evaluate and validate the DISCOVER conceptual framework. Methods We conducted one-on-one semistructured interviews via Zoom (Zoom Video Communications) with 12 multidisciplinary CA experts using an interview guide based on our framework. The interviews were audio recorded, transcribed by the research team, and analyzed using thematic analysis. Results Following participants’ input, we defined CAs as digital interfaces that use natural language to engage in a synchronous dialogue using ≥1 communication modality, such as text, voice, images, or video. CAs were classified by 13 categories: response generation method, input and output modalities, CA purpose, deployment platform, CA development modality, appearance, length of interaction, type of CA-user interaction, dialogue initiation, communication style, CA personality, human support, and type of health care intervention. Experts considered that the conceptual framework could be adapted for artificial intelligence–based CAs. However, despite recent advances in artificial intelligence, including large language models, the technology is not able to ensure safety and reliability in health care settings. Finally, aligned with participants’ feedback, we present an updated iteration of the conceptual framework for health care conversational agents (CHAT) with key considerations for CA design, development, and evaluation and implementation, complemented by 3 cross-cutting themes: ethics, user involvement, and data privacy and security. Conclusions We present an expanded, validated CHAT and aim at guiding researchers from a variety of backgrounds and with different levels of expertise in the design, development, and evaluation and implementation of rule-based CAs in health care settings.

show abstract

Investigating conversational agents in healthcare: Application of a technical-oriented taxonomy

Cited by 5 publications

References 23 publications

Potential of Large Language Models in Health Care: Delphi Study

Potential of Large Language Models in Health Care: Delphi Study

Evolving Conversations: A Review of Chatbots and Implications in Natural Language Processing for Cultural Heritage Ecosystems

Conversational Agents in Health Care: Expert Interviews to Inform the Definition, Classification, and Conceptual Framework

Contact Info

Product

Resources

About