Generating Informed Consent Documents Related to Blepharoplasty Using ChatGPT

Abstract

Purpose: This study aimed to demonstrate the performance of the popular artificial intelligence (AI) language model, Chat Generative Pre-trained Transformer (ChatGPT) (OpenAI, San Francisco, CA, U.S.A.), in generating the informed consent (IC) document of blepharoplasty. Methods: A total of 2 prompts were provided to ChatGPT to generate IC documents. Four board-certified plastic surgeons and 4 nonmedical staff members evaluated the AI-generated IC documents and the original IC document currently used in the clinical setting. They assessed these documents in terms of accuracy, informativeness, and accessibility. Results: Among board-certified plastic surgeons, the initial AI-generated IC document scored significantly lower than the original IC document in accuracy (p < 0.001), informativeness (p = 0.005), and accessibility (p = 0.021), while the revised AI-generated IC document scored lower compared with the original document in accuracy (p = 0.03) and accessibility (p = 0.021). Among nonmedical staff members, no statistical significance of 2 AI-generated IC documents was observed compared with the original document in terms of accuracy, informativeness, and accessibility. Conclusions: Our results showed that current ChatGPT cannot be used as a distinct patient education resource. However, it has the potential to make better IC documents when improving the professional terminology. This AI technology will eventually transform ophthalmic plastic surgery healthcare systematics by enhancing patient education and decision-making via IC documents.

Systematic Review of Large Language Models for Patient Care: Current Applications and Challenges

“…The authors were primarily affiliated with institutions in the United States (n=47 of 122 different countries identified per publication, 38.5%), followed by Germany (n=11/122, 9%), Turkey (n=7/122, 5.7%), the United Kingdom (n=6/122, 4.9%), China/Australia/Italy (n=5/122, 4.1%, respectively), and 24 (n=36/122, 29.5%) other countries. Most studies examined one or more applications based on the GPT-3.5 architecture (n=66 of 124 different LLMs examined per study, 53.2%) 13,26–29,31–34,36–40,42–49,52–54,56–61,63,65–67,71,72,74,75,77,78,81–89,91,92,94,95,97–100,102–104,106–109,111 , followed by GPT-4 (n=33/124, 26.6%) 13,25,27,29,30,34–36,41,43,50,51,54,55,58,61,64,68–70,74,76,79–81,83,87,89,90,93,96,98,99,101,105 , Bard (n=10/124, 8.1%; now known as Gemini) 33,48,49,55,73,74,80,87,94,99 , Bing Chat (n=7/124, 5.7%; now Microsoft Copilot) 49,51,55,73,94,99,110 , and other applications based on Bidirectional Encoder Representations from Transformers (BERT; n=4/124, 3...…”

“…Most studies examined one or more applications based on the GPT-3.5 architecture (n=66 of 124 different LLMs examined per study, 53.2%) 13,26–29,31–34,36–40,42–49,52–54,56–61,63,65–67,71,72,74,75,77,78,81–89,91,92,94,95,97–100,102–104,106–109,111 , followed by GPT-4 (n=33/124, 26.6%) 13,25,27,29,30,34–36,41,43,50,51,54,55,58,61,64,68–70,74,76,79–81,83,87,89,90,93,96,98,99,101,105 , Bard (n=10/124, 8.1%; now known as Gemini) 33,48,49,55,73,74,80,87,94,99 , Bing Chat (n=7/124, 5.7%; now Microsoft Copilot) 49,51,55,73,94,99,110 , and other applications based on Bidirectional Encoder Representations from Transformers (BERT; n=4/124, 3.2%) 13,83,84 , Large Language Model Meta-AI (LLaMA; n=3/124, 2.4%) 55 , or Claude by Anthropic (n=1/124, 0.8%) 55 . The majority of applications were p...…”

“…An overview of the presence of codes for each study is provided in Supplementary Section 3. The majority of articles investigated the use and feasibility of LLMs as medical chatbots (n=84/89, 94.4%) 13,24–62,64–66,68,69,71–96,98–111 , while fewer reports additionally or exclusively focused on the generation of patient information (n=19/89, 21.4%) 24,31,43,48,49,57,59,62,67,70,79,88–91,97,102,106,107 , including clinical documentation such as informed consent forms (n=5/89, 5.6%) 43,67,91,97,102 and discharge instructions (n=1/89, 1.1%) 31 , or translation/summarization tasks of medical texts (n=5/89, 5.6%) 24,49,57,79,89 , creation of patient education materials (n=5/89, 5.6%) 48,62,90,106,107 , and simplification of radiology reports (n=2/89, 2.3%) 59,88 . Most reports evaluated LLMs in English (n=88/89, 98.9%) 13,24–103,105–111 , followed by Arabic (n=2/84, 2.3%) 32,104 , Mandarin (n=2/84, 2.3%) 36,75 , and Korean or Spanish (n=1/89, 1.1%, respectively) 75 .…”

“…Most reports evaluated LLMs in English (n=88/89, 98.9%) 13,24–103,105–111 , followed by Arabic (n=2/84, 2.3%) 32,104 , Mandarin (n=2/84, 2.3%) 36,75 , and Korean or Spanish (n=1/89, 1.1%, respectively) 75 . The top-five specialties studied were ophthalmology (n=10/89, 11.2%) 37,40,48,51,65,74,97,98,100,101 , gastro-enterology (n=9/89, 10.1%) 25,32,34,36,39,61,62,72,96 , head and neck surgery/otolaryngology (n=8/89, 9%) 35,42,56,64,66,76,78,79 , and radiology 59,70,88–90,110 or plastic surgery 45,47,49,102,107,108 (n=6/89, 6.7%, respectively). A schematic illustration of the identified concepts of LLM applications in patient care is shown in Figure 2.…”

“…An overview of the presence of codes for each study is provided in Supplementary Section 3. The majority of articles investigated the use and feasibility of LLMs as medical chatbots (n=84/89, 94.4%) 13,[64][65][66]68,69,[98][99][100][101][102][103][104][105][106][107][108][109][110][111] , while fewer reports additionally or exclusively focused on the generation of patient information (n=19/89, 21.4%) 24,31,43,48,49,57,59,62,67,70,79,[88][89][90][91]97,102,106,107 , including clinical documentation such as informed consent forms (n=5/89, 5.6%) 43,67,91,97,102 and discharge instructions (n=1/89, 1.1%) 31 , or translation/summarization tasks of medical texts (n=5/89, 5.6%) 24,49,…”