Quality and Readability of Online Patient Information on Adolescent Bariatric Surgery

Rouhi, Armaun D.; Ghanem, Yazid K; Hoeltzel, Gerard D.; Yi, William; Collins, Joy L.; Prout, Elizabeth Parks; Williams, Noel N.; Dumon, Kristoffel R.

doi:10.1007/s11695-022-06385-2

Cited by 5 publications

(2 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Notably, google searches of vasectomy have significantly increased since the Dobbs vs. Jackson ruling in the United States [ 17 ]. Ease of reading is essential for the accessibility of health information, yet several studies have shown the current reading level of online health information to be too complex for an average person living in America [ 18 19 20 21 ]. Unfortunately, our study demonstrates this applies to online vasectomy content as well.…”

Section: Discussionmentioning

confidence: 99%

Patient-Directed Vasectomy Information: How Readable Is It?

Kianian,

Hu,

Lavold

et al. 2024

World J Mens Health

View full text Add to dashboard Cite

Purpose To assess the quality and readability of online health information on vasectomy using validated readability and quality assessment tools. Materials and Methods The top 50 search results for "vasectomy" on Google, Bing, and Yahoo were selected. Duplicate links, advertisements, blog posts, paid webpages, and information intended for healthcare providers were excluded. Flesch Reading Ease score, Flesch–Kincaid Grade level, Gunning Fog Index, and Simple Measure of Gobbledygook (SMOG) index were used to assess readability, with optimal readability level for online health information established as being at sixth grade reading level. DISCERN Instrument and JAMA Benchmark were used to assess the quality of selected webpages. Inter-assessment score correlation and results by webpage type were analyzed. Results We analyzed 44 webpages, including 16 academic, 5 hospital-affiliated, 6 commercial, 13 non-profit health advocacy, and 4 uncategorized sources. The average readability of the evaluated webpages was at a 10th grade reading level as measured by the Flesch Kincaid Assessment tool, and an undergraduate reading level per the SMOG and Gunning Fog indices. Non-profit health advocacy webpages had the best reading level but still was not at the recommended level of grade 6 to 7. The overall DISCERN quality of the webpages was “fair”, with non-profit health advocacy pages performing best. Conclusions The assessed webpages offer education on vasectomy in a language that is too complex for the general population to understand. Furthermore, several sources for online health information, such as non-profits, outperformed webpages by academic institutions. Increased healthcare collaboration and dedication to producing quality online patient resources is necessary to address these shortcomings and build trust among patients to increase utilization of vasectomy and decrease decisional regret.

show abstract

Section: Discussionmentioning

confidence: 99%

Patient-Directed Vasectomy Information: How Readable Is It?

Kianian,

Hu,

Lavold

et al. 2024

World J Mens Health

View full text Add to dashboard Cite

show abstract

“…They have also been proposed for use in improving existing patient education materials [4]. Currently available online medical information on a multitude of surgical diseases and procedures has repeatedly been shown to be of variable quality and difficult readability for the public [5][6][7][8][9]. The American Medical Association (AMA) and National Institutes of Health (NIH) recommend that online medical information be written at or below the 6th-grade reading level [10,11].…”

mentioning

confidence: 99%

Dr. Google to Dr. ChatGPT: assessing the content and quality of artificial intelligence-generated medical information on appendicitis

Ghanem,

Rouhi,

Al-Houssan

et al. 2024

Surg Endosc

Self Cite

View full text Add to dashboard Cite

Introduction Generative artificial intelligence (AI) chatbots have recently been posited as potential sources of online medical information for patients making medical decisions. Existing online patient-oriented medical information has repeatedly been shown to be of variable quality and difficult readability. Therefore, we sought to evaluate the content and quality of AI-generated medical information on acute appendicitis. Methods A modified DISCERN assessment tool, comprising 16 distinct criteria each scored on a 5-point Likert scale (score range 16–80), was used to assess AI-generated content. Readability was determined using the Flesch Reading Ease (FRE) and Flesch-Kincaid Grade Level (FKGL) scores. Four popular chatbots, ChatGPT-3.5 and ChatGPT-4, Bard, and Claude-2, were prompted to generate medical information about appendicitis. Three investigators independently scored the generated texts blinded to the identity of the AI platforms. Results ChatGPT-3.5, ChatGPT-4, Bard, and Claude-2 had overall mean (SD) quality scores of 60.7 (1.2), 62.0 (1.0), 62.3 (1.2), and 51.3 (2.3), respectively, on a scale of 16–80. Inter-rater reliability was 0.81, 0.75, 0.81, and 0.72, respectively, indicating substantial agreement. Claude-2 demonstrated a significantly lower mean quality score compared to ChatGPT-4 (p = 0.001), ChatGPT-3.5 (p = 0.005), and Bard (p = 0.001). Bard was the only AI platform that listed verifiable sources, while Claude-2 provided fabricated sources. All chatbots except for Claude-2 advised readers to consult a physician if experiencing symptoms. Regarding readability, FKGL and FRE scores of ChatGPT-3.5, ChatGPT-4, Bard, and Claude-2 were 14.6 and 23.8, 11.9 and 33.9, 8.6 and 52.8, 11.0 and 36.6, respectively, indicating difficulty readability at a college reading skill level. Conclusion AI-generated medical information on appendicitis scored favorably upon quality assessment, but most either fabricated sources or did not provide any altogether. Additionally, overall readability far exceeded recommended levels for the public. Generative AI platforms demonstrate measured potential for patient education and engagement about appendicitis.

show abstract