Readability of spine-related patient education materials: a standard method for improvement

Baumann, John; Marshall, Samuel F.; Groneck, Andrew; Hanish, Stefan; Choma, Theodore J.; DeFroda, Steven F.

doi:10.1007/s00586-023-07856-5

Cited by 4 publications

(2 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is recommended that PEM readability should be at or below the 6th-grade level. However, studies consistently show that these recommendations are rarely met in PEMs related to orthopaedic diseases, conditions, treatments, and recovery (Abdullah et al, 2022;Baumann et al, 2023;Ghanem et al, 2023;Hanish et al, 2023;Stelzer et al, 2022;Sudah et al, 2023). Similar findings have consistently been shown in PEMs related to THA and TKA, which is particularly problematic as these surgeries are extremely common (Doinn et al, 2020;Karimi et al, 2023;Murphy et al, 2022;Shnaekel et al, 2018).…”

mentioning

confidence: 59%

“…To address this issue, organisations such as the AMA and NIH recommend that all PEMs should have readability at or below the sixth-grade reading level. However, PEMs within the field of orthopaedics have consistently been shown to rarely meet these recommendations, potentially having a negative impact on patient health literacy and outcomes (Abdullah et al, 2022;Baumann et al, 2023;Ghanem et al, 2023;Hanish et al, 2023;Stelzer et al, 2022;Sudah et al, 2023).…”

Section: F I G U R Ementioning

confidence: 99%

See 1 more Smart Citation

A standardised method for improving the readability of patient education materials for total hip & knee arthroplasty patients

Baumann,

Marshall,

O’Malley

et al. 2024

Musculoskeletal Care

View full text Add to dashboard Cite

PurposeOrthopaedic Patient Education Materials (PEMs) related to total hip and total knee arthroplasty (TKA) have repeatedly been shown to not meet American Medical Association and National Institutes of Health recommendations for readability. PEMs that are written with too much complexity limit the ability of some readers to comprehend the material. These complex PEMs also limit the health literacy of patients, a key determinant of health status and outcomes. The purpose of this study was to improve the readability of total hip and TKA‐related PEMs by limiting the use of sentences with ≥15 words and limiting the use of words with ≥ three syllables.MethodsThe readability of all 26 PEMs in this study was assessed before and after editing. Editing of articles included limiting the use of sentences with ≥15 words and limiting the use of words with ≥three syllables while preserving PEM content.ResultsA total of 26 PEMs were available for use in this study after the application of inclusion and exclusion criteria. The percentage of original PEMs at or below the recommended sixth‐grade reading level was 0% (0 out of 26), while the percentage of edited PEMs at or below the sixth‐grade reading level was 50% (13 out of 26).ConclusionsUsing this standardised method for reducing sentence length to <15 words and limiting the use of words with >2 syllables, while preserving key content, significantly improved the readability of PEMs related to total hip and TKA.

show abstract

mentioning

confidence: 59%

Section: F I G U R Ementioning

confidence: 99%

A standardised method for improving the readability of patient education materials for total hip & knee arthroplasty patients

Baumann,

Marshall,

O’Malley

et al. 2024

Musculoskeletal Care

View full text Add to dashboard Cite

show abstract

Evaluation of Generative Language Models in Personalizing Medical Information: Instrument Validation Study

Spina,

Andalib,

Flores

et al. 2024

JMIR AI

View full text Add to dashboard Cite

Background Although uncertainties exist regarding implementation, artificial intelligence–driven generative language models (GLMs) have enormous potential in medicine. Deployment of GLMs could improve patient comprehension of clinical texts and improve low health literacy. Objective The goal of this study is to evaluate the potential of ChatGPT-3.5 and GPT-4 to tailor the complexity of medical information to patient-specific input education level, which is crucial if it is to serve as a tool in addressing low health literacy. Methods Input templates related to 2 prevalent chronic diseases—type II diabetes and hypertension—were designed. Each clinical vignette was adjusted for hypothetical patient education levels to evaluate output personalization. To assess the success of a GLM (GPT-3.5 and GPT-4) in tailoring output writing, the readability of pre- and posttransformation outputs were quantified using the Flesch reading ease score (FKRE) and the Flesch-Kincaid grade level (FKGL). Results Responses (n=80) were generated using GPT-3.5 and GPT-4 across 2 clinical vignettes. For GPT-3.5, FKRE means were 57.75 (SD 4.75), 51.28 (SD 5.14), 32.28 (SD 4.52), and 28.31 (SD 5.22) for 6th grade, 8th grade, high school, and bachelor’s, respectively; FKGL mean scores were 9.08 (SD 0.90), 10.27 (SD 1.06), 13.4 (SD 0.80), and 13.74 (SD 1.18). GPT-3.5 only aligned with the prespecified education levels at the bachelor’s degree. Conversely, GPT-4’s FKRE mean scores were 74.54 (SD 2.6), 71.25 (SD 4.96), 47.61 (SD 6.13), and 13.71 (SD 5.77), with FKGL mean scores of 6.3 (SD 0.73), 6.7 (SD 1.11), 11.09 (SD 1.26), and 17.03 (SD 1.11) for the same respective education levels. GPT-4 met the target readability for all groups except the 6th-grade FKRE average. Both GLMs produced outputs with statistically significant differences (P<.001; 8th grade P<.001; high school P<.001; bachelors P=.003; FKGL: 6th grade P=.001; 8th grade P<.001; high school P<.001; bachelors P<.001) between mean FKRE and FKGL across input education levels. Conclusions GLMs can change the structure and readability of medical text outputs according to input-specified education. However, GLMs categorize input education designation into 3 broad tiers of output readability: easy (6th and 8th grade), medium (high school), and difficult (bachelor’s degree). This is the first result to suggest that there are broader boundaries in the success of GLMs in output text simplification. Future research must establish how GLMs can reliably personalize medical texts to prespecified education levels to enable a broader impact on health care literacy.

show abstract

Assessment of Readability, Understandability and Actionability of a Fully AI-generated Patient Information Brochure, a Cross-sectional Study (Preprint)

Mansour,

Zaghal,

Rahhal

et al. 2023

Preprint

View full text Add to dashboard Cite

BACKGROUND A gap exists between the readability of readily available patient education materials and the literacy skill of the average US adult. With the current advancements in artificial intelligence (AI), AI tools could provide the solution. OBJECTIVE To evaluate the readability, understandability and actionability of patient information brochures fully generated by artificial intelligence. METHODS Patient information brochures about the three most common surgical pathologies (appendicitis, cholecystitis, inguinal hernia) were generated by ChatGPT-4 and Bing Image Creator and subsequently evaluated for readability using readability scores and for understandability and actionability by surgeons using Patient Educational Materials Assessment for Printable materials (PEMAT-P). RESULTS Brochures generated by ChatGPT and DALL-E scored well on text understandability (93.6%) but poorly on visual aids (18.67%) and actionability (64%). The mean Flesch reading ease was 41.54, Flesch-Kincaid grade level: 10.52, Gunning Fog Index: 11.98, SMOG Index: 12.78, Coleman Liau Index: 13.3 and Automated Readability Index: 9.98. CONCLUSIONS Patient information brochures generated by ChatGPT and Bing Image Creator had high text understandability, low visual aids understandability and low actionability. The readability levels of the generated content exceed the average literacy level of the US adult population.

show abstract

Readability of spine-related patient education materials: a standard method for improvement

Cited by 4 publications

References 21 publications

A standardised method for improving the readability of patient education materials for total hip & knee arthroplasty patients

A standardised method for improving the readability of patient education materials for total hip & knee arthroplasty patients

Evaluation of Generative Language Models in Personalizing Medical Information: Instrument Validation Study

Assessment of Readability, Understandability and Actionability of a Fully AI-generated Patient Information Brochure, a Cross-sectional Study (Preprint)

Contact Info

Product

Resources

About